Vincent Abbott | Deep Learning @vtabbott_
Maker of *those* diagrams for deep learning algorithms | 🇦🇺 | https://t.co/kE2BIGCiMY vtabbott.io Perth, Western Australia Joined July 2022-
Tweets321
-
Followers3K
-
Following200
-
Likes959
This is so awesome !! Link to get a poster at -> b3bcc7-e5.myshopify.com
This is so awesome !! Link to get a poster at -> b3bcc7-e5.myshopify.com
Reminder to always give networks the identity as an option. arxiv.org/abs/1603.05027
Reminder to always give networks the identity as an option. arxiv.org/abs/1603.05027
Did I forget to mention I'm working on neural circuit diagram posters? Just received one in the mail and am super happy with the print quality!!! U$25 for a 24x32" / 60x80cm poster at the link 👇 b3bcc7-e5.myshopify.com
Haven't posted in the last week because I've been working on three papers due soon. With any luck I'll get them finished in time 🤣 Neural circuit diagrams are based on robust category theory - and I'll be going into the rigorous mathematics behind them. I'll be proving the…
Reminder that I made a brief video that covers how to read Neural Circuit Diagrams as part of my @TmlrOrg publication! I cover the problem they address and the data / operations with which diagrams correspond!
I've just made a Jupyter notebook guide to Implementing Mixtral with Neural Circuit Diagrams! I include diagrams to explain what's happening. This guide won't give a model you can run, that'll come later, so follow to stay updated! github.com/vtabbott/Neura…
@vtabbott_ The current best hypothesis around transformers is that FF Is the actual place where facts like Obama is the 44th President of the US is stored. Attention on the other hand is where the knowledge scattered around multiple layers in transformers, attached to different tokens,…
The major improvement on AlexNet (2012) in 2013 and 2014 was going from 5x5 convolution layers to a larger number of 3x3 convolution layers. What would a similar improvement for 2024 look like? Improving a Deep Learning Architecture Idea: Halve the inner dimension and double…
It's ironic that >95% of the weights in transformers are in the feed forward layers rather than attention layers. In Mixtral, every ff has 1.3B parameters and every MQA has 40M parameters. Besides windowing, optimising the attention performance is really just garnish.
Goated Python package
I need to know which G20 countries have a smaller economy than the Waymo catchment.
I need to know which G20 countries have a smaller economy than the Waymo catchment.
Excellent work from @danielhanchen! One of the goals of my diagrams is to allow for models to be exactly expressed. It's really hard to check specific details like approximates etc. without a clear blueprint. Code fails as a reliable blueprint because of issues like this where…
Excellent work from @danielhanchen! One of the goals of my diagrams is to allow for models to be exactly expressed. It's really hard to check specific details like approximates etc. without a clear blueprint. Code fails as a reliable blueprint because of issues like this where…
Nicolás Visca @nicolasmvisca
2 Followers 109 Following
Omipop @Omnipip
46 Followers 67 Following
ROSguy @roboot_mobile
8 Followers 221 Following
Vaish @svaish610
242 Followers 369 Following my chai , broken humour and 2 hrs of sleep against the world // mildly cringe
Sarmad Imran @AeonBrain
6 Followers 28 Following Deep learning engineer in computer vision & disease classification. Specializing in RNN & CNN models. Passionate about AI in healthcare.
Daniel Vayalil @danielson2002
35 Followers 1K Following studying at Seattle University, '24 BS in CS Major
Dhayalan P @p18937
0 Followers 16 Following
Celia Lathem @CeliaLathe3575
78 Followers 5K Following
Fikadu Mulugeta @fikemulugeta
88 Followers 526 Following #BiomedicalEngineer #Medicalimaging #MRI #MachineLearning #AI #Scholarships "Normal is an illusion, what’s normal for a spider might be chaos for a fly"
Dr AI @TheAIScholar
0 Followers 59 Following دكتوراه في الذكاء الاصطناعي - أقوم بتدريس خوارزميات التعلم العميق لطلاب الماجستير بجامعة بريطانية - أرى أن تبسيط الذكاء الاصطناعي واجب أخلاقي للعلماء العرب
Lura Penzel @lur_penz
80 Followers 5K Following
Delia Mitschelen @DMitschele91059
105 Followers 5K Following
siddhant-0707 @Ramb00707
4 Followers 87 Following
$aid dazz @said_dazz
1 Followers 53 Following
Swgam @Swgam12
109 Followers 1K Following
nikolei776 @nikolei776
24 Followers 46 Following A passionate clash of clans esports critic with a Journalistic background designed to hold all teams accountable under the Motto say it like it is.
Lainey Dutremble @l_dutrem
41 Followers 5K Following
John @DeepAlphaByte
0 Followers 681 Following
Eloise Lutkins @eloi_lutki
35 Followers 5K Following
tom @0xluciusv
145 Followers 452 Following i like cuda kernels, c++, rust, go, and nvim. (cons e/λx.x 🌎/acc) wrong about a lot of things but trying to learn
Milena Vance @milena_van9380
83 Followers 5K Following
钟辉 @zhnghu008049148
9 Followers 333 Following
luolu @luolu823902886
82 Followers 1K Following
Simon Lee @SimonLee79475
40 Followers 104 Following @CompMedUCLA Ph.D Student | LLMs, Ai in Healthcare | Overthinker | Cat Enthusiast
Jhahahaha @Jhahahaha1121
12 Followers 49 Following
Moein Shariatnia @MoeinShariatnia
828 Followers 2K Following ML Dev & Researcher | 🩺 Med Student at TUMS | CV and NLP | Writer @YuanCommunity and @Medium | Freelancer AI dev
Marcus Brubaker @marcusabrubaker
2K Followers 1K Following Assoc Prof @YorkUniversity Affiliate @VectorInst Status @UofTCompSci Cofounder @StructuraBio Visiting Prof @SamsungResearch Toronto Advisor @BorealisAI He/Him
Jeff Barr ☁️ (@ �.. @jeffbarr
231K Followers 13K Following Chief Evangelist @Amazon Web Services: follow me for AWS updates & chatter. Father of 5, grandfather of 6. Author, Maker, UW MCDM. @[email protected]
Martin Treiber @martintreiber
260 Followers 938 Following Hello Artificial Intelligence! I here to help you to discover, understand and successfully deploy AI solutions.
Towanda Danko @towa_danko
75 Followers 5K Following
Billie Bensberg @b_bensbe
45 Followers 5K Following
Niharika Jain @nj180903
0 Followers 157 Following
张洋 @touwenameng
21 Followers 1K Following
Kuda Madzima @KudaMadzima
88 Followers 742 Following
Melani @melanimaheswar1
41 Followers 441 Following LLM fan girl | prev data quality @cohere | optimize for knowledge | Toronto | 🇨🇦
Pradeep @prads
208 Followers 1K Following
James Nichols @james_nichols
570 Followers 1K Following Mathematics Enjoyer @ Australian National University
Vamsi Nimmala @vamsi_nimmala
807 Followers 4K Following Sr Al/ML Engineer | Systems Thinker | AI & NLP | OSS | TruthGPT🇮🇳🇺🇸 |Abundance ∞| Critical Engineering Mindset | Techno Humanist | ~ No labels
⿻ barton 🦺𑗊 @bmorphism
2K Followers 4K Following applied categorical duck cyberneticist • building for agencies in the 21st century • inventor of the operadic cognitive diagram cognitive continuation standard
Zachary Burkett @zburkett
107 Followers 445 Following you can happen to the world - building automatons - @codingscape
James Nichols @james_nichols
570 Followers 1K Following Mathematics Enjoyer @ Australian National University
François Fleuret @francoisfleuret
31K Followers 456 Following Prof. @Unige_en, Adjunct Prof. @EPFL_en, Research Fellow @idiap_ch, co-founder @nc_shape. AI and machine learning since 1994. I like reality.
CyberCat Institute @CyberCatInst
1K Followers 101 Following Institute for Categorical Cybernetics - applied category theory for social good in economics, machine learning and software engineering
Vincent Wang-Maścian.. @vinnylarouge
480 Followers 350 Following CS Dr. @ Oxford, aspirant friend of diagrams
Osmo @Osmo_Labs
659 Followers 43 Following Osmo is giving computers a sense of smell to improve the health and wellbeing of human life.
Daniel Han @danielhanchen
7K Followers 935 Following Building @UnslothAI. Finetune LLMs 30x faster https://t.co/aRyAAgKOR7. Prev ML at NVIDIA. Hyperlearn used by NASA. I like maths, making code go fast
googledownunder @googledownunder
74K Followers 590 Following Welcome to the official Google Australia and New Zealand handle, posting tweets to update, inform and delight.
James Dingley @AtomicFrontiers
920 Followers 50 Following Engineering the world one video at a time.
Zanzi Tangle, now at .. @tangled_zans
3K Followers 320 Following Turning Category Theory into code https://t.co/B0egvR0lmb
Neal Wu @WuNeal
15K Followers 390 Following Building @cognition_labs. Previously @tryramp, @GoogleBrain, @Harvard, competitive programming (featured in @Wired). Created https://t.co/pihw5AGvbV.
Nathan Labenz @labenz
14K Followers 2K Following AI Scout, building text-2-video @Waymark, host of The Cognitive Revolution podcast
Akshay 🚀 @akshay_pachaar
136K Followers 417 Following Simplifying LLMs, MLOps, Python & Machine Learning for you! • AI Engineering @LightningAI • Lead DataScientist • BITS Pilani • 3 Patents
XY Han @XYHan_
1K Followers 979 Following Incoming Assistant Professor @ChicagoBooth | Postdoc @Stanford | Papers: “Neural Collapse in Deep Nets” & “Survey Descent: Nonsmooth Gen. of GD”
Rohan Paul @rohanpaul_ai
12K Followers 840 Following ML Engineer (e/acc) 📌 https://t.co/x0IIWfnOt8 🚀 https://t.co/QEO4CKRl1b Open LLMs is Happiness 💡 Ex Deutsche & HSBC. DM for collaboration.
DeepSpeed @MSFTDeepSpeed
3K Followers 88 Following Official account for @Microsoft DeepSpeed, a library that enables unprecedented scale and speed for deep learning training + inference. 日本語 : @MSFTDeepSpeedJP
mobicham @mobicham
66 Followers 22 Following I like to shrink dem models 🤏 Co-Founder & Principal Scientist @mobius_labs PhD @inria
Mobius Labs @Mobius_Labs
3K Followers 105 Following Multimodal AI for the world's scale. Proponents of Open Source and Open Intelligence. https://t.co/1nC6r8hOrE for some of our recent work.
Tom Sercu @TomSercu
2K Followers 694 Following building something new and hiring. Ex-Meta FAIR, Ex-IBM Research. Alum @NYU, @ugent.
George Hotz 🌑 @realGeorgeHotz
248K Followers 174 Following President @comma_ai. Founder @__tinygrad__
the tiny corp @__tinygrad__
33K Followers 63 Following We make tinygrad. Our mission is to commoditize the petaflop.
Julian Harris @julianharris
2K Followers 4K Following 30 years making internet software ex-Googler & ex-founder and wannabe ML engineer solving climate problems demanding greater-than-human-scale solutions with AI.
kache (dingboard.com) @yacineMTB
53K Followers 3K Following i'm a swe. go to https://t.co/pWRBfY8kn2 - AI image editing IN YOUR BROWSER! follow to watch a self funded founder beat VC backed AI startups with @dingboard_
Vi @AvimanyuRoy3
578 Followers 2K Following 🍎🕊/🦦☕️/😴🛌/he/him Shouting into the Void (TM) GPU poor peasant
Eric Xu 🇺🇦 @xleaps
33K Followers 3K Following polymath, polyglot, root of a ternary tree. Sr Director of AI @HubSpot 🧑💻 prev @Meta @Google @Reddit 三脚猫 martial artist 🥋 Rookie pilot 🛩️
Rogerio Sampaio @rsalmei
42 Followers 108 Following
Codetard @codetarded
60 Followers 99 Following Learning to code at the end of history Idiot-proofing consultant. If I can figure it out, anyone can.
Ben (e/sqlite) @andersonbcdefg
3K Followers 3K Following 🤖 Computer scientist, next-word-prediction enjoyer 📊 Prev. research fellow @ Stanford RegLab 🛠️ bUiLdiNg sOmeThiNg nEw (https://t.co/mdYPZmjSzN - YC S23) 🏳️🌈
Horace He @cHHillee
23K Followers 449 Following Working at the intersection of ML and Systems @ PyTorch "My learning style is Horace twitter threads" - @typedfemale
Albert Jiang @AlbertQJiang
2K Followers 408 Following AI4Maths @Cambridge_CL Science @MistralAI I bake my own opinions at temperature=2.0
SMA 🏴☠️ @generic_void
8K Followers 2K Following dark empress. e/acc. shapecelword-rotator. transhumanist. {Research} ⊃ {Post-Scarcity Econ, Game/Info/Network Theory, ML/DL & Econometrics, ..., N}
Charlie O'Neill @charles0neill
344 Followers 1K Following Maths + Comp Sci + Economics @ ANU. Using mech interp to build hierarchical planning modules into transformers
David Jurgens @david__jurgens
2K Followers 558 Following Associate Professor at @UMSI and @UMichCSE working in computational social science and NLP. PI of the Blablablab https://t.co/pt1UFJuBiU
Jake Lee @jakehlee1
77 Followers 179 Following Data Scientist at NASA/Caltech JPL, helping scientists science better with ML/AI. Columbia CS '19/'20, FIRST robotics coach, avid maker & tinkerer.
Lysandre @LysandreJik
7K Followers 582 Following Head of Open-Source at Hugging Face. Maintainer of 🤗/Transformers. I tweet about Open Source. He/him
Andrew Carr (e/🤸) @andrew_n_carr
15K Followers 3K Following science @getcartwheel AI writer @tldrnewsletter advisor @arcade_ai Past - Codegen @OpenAI, Brain @GoogleAI, world ranked Tetris player
Abhishek Cauligi @ACauligi
238 Followers 559 Following Robotics technologist at NASA Jet Propulsion Lab. PhD from @StanfordASL.
Tomas Pueyo @tomaspueyo
334K Followers 548 Following Understand deeply how the world works today to navigate the world of tomorrow. Join 80k ppl in my free newsletter:
clem 🤗 @ClementDelangue
91K Followers 5K Following Co-founder & CEO @HuggingFace 🤗, the open and collaborative platform for AI builders
Stability AI Japan @StabilityAI_JP
38K Followers 98 Following 私たちは人類の可能性を広げるための基盤を構築しています。 We are building the foundation to activate humanity's potential. #StableDiffusion #StableLM #StableVideo #StableAudio
Paul Atherton @PaulAtherton13
2K Followers 1K Following Restless thinker trying new ways to solve old problems. Purveyor of all things Fab and https://t.co/AvcagTbYSp With some WAFC and pies on the side.
jack morris @jxmnop
10K Followers 761 Following getting my phd in nlp @cornell_tech 🚠 // academic optimist // tweeting from the snack aisle at trader joes
Eric Auld @AuldEric
311 Followers 688 Following AI, math, CS. Former @uclamath. I’ll let you be in my dream if I can be in yours
Diagramming AI @Diagrammin71888
403 Followers 230 Following
Ankush Singal @andysingal
550 Followers 255 Following https://t.co/AEA5PtXqjc I am a traveller, photographer and Data Scientist enthusiast Gumroad content: https://t.co/if9OmgcRBsThere is a certain sadness in beautifully optimizing a piece of PyTorch while knowing it will never come even close to an adequate cuda kernel.
Conquering the world, one reader at a time.
common dark thought pattern in research > run baseline experiment > change thing A > also change thing B > run new experiment > collect results > "wow, thing A works!"
New blog post! Build Your Own Open Games Engine Bootcamp, Part I: Lenses - by Daniele Palombi @dpl0a (cross-posted from the 20[ ] blog) cybercat.institute/2024/04/22/ope…
@danielsateler1 YES. I'm so bothered by this always, it causes me suffering to wait for my program to start. Computers are FAST. They have dozens of fancy cores capable of billions of instructions per second and a perfected memory hierarchy. What is even happening? I categorically refuse to wait…
🔥llm.c update: Our single file of 2,000 ~clean lines of C/CUDA code now trains GPT-2 (124M) on GPU at speeds ~matching PyTorch (fp32, no flash attention) github.com/karpathy/llm.c… On my A100 I'm seeing 78ms/iter for llm.c and 80ms/iter for PyTorch. Keeping in mind this is fp32,…
@dejavucoder For Mixtral-8x7b, check this out. Need more of this kind of diagrams. x.com/vtabbott_/stat…
I've just made a Jupyter notebook guide to Implementing Mixtral with Neural Circuit Diagrams! I include diagrams to explain what's happening. This guide won't give a model you can run, that'll come later, so follow to stay updated! github.com/vtabbott/Neura…
You might remember my "category theory notes". The book arises from them, with massive updates, and a new chapter on monoidal categories.
My book is out! worldscientific.com/worldscibooks/…
Are you transformermaxxing anon? (Thanks @vtabbott_ keep making cool stuff)
Mistral 8x22B Instruct got released! Absolutely wild! I added an extra row to the table from their blog post :)
Mixtral 8x22B sets a new standard for performance and efficiency for the AI community. Apache 2.0. mistral.ai/news/mixtral-8…
Damn straight! Mistral just dropped the Mistral 8x22B Instruct weights 🔥 > 90.8% on GSM8K maj@8 > 44.6% on math maj@4 Also Mistral throwing shade on Cohere lol
Diagrams of composing compilers look like the Para construction diagrams
Oh this is really cool! I was wondering if there exists a suitable graphical language to talk about stuff like this
I do remember thinking a while back that a compiler from X -> Y written in V should perhaps be thought of as a map X -> Y in a V-enriched category, but I haven't pursued this idea further
Oh this is really cool! I was wondering if there exists a suitable graphical language to talk about stuff like this
Here’s a new blog post by Paul Brunet and me about diagrammatic representations of compilers. johnwickerson.wordpress.com/2020/05/21/dia…
Categorical Deep Learning has been taught at the University of Cambridge!
It's been a while since I posted lectures on YouTube... so here's two 🫡 #1: Into the realm Categorical youtube.com/watch?v=yUxiDO… #2: Categorical Deep Learning youtube.com/watch?v=QEL4dj… (assumes familiarity w/ geometric DL) @bgavran3 @PaulRoyLessard @andrewdudzik @tlvg @_joaogui1
Fascinating work by Nicolas extracting single experts from @MistralAI's 8x22b new MoE model, and being able to finetune it to make it morph into a single 22b model! Ingenious!
🚀 Introducing Mistral-22b-V.01 A breakthrough in AI! 🧠💡 - First-ever MOE to Dense model conversion🔥 #Mistral22bV01 This model is NOT an MOE (It only has 22B params.) huggingface.co/Vezora/Mistral…
Pretty strong results on long context tasks like passkey detection and book summarisation. Glad to be interning with @TsendeeMTS @tuvllms at Google this summer. Looking forward to doing something great soon.
Google presents Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention 1B model that was fine-tuned on up to 5K sequence length passkey instances solves the 1M length problem arxiv.org/abs/2404.07143
Trends for United States
You might like
