Jamie Simon @learning_mech
doing fundamental science of deep learning | PhD from Berkeley | can catch a whole egg in my mouth jamiesimon.io Berkeley, CA Joined December 2023-
Tweets140
-
Followers1K
-
Following75
-
Likes195
I quite like this line of work! blends "mechinterp" and "learning mechanics" approaches to fundamental science of deep learning.
Scaling laws describe how loss changes with scale. Do neurons inside models change predictably too? We study vision and language models up to 30B params and find systematic scaling in neuron universality, specialization, and selectivity. Paper+code: avdravid.github.io/rosetta-neuron… 1/n
I thoroughly enjoyed reading this recent paper by @yasamanbb et al (arxiv.org/pdf/2602.15029) that derives analytically why certain latent variables must lead to geometry in word embeddings. (getting Fourier modes even with open boundary but exponential kernel is neat!) I think it would be great to compare this to some of @prfsanjeevarora et al's work on this (eg tinyurl.com/4az4325n) More broadly, I have been thinking about the right data generating process for language. For vision, we have latent spaces with great manifold structure (eg the SO3 pose of an object) and nonlinear mixing functions. But for language? Are there really any continuous latent variables? What is the "DSprites" of language? Is it all just co-occurrence stats or is there something more in LLM word embeddings?
@KrzakalaF ah, lovely! I quite like this sort of "posit a simpler dynamics and study it" sort of approach. very physicsy! I like Misha Belkin's NFA/RFM stuff (which this reminds me of) for the same reason. possible path towards an answer to our Open Dir 1?? learningmechanics.pub/openquestions/…
@electro_vansh ah, thx! unf don't think I'll have the time, but honored :)
did you know that with a few modifications, you can get the Ising model to simulate cells fighting to the death? one of my favorite side projects of all time: jamiesimon.io/blog/cell-figh…
Leaked Sam Altman messages (2023)
Mechanistic interpretability aspires to be the biology of deep learning. @KuninDaniel and @learning_mech say that an emerging theory of deep learning they and their team call 🛠️ learning mechanics 🛠️ will be the physics.
crazy how you can pinpoint the exact curvature of a trillion dimensional model based on how wiggly the loss curve is
@MattHausmannAtx (yes, look up the Cellular Potts model)
@MattHausmannAtx psh who would want to
@guzmansalv me too. my favorite method is having an immune system!
for the bright-eyed and bushy-tailed: there's a Learning Mechanics discord! young academics who want to do research in this area should especially consider joining + starting convos. discord.gg/GTHfUnf7hz
Excited to share that our paper “Sequential Group Composition: A Window into the Mechanics of Deep Learning” was accepted to ICML 2026 in Seoul! Co-led with @giovannimarchet and @AdeleMyersPhD @hopfbifurcator @ninamiolane Paper: arxiv.org/abs/2602.03655
Palantir office speedrun
ditto. props to @justanotherlaw for taking a second look :) (though I also found merit in the criticisms in the first version.) hopeful we can eventually (hopefully soon enough...) make contact w/ AI alignment + governance, whose noble causes we would v much like to aid.
This is a great post and I especially respect the author for updating his view when presented with new information. I strongly encourage young researchers interested in interpretability, science of DL, and safety to look at it. lesswrong.com/posts/6SRq7mZ9…
It seems we're at a stage where deep learning is evolving from alchemy into an engineering discipline; this is an exciting paper which lays out that a scientific theory is emerging for Deep Learning. Paper: arxiv.org/abs/2604.21691 Tweet: x.com/learning_mech/…
1/ Deep learning is going to have a scientific theory. We can see the pieces starting to come together, and it's looking a lot like physics! We're releasing a paper pulling together these emerging threads and giving them a name: learning mechanics. 🔨 arxiv.org/pdf/2604.21691 🔧
yeah, totally! I once messaged everyone on facebook with my first and last name. I eventually made a big group chat! v ethnically + geographically diverse. probs the closest I've gotten as an adult to meeting a truly random slice of the US.
Jury selection is cool. It's probably the closest you ever get to seeing a true random sample of the population
Aren’t diffusion models explicitly derived from a correspondence with physics and entirely consistent with how physics says you should model systems over a range of scales ( ie mori zwanzig theory: langevin dynamics with a fitted vector field ? ) what more do you want?
Sorry, but these correspondences between AI and physics are vacuous. People have been making them since (at least) the 80s, and they always come to nothing.
Praneet @prntvctr
46 Followers 666 Following
Ivan Drozdov @vipeink
0 Followers 7 Following
Abhijit Bakshi @abh1zen
23 Followers 7K Following
Imp @impcampsus
39 Followers 2K Following
Fellah mohamed @moh93344
0 Followers 82 Following
Max Ficco @maxficco
3 Followers 104 Following
nahrule @nahrule90000
0 Followers 183 Following
kay @snng24
8 Followers 2K Following
Talha @ecstasy496
11 Followers 699 Following A naughty thought a day keeps the stress away. 😜 Robotics, AI, CV, ML, DL, RL, RPA, IoT, Software, Cloud, Blockchain, Data and Automation Expert
Fadi @Zara_Syrien
139 Followers 985 Following zu spät für die Götter 💀 zu früh für die große Politik ☠️ Atheist | Agonistic Liberal (Neo-reactionary)
Xi Tang @Julie17_XI
0 Followers 11 Following
landimension @landimension1
2 Followers 106 Following
Dario Bojanjac @DarioBojanjac
26 Followers 580 Following
Tran Phong @HbkTran
60 Followers 2K Following My own profile dedicated for AI research in robotics. Engineer @dyson. prev at @NUSingapore
satuki @satuki889650615
9 Followers 162 Following
Alison Pouplin (on Bl... @a_ppln
844 Followers 701 Following ML researcher. Works on geometry, uncertainty quantification for ML. 🧬 Mostly on bluesky 🦋
Samuel Knoche 🔎 @SamuelKnoche
329 Followers 1K Following
Zhao Meihan @MeihanZhao315
3 Followers 76 Following
Jan Dubiński @CVPR @jan_dubinski_
371 Followers 960 Following Astra Fellow at Constellation right now | PhD student at Warsaw University of Technology and NASK working on AI safety and generative models
Philipp Nazari @philna00
106 Followers 249 Following PhD Candidate at Max Planck ETH Center for Learning Systems
Xinpeng Wang @XinpengWang_
312 Followers 501 Following Incoming Assistant Professor/Faculty Fellow @NYUDataScience |PhD student @cislmu | Supervision, Alignment | RS intern @AIatMeta
Nathan Henry @nathan_w_henry
115 Followers 153 Following AI Safety & Deep Learning Theory - PhD @ UC Berkeley
Diandiallo Diallo @Diandiallo4897
154 Followers 4K Following
Nima Dehghani @neurovium
3K Followers 5K Following https://t.co/UiK6usETTi N4: NeuroPhysics, NeuroComputation, NeuroDynamics, NeuroAI https://t.co/L44S1cC3s9
Taylor Hodge @jtaylorhodge
355 Followers 278 Following Striving to create that in the presence of which we feel more alive Software gardener/eng: https://t.co/prDUQqX2SM ↝ @swellai @rupa_health Narrative Science @workos
Marcos Bolaños @marcosqanil
12 Followers 118 Following
beanzie @beanzieBanzai
36 Followers 747 Following
Siru Zhong @zhongsiru191756
2 Followers 179 Following
Maryam Taj @MaryamTaj2004
2 Followers 64 Following
Daniel Korchinski @DanKorchinski
312 Followers 262 Following Complex system studying complex systems. Postdoc at EPFL. Let's go climbing!
Alexi Gladstone @AlexiGlad
3K Followers 659 Following PhD @ UIUC and doing stuff @flappyairplanes. Working on EBTs/EBMs, World Models, Reasoning. Prev @Meta, @PalantirTech, @UVA
Amir Arsalan Soltani @Amir_Arsalan_
339 Followers 975 Following In pursuit of the Truth without fear of asking questions in this judgemental world.RTs/sharing/likes/comments are result of my struggle for this,not endorsement
Nico @xxxxfreeland
0 Followers 461 Following
PeopleSearches.com �... @peoplesearches
294K Followers 94K Following Find Info and Resources #MissingPersons #LawEnforcement: Anonymously https://t.co/MSJPWSWc7V #FBI - #Trafficking https://t.co/oCj2i0F2A9 https://t.co/nkdbHv8SNn #AMBERAlert #SILVERAlert
Unidad básica Tom Sa... @molagbolas
999 Followers 2K Following Econochanta de mandrilandia. Gordo anime prog/fusión. Mi lista de libros y recetas de cocina en el link.
Alex Dong @alexdong
2K Followers 866 Following RL for domains where human judgment is critical. AI Lead/Cofounder @tacitco. Epeeist. Gardener. Reader. I love low-ego, positive, competent and earnest people.
Assertoric @KantEnjoyer
0 Followers 164 Following
Jan Feyereisl @thefillm
755 Followers 4K Following Senior Research Scientist - GoodAI (@GoodAIdev) & Executive Director - AI Roadmap Institute (@AIroadmap)
Cladach @Cladach_em
5 Followers 607 Following
Jeff Rose @rosejn
313 Followers 850 Following Spatial Intelligence | AI Researcher | Software Entrepreneur | Former Deepmind
Yaroslav Bulatov @yaroslavvb
11K Followers 1K Following @southparkcommons (early OpenAI, Google Brain, Meta) https://t.co/bxo5udY3ib https://t.co/SLix8Hrt4w https://t.co/Ur3GWKpmp6
Sophia Sanborn @naturecomputes
6K Followers 3K Following Scientist / Founder | Neuro + AI | Prof @stanford | https://t.co/ymy1Hq4vbf & @metamorphiclabs
Erin Grant @ermgrant
3K Followers 2K Following Assistant Professor / Faculty Fellow @NYUDataScience studying cognition in mind & brain with neural nets, Bayes, other tools | https://t.co/aC3k1IFSl4
Lawrence Chan @justanotherlaw
3K Followers 170 Following I do AI Alignment Research. Formerly @METR_Evals, @redwood_ai; on leave from my PhD at UC Berkeley’s @CHAI_berkeley. Opinions are my own.
Prof. Feynman @ProfFeynman
1.4M Followers 0 Following A universe of atoms, an atom in the universe. Tribute to the great explainer. Tweets about Science and Wisdom. Portrait by L.V Patten.
David Bau @davidbau
7K Followers 287 Following Computer Science Professor at Northeastern, Ex-Googler. Believes AI should be transparent. @[email protected] @davidbau.bsky.social https://t.co/wmP5LV0pJ4
Quanta Magazine @QuantaMagazine
362K Followers 615 Following Illuminating math and science. Supported by @SimonsFdn. 2022 Pulitzer Prize in Explanatory Reporting.
CERN @CERN
2.4M Followers 447 Following CERN is the European laboratory for particle physics, home to the #LHC. — All media © CERN, human-generated unless otherwise stated. French: @CERN_FR
Steven Strogatz @stevenstrogatz
180K Followers 3K Following Mathematician, writer, Cornell professor. All cards on the table, face up, all the time.
Greg Egan @gregeganSF
26K Followers 309 Following SF writer / computer programmer Latest novel: MORPHOTROPHIC Latest collection: SLEEP AND THE SOUL Web site: https://t.co/yeU5bLA3mx Also: @[email protected]
Ben Edelman @EdelmanBen
638 Followers 53 Following Agent Security lead @ U.S. Center for AI Standards and Innovation. Prev: science of deep learning PhD @ Harvard
Scientific Methods fo... @scifordl
502 Followers 22 Following Workshop @ ICLR 2026. Using controlled experiments to test hypotheses about the inner workings of deep networks.
Jesse Hoogland @jesse_hoogland
2K Followers 2K Following Researcher and decel working on developmental interpretability. Executive Director @ Timaeus
Daniel Murfet @danielmurfet
2K Followers 583 Following Mathematician. Head of Research at Timaeus. Working on Singular Learning Theory and AI alignment.
Neil Mallinar @nmallinar
277 Followers 664 Following PhD student @ UCSD, prior to that: Research Intern @ Google Research & MSR NE, Research Engineer at Pryon Inc & IBM Watson.
cathzhao_4 @cathzhao_4
13 Followers 61 Following SF. chief of staff @imbue_ai 🛠️ designer, builder, prev @nbc @upenn etc etc
Krzakala Florent @KrzakalaF
3K Followers 439 Following I like Physics, Statistics, Machine learning, Computer Science & above all playing 🎸. Happy dad 👧 👧. Also professor @ EPFL. Views are my own.
Joey Turnbull @joey__turnbull
14 Followers 21 Following Physics PhD @ Berkeley researching deep learning
Andrea Montanari @Andrea__M
9K Followers 314 Following Professor, Statistics and Mathematics, Stanford University. (Opinions are my own)
Cengiz Pehlevan @CPehlevan
3K Followers 1K Following Theoretical neuroscience, theory of neural computation, physics of learning and intelligence. Associate Professor of Applied Mathematics @Harvard SEAS
Lénaïc Chizat @LenaicChizat
2K Followers 280 Following Assistant professor @epfl (previously @cnrs researcher). Mathematics of AI and data science
Berfin Simsek @humansimsek
769 Followers 452 Following I never learn better in here. Just wanted to let you know
Tengyu Ma @tengyuma
43K Followers 577 Following Assistant prof. @ Stanford; Chief AI Scientist @ MongoDB; Former Co-founder/CEO of Voyage AI Working on ML, DL, RL, LLMs, and their theory.
Abdulkadir Canatar @canatar_a
356 Followers 995 Following Research Fellow @FlatironCCN. Theoretical Neuroscience, Machine Learning, Physics. Prev: @Harvard and @sabanciu
Dan Roberts @danintheory
7K Followers 735 Following Scientist @OpenAI. Prev. co-founder @diffeo, acquired by @salesforce // co-authored The Principles of Deep Learning Theory // studied gravity.
Arthur Jacot @ArthurJacot3
207 Followers 35 Following Researcher in the theory of deep learning. My biggest hit (yet): the Neural Tangent Kernel.
Florentin Guth @FlorentinGuth
584 Followers 196 Following Postdoc at @NYUDataScience and @FlatironCCN. Wants to understand why deep learning works.
Andrew Lampinen @AndrewLampinen
12K Followers 2K Following Interested in cognition and artificial intelligence. MTS at @AnthropicAI. Previously @DeepMind, cognitive science @StanfordPsych. Tweets are mine.
Alex Atanasov @ABAtanasov
2K Followers 2K Following Fascinated by scaling and universality. PhD from @harvardphysics. Not a Bayesian. Opinions my own 🇧🇬
Hamza Tahir Chaudhry @hamzatchaudhry
200 Followers 466 Following Applied Math Ph.D. Student at Harvard University
Hidenori Tanaka @Hidenori8Tanaka
6K Followers 1K Following Group Leader, Physics of Intelligence Program at Harvard University Physics of Artificial Intelligence Group, NTT Research, Inc.
Daniel Yamins @dyamins
5K Followers 601 Following CS, psych, and neuro prof @ Stanford. NeuroAI and "regular AI". Also harpsichords and bonsai. https://t.co/xCFbmgT6TG
Eric J. Michaud @ericjmichaud_
4K Followers 1K Following Trying to make deep neural networks among the best understood objects in the universe. 💻🤖🧠👽🔭🚀
Berkeley AI Research @berkeley_ai
272K Followers 456 Following We're graduate students, postdocs, faculty and scientists at the cutting edge of artificial intelligence research.
Jacob Steinhardt @JacobSteinhardt
12K Followers 82 Following Associate Professor of Statistics and EECS, UC Berkeley // Co-founder and CEO, @TransluceAI
Nika Haghtalab @nhaghtal
4K Followers 276 Following Associate Professor @Berkeley_EECS. Research on Foundations of ML and AI.
Niloofar @niloofar_mire
10K Followers 2K Following Technical staff @humansand, incoming asst. prof @LTIatCMU @CMU_EPP, ex RS in @AIatMeta, postdoc @uwcse, Ph.D. @ucsd_cse, former @MSFTResearch -Privacy, ML, NLP
Ziming Liu @ZimingLiu11
14K Followers 881 Following Assistant Professor @ Tsinghua CollegeAI (incoming), Postdoc @ Stanford, PhD @ MIT, BS @ PKU. Physics of AI, interpretability, Structuralism, KAN
Michael Nielsen @michael_nielsen
119K Followers 5K Following Searching for the numinous 🇦🇺 🇨🇦, currently live in 🇺🇸 Research @AsteraInstitute https://t.co/maezekzRUb https://t.co/2dWwZKrvrn
Stella Biderman @BlancheMinerva
18K Followers 858 Following Ensuring that tech companies don't have a monopoly on being able to do research on cutting edge AI @AiEleuther. She/her
Ben Recht @beenwrekt
34K Followers 326 Following optimization. machine learning. uc berkeley. I blog at https://t.co/fkJujOPsJb The world won't end.You might like
