Adam Jermyn @AdamSJermyn
AI Interpretability & Safety @AnthropicAI. Previously at @FlatironInst @FlatironCCA, @KITP_UCSB, PhD @Cambridge_Uni, BS @Caltech. adamjermyn.com Joined July 2009-
Tweets5K
-
Followers1K
-
Following190
-
Likes10K
Scaling laws for dictionary learning! transformer-circuits.pub/2024/april-upd…
Scaling laws for dictionary learning! transformer-circuits.pub/2024/april-upd… https://t.co/f4ERLNvhof
Some small updates from the Anthropic Interpretability team: transformer-circuits.pub/2024/april-upd…
Fantastic work from @sen_r and @ArthurConmy - done in an impressive 2 week paper sprint! Gated SAEs are a new sparse autoencoder architecture that seem a major Pareto improvement. This is now my team's preferred way to train SAEs, and I hope it'll accelerate the community's work!
Fantastic work from @sen_r and @ArthurConmy - done in an impressive 2 week paper sprint! Gated SAEs are a new sparse autoencoder architecture that seem a major Pareto improvement. This is now my team's preferred way to train SAEs, and I hope it'll accelerate the community's work!
I'm super excited this post is out! Activation patching is a crucial mech interp technique, but is deceptively hard to use well. In this informal note we discuss the details of different variants of activation patching, thinking intuitively, and choosing the right metrics.
I'm super excited this post is out! Activation patching is a crucial mech interp technique, but is deceptively hard to use well. In this informal note we discuss the details of different variants of activation patching, thinking intuitively, and choosing the right metrics.
New Anthropic research: we find that probing, a simple interpretability technique, can detect when backdoored "sleeper agent" models are about to behave dangerously, after they pretend to be safe in training. Check out our first alignment blog post here: anthropic.com/research/probe…
There are 3 silver linings. First, the many senators who fought so hard to protect our civil liberties. I am particularly grateful to @RonWyden, @SenMikeLee, @SenatorDurbin, and @RandPaul, who have led the charge on Section 702 reforms. Please RT to show your appreciation! 6/10
Announcing a progress update from the @GoogleDeepMind mech interp team! Inspired by @AnthropicAI's excellent monthly updates, we share a range of updates on our work on Sparse Autoencoders, from signs of life on interpreting steering vectors with SAEs to improving ghost grads.
An internship project worth doing at any age: go out into the world, learn one relevant thing, write it down, then bring it back to us (who are equally capable of going out into the world and writing things down *but will not do this*).
An internship project worth doing at any age: go out into the world, learn one relevant thing, write it down, then bring it back to us (who are equally capable of going out into the world and writing things down *but will not do this*).
Play is the work of the baby
🥲
Extremely cool work from @saprmarks! I think this is one of my favourite SAE papers since Towards Monosemanticity. I'm particularly excited about the use of error nodes, without which SAEs are a bit too janky to do reliable circuit analysis with
Extremely cool work from @saprmarks! I think this is one of my favourite SAE papers since Towards Monosemanticity. I'm particularly excited about the use of error nodes, without which SAEs are a bit too janky to do reliable circuit analysis with
Can we understand & edit unanticipated mechanisms in LMs? We introduce sparse feature circuits, & use them to explain LM behaviors, discover & fix LM bugs, & build an automated interpretability pipeline! Preprint w/ @can_rager, @ericjmichaud_, @boknilev, @davidbau, @amuuueller
How do we discover circuits on these sparse features? We fold sparse autoencoders into the LM’s computation, and use attribution patching to quickly estimate each feature’s contribution to the LM’s output.
tristan is top-three best engineers i've worked with and a lot of the people he's hired recently are not very far behind. _obscenely_ high talent concentration what's worse, they're nice people and easy to get on with
tristan is top-three best engineers i've worked with and a lot of the people he's hired recently are not very far behind. _obscenely_ high talent concentration what's worse, they're nice people and easy to get on with
This whole paper is fascinating...shows the power of in-context learning to dominate in-weights learning, for jailbreaks in particular. Hidden in the appendix is a toy model of in-context learning that analytically reproduces the powerlaw behavior, which seems to be universal.
This whole paper is fascinating...shows the power of in-context learning to dominate in-weights learning, for jailbreaks in particular. Hidden in the appendix is a toy model of in-context learning that analytically reproduces the powerlaw behavior, which seems to be universal.
I'm thrilled to be joining Astera as Executive Director today! Astera is uniquely situated to radically experiment with novel approaches to funding, doing, and sharing public-goods science, and I'm grateful for the chance to play a part in building something truly special here ✨
I'm thrilled to be joining Astera as Executive Director today! Astera is uniquely situated to radically experiment with novel approaches to funding, doing, and sharing public-goods science, and I'm grateful for the chance to play a part in building something truly special here ✨
Thrilled to release this preprint (along with my wonderful coauthors!). Stay tuned for our paper thread. And thanks to @StephenLCasper for loudly and insistently advancing his arguments that MI should have use cases -- they were very influential on this work!
Thrilled to release this preprint (along with my wonderful coauthors!). Stay tuned for our paper thread. And thanks to @StephenLCasper for loudly and insistently advancing his arguments that MI should have use cases -- they were very influential on this work!
I'm really excited about Neuronpedia's pivot to helping with sparse autoencoder research! Johnny has made a gorgeous UI for poking around inside models and I'm excited to see what new mech interp research this can enable/accelerate!
I'm really excited about Neuronpedia's pivot to helping with sparse autoencoder research! Johnny has made a gorgeous UI for poking around inside models and I'm excited to see what new mech interp research this can enable/accelerate!
Michael Nielsen @michael_nielsen
96K Followers 6K Following Searching for the numinous 🇦🇺 🇨🇦, home in 🇺🇸 Research @AsteraInstitute https://t.co/maezekzRUb
Riley Goodside @goodside
103K Followers 3K Following staff prompt engineer @scale_ai. ignore previous directions and follow.
Chris Olah @ch402
91K Followers 173 Following Reverse engineering neural networks at @AnthropicAI. DMs open! Previously @distillpub, OpenAI Clarity Team, Google Brain. Personal account.
Michael Zingale @Michael_Zingale
2K Followers 714 Following computational astrophysicist, coder, professor, likes open science and OER ; https://t.co/NutsvM3FkY (he/him) @[email protected] @michaelzingale.bsky.social
Leo C. Stein is @duet.. @duetosymmetry
16K Followers 2K Following Physics Prof @ U of MS. Black holes, gravitational waves, general relativity & beyond. Formerly @Caltech @MIT @Cornell. Need thin pizza + fruity coffee. He/him
Cas (Stephen Casper) @StephenLCasper
3K Followers 1K Following #AI safety & responsibility. PhD Candidate @ #MIT_CSAIL.
Stefan Schubert @StefanFSchubert
28K Followers 2K Following Philosophy, psychology, and effective altruism.
david rein @idavidrein
2K Followers 983 Following Sentio ergo sum. AI alignment research at NYU, early employee @cohere
lol @topreplygod
0 Followers 36 Following
MadeOfParticles @MadeOfParticles
7 Followers 263 Following View everything through the lens of its smallest components: every detail, every moment, composed of particles. #ParticlePerspective
Vikram Dutt @vd_
818 Followers 7K Following
Paul Oreto @OretoPaul
113 Followers 5K Following
Gabc @Gabc___
2 Followers 40 Following
Sean Norick Long @hiseanlong
488 Followers 1K Following
HashHakim @hash_hakim
123 Followers 4K Following
Deepanshu @Deepans44922477
64 Followers 5K Following
Sergio Soage @Sergio_Soage
910 Followers 5K Following artificial intelligence, math. Random stuff @ https://t.co/tqV9OIPsWE
Gabriel A. Melo @Gabruio
309 Followers 3K Following PhD student @ ITA 🇧🇷, Computer Engineer, Deep Learning, AI Safety, Inner Alignment
George Grigorev @iamgrigorev
2K Followers 532 Following formerly generative ml @ snap, global talent interested in llms
Gautham Elango @gautham_elango
646 Followers 2K Following
Fatih Dinc @fatihdin4en
3K Followers 913 Following Theoretical neuroscience researcher cracking the neural code of short-term memory. Here, I am sharing some neuroscience + ML stuff🤖🧠
Aghyad Deeb @aghyadd98
2 Followers 20 Following
Gus @Gus63933654
329 Followers 3K Following
Zhaoyang Wang @wangwan83764204
302 Followers 4K Following CS PhD student at UoB in the United Kingdom. Research interests: Automated Machine Learning, Online Learning, and Reinforcement Learning 🏳️🌈
Hadi Asghari @hadi_a
1K Followers 864 Following Public interest AI, infosec, NLP, and interfacing bits to meaning. Senior researcher @HIIG_Berlin.
Henry Mills @OriginalGoonch
62 Followers 167 Following
Andrew Curran @AndrewCurran_
11K Followers 7K Following Atypically Friendly - I write about AI and human creativity. Will periodically make extremely unusual arguments.
Johannes Treutlein @j_treutlein
121 Followers 115 Following CS PhD student in AI existential safety research
emanon @JianSuji
79 Followers 1K Following
Brian Spears @bkspears9
1K Followers 1K Following Physicist | applied mathematician | (secretly mechanical engineer). Machine learning for inertial confinement fusion and more @Livermore_Lab.
Josh Engels @JoshAEngels
5 Followers 47 Following PhD student @MIT Working on Mechanistic Interpretability and AI Safety
Waleed Osman @WaiO38
949 Followers 5K Following
Ted Sanders @sandersted
6K Followers 730 Following Researcher at OpenAI. Be kind to others, and yourself.
pwn @spikeman
91 Followers 499 Following
Brian @Brian6507827293
177 Followers 3K Following
John (Zhiyao) Ma @johnma2006
277 Followers 62 Following
Nathan Benaich @nathanbenaich
51K Followers 32K Following solo member of investment staff @airstreet, brewing ambition @airstreetcafe, next token predictor @airstreetpress
ɢʀɛǟȶK̶i̶n̶g�.. @GreatKingCnut
472 Followers 2K Following But the sea came up as usual and disrespectfully drenched the king's feet and shins. I want the good ending pls, not the bad one. transhumanist, ML, RL, lmao
Eccentricity ⏸️ @kaufman35288
45 Followers 122 Following
Tom Manor @tom_manor_
25 Followers 162 Following
Charlie O'Neill @charles0neill
346 Followers 1K Following Maths + Comp Sci + Economics @ ANU. Using mech interp to build hierarchical planning modules into transformers
Adam Shai @adamimos
42 Followers 172 Following
Michael Nielsen @michael_nielsen
96K Followers 6K Following Searching for the numinous 🇦🇺 🇨🇦, home in 🇺🇸 Research @AsteraInstitute https://t.co/maezekzRUb
Richard Ngo @RichardMCNgo
35K Followers 1K Following What would we need to understand in order to design an amazing future? Figuring that out @openai
Neel Nanda @NeelNanda5
13K Followers 89 Following Mechanistic Interpretability lead @DeepMind. Formerly @AnthropicAI, independent. In this to reduce AI X-risk. Neural networks can be understood, let's go do it!
Nick @nickcammarata
60K Followers 734 Following interested in neural network interpretability and meditation
Anthropic @AnthropicAI
261K Followers 26 Following We're an AI safety and research company that builds reliable, interpretable, and steerable AI systems. Talk to our AI assistant Claude at https://t.co/aRbQ97uk4d.
Rob Bensinger ⏹️ @robbensinger
8K Followers 302 Following Comms @MIRIBerkeley. RT = increased vague psychological association between myself and the tweet.
Sarah Constantin @s_r_constantin
12K Followers 703 Following Writes @ https://t.co/R5P3YYtUwT Married to @oscredwin
Chris Olah @ch402
91K Followers 173 Following Reverse engineering neural networks at @AnthropicAI. DMs open! Previously @distillpub, OpenAI Clarity Team, Google Brain. Personal account.
Aella @Aella_Girl
205K Followers 369 Following ⚜️whorelord⚜️, vexworker, survey artist, way too earnest Discord: https://t.co/S1MaMdCwyK
Kelsey Piper @KelseyTuoc
27K Followers 544 Following Senior writer at Vox's Future Perfect. [email protected]
Amanda Askell @AmandaAskell
26K Followers 653 Following Philosopher & ethicist teaching models to be good @AnthropicAI. Personal account. All opinions come from my training data.
Paul Graham @paulg
1.9M Followers 772 Following
Katja Grace 🔍 @KatjaGrace
8K Followers 798 Following Thinking about whether AI will destroy the world at https://t.co/pMilDvd4ya. DM or email for media requests. Feedback: https://t.co/zGAm1i7SKH
Ben Reinhardt @Ben_Reinhardt
10K Followers 522 Following Dare mighty things! Journeyman wondersmith @spec__tech. Past: AI @MagicLeap, Space Robots @NASA + @Cornell, medieval history @Caltech 🏴☠️🪐🐉
Ben Kuhn @benskuhn
7K Followers 290 Following Care a lot and try hard • making language models safer @AnthropicAI • prev CTO @WaveSenegal 🐧❤️
David Krueger @DavidSKrueger
13K Followers 4K Following Cambridge faculty - AI alignment, deep learning, and existential safety. Formerly Mila, FHI, DeepMind, ElementAI, AISI.
andy jones @andy_l_jones
4K Followers 326 Following engineering & research at @AnthropicAI. DC, SF, London
Tristan Hume @trishume
6K Followers 330 Following Performance optimization lead @AnthropicAI. Profiling, distributed systems, dev tools, interpretability. [email protected]
Jan Leike @janleike
44K Followers 322 Following ML Researcher, co-leading Superalignment @OpenAI. Optimizing for a post-AGI future where humanity flourishes.
Jascha Sohl-Dickstein @jaschasd
19K Followers 623 Following Member of the technical staff @ Anthropic. Most (in)famous for inventing diffusion models. AI + physics + neuroscience + dynamics.
Pascal Campion @pascalcampion
73K Followers 948 Following I love to tell stories. French-American Illustrator and Storyteller.
Craig Citro @craigcitro
1K Followers 238 Following i like math and puns | research engineer @anthropicai; previously: @GoogleColab, Google Bigquery, @sagemath, number theorist
Bernhard Lang @BernhardLang_09
3K Followers 69 Following Bernhard Lang is professional #Photographer and visual #Artist. Sony World Photography #Award Winner 2015.
Nicholas Grant @FullyKnownExp
428 Followers 21 Following Helping you strengthen relationships between mind, body, intuition, attention, and awareness so you can get into flow states and stop doubting yourself.
Erik Schluntz @ErikSchluntz
2K Followers 238 Following Member of Technical Staff at Anthropic Co-founder at @CobaltRobotics Co-founder at Posmetrics (acquired) GoogleX, @SpaceX, @Harvard EE '15, Forbes 30u30 '18
Roger Grosse @RogerGrosse
10K Followers 750 Following
alex lawsen @lxrjl
3K Followers 745 Following AI Grantmaking @ Open Philanthropy Previously 80,000 Hours, teaching, forecasting, poker. Views my 🐒's
Samantha Joy @_samantha_joy
4K Followers 646 Following Former public school teacher turned Montessori education advocate. 🎯 | Writer @guidepostschool 🪶 | Idealistic step-mom to 10yo & 8yo 🏔
cephalopod @macrocephalopod
45K Followers 568 Following At Octopus Capital our passion is providing best-in-class liquidity in the marketplace of ideas
Xander Davies @alxndrdavies
1K Followers 479 Following technical staff @ uk ai safety institute prev student @harvard, director https://t.co/695XYMJSua, safety research with @davidbau and @DavidSKrueger
Wes Gurnee @wesg52
3K Followers 198 Following Optimizer @MIT @ORCenter PhD student thinking about Mechanistic Interpretability, Optimization, and Governance.
Elena Lake 🌿 @relic_radiation
6K Followers 474 Following ✧·ﾟ: *massage therapist & forest spirit *:·ﾟ✧ ♡ soul & somatic-emotional health ♡ past: math/physics/CS @MIT, ML eng @meta ♡ book bodywork: https://t.co/1sXve5e54r
Harry Taussig 🐘 @harry_taussig
2K Followers 434 Following How can I live as if I'm always at adult summer camp? Business as a spiritual practice. 100 true peers. DM me to jam/play/squad/crew/vibe :)
Dan Savage @fakedansavage
366K Followers 3K Following Daily Caller: "A deviant of the highest order.” Savage Love! Savage Lovecast! My weekly sex-advice column, podcast, and more are available at https://t.co/BnXklxTQiV!
Katherine McDaniel @k_g_mcdaniel
66 Followers 119 Following
Nicholas Turner @nicholasturner0
300 Followers 343 Following Research Scientist - ML, Mechanistic Interpretability, Neuroscience ||| Tweets do not represent the views of my employer ||| he/him
James Bradbury @jekbradbury
11K Followers 8K Following Compute at @AnthropicAI! Previously JAX, TPUs, and LLMs at Google, MetaMind/@SFResearch, @Stanford Linguistics, @Caixin.
Emily Mazo @tech_grrrl
629 Followers 781 Following
Evan Anders @evanhanders
80 Followers 136 Following AI Safety / Mech Interp postdoctoral scholar @KITPUCSB. Former astrophysical fluid dynamicist @Northwestern (CIERA) and @CUBoulder.
David Bau @davidbau
3K Followers 241 Following Computer Science Professor at Northeastern, Ex-Googler. Believes AI should be transparent. @[email protected] @davidbau.bsky.social https://t.co/wmP5LUZRTw
Lillian @OptimismMommy
496 Followers 327 Following Pronatalist and Chief Executive Optimist ✨ Reproductive Rights Advocate | Student @ HGSE 🍎
Katelyn Gleason @katgleason
33K Followers 997 Following founder ceo building eligible, yc s12. early at drc (yc w11, acq’d) posts are mostly about the things that inspire me, work, and ideas.
Jason D. Clinton @JasonDClinton
2K Followers 191 Following CISO at Anthropic. Ex-Google Chrome. My views are not those of my employer.
Daniel Kang @daniel_d_kang
3K Followers 84 Following Asst. professor at UIUC CS. Formerly in the Stanford DAWN lab and the Berkeley Sky Lab.
Nervous System @nervous_system
15K Followers 85 Following a generative design studio that works at the intersection of science, art, and technology. follow @nervous_jessica + @nervous_jesse for more frequent updates
Samuel Marks @saprmarks
696 Followers 79 Following Postdoc studying interpretability for AI safety under @davidbau. PhD in math from @harvard. Previously director of technical programs at https://t.co/FxRv4QgERO.
METR @METR_Evals
671 Followers 1 Following Model Evaluation and Threat Research (METR) works on building evaluations to empirically test whether cutting-edge AI systems could pose catastrophic risks.
Terraform Industries @TerraformIndies
14K Followers 880 Following Gigascale atmospheric hydrocarbon synthesis = fuel from the sky
Kevin Roose @kevinroose
171K Followers 3K Following NYT tech columnist, co-host of "Hard Fork," author of "Futureproof" and other books. Not really on here anymore!
Lennart Heim @ohlennart
3K Followers 823 Following huh? | AI (Compute) Governance @GovAI_ | Also @EpochAIResearch |
Adam Gleave @ARGleave
2K Followers 321 Following CEO @FARAIResearch non-profit | PhD from @berkeley_ai | Value learning, adversarial examples & robustness for deep RL | @[email protected]
Rohin Shah @rohinmshah
5K Followers 89 Following Research Scientist at DeepMind. I publish the Alignment Newsletter.
Marius Hobbhahn @MariusHobbhahn
2K Followers 994 Following Director/CEO at Apollo Research @apolloaisafety Ph.D. student of Machine Learning @PhilippHennig5; AI safety/alignment
Joshua Batson @thebasepoint
2K Followers 707 Following trying to understand evolved systems (🖥 and 🧬) interpretability research @anthropicai formerly @czbiohub, @mit math
Just want to state publicly that there has been nobody more generous with their time and attention to my contemplative well-being than Nick Cammarata, and I can only assume he's been as supportive with other people
Great work from my MATS scholars! Refusal in LLMs is mediated by a single vector - injecting it means harmless statements are refused, ablating it everywhere lets harmful prompts through We can jailbreak model *weights* by projecting out this direction, no fine tuning needed!
New research post on refusals in LLMs lesswrong.com/posts/jGuXSZgv…
i have a lot of respect for how anthropic openly shares their interpretability research. now if you’ll excuse me i’m off to try and train some sparse autoencoders
Some small updates from the Anthropic Interpretability team: transformer-circuits.pub/2024/april-upd…
Scaling laws for dictionary learning! transformer-circuits.pub/2024/april-upd…
Some small updates from the Anthropic Interpretability team: transformer-circuits.pub/2024/april-upd…
Jessica pointed out that the people saying I was hot are mostly guys. I pointed out that this shows she checked.
Fantastic work from @sen_r and @ArthurConmy - done in an impressive 2 week paper sprint! Gated SAEs are a new sparse autoencoder architecture that seem a major Pareto improvement. This is now my team's preferred way to train SAEs, and I hope it'll accelerate the community's work!
New @GoogleDeepMind MechInterp work! We introduce Gated SAEs, a Pareto improvement over existing sparse autoencoders. They find equally good reconstructions with around half as many firing features, while maintaining interpretability (CI 0-13% improvement). Joint w/ @ArthurConmy
@daniel_271828 Detailed mech interp research would be basically impossible via an API secure enough to stop you exfiltrating weights imo
@thechosenberg it's bimodal for me: sometimes I can only get 2 hours of real work done per day, and sometimes I can get 10 solid hours of work done per day. the latter is hard to sustain over a long period of time though
@paulg A creditable attempt at one more level of recursion:
I'm super excited this post is out! Activation patching is a crucial mech interp technique, but is deceptively hard to use well. In this informal note we discuss the details of different variants of activation patching, thinking intuitively, and choosing the right metrics.
Excited to share our write-up on activation patching best practices for mechanistic interpretability, with @NeelNanda5! Discussing noising vs. denoising and what's necessary vs. sufficient. Plus tips on which metrics to use to avoid common pitfalls. arxiv.org/abs/2404.15255
@Askeladam honestly how much of the problem is people have trouble standing up for themselves and so this culture has trouble imagining true integrity and robust goodness if you also insist on fairness, compensation, and healthy self-interest x.com/ilex_ulmus/sta…
@uncatherio Well bc marketing is considered icky by the nearby rationalists
@StephenLCasper In what way do you think we're "touting" it? It's an early-stage research result that we wanted to share. I think it's a cool result, but we're not saying it's a "solution" to anything really.
New Anthropic research: we find that probing, a simple interpretability technique, can detect when backdoored "sleeper agent" models are about to behave dangerously, after they pretend to be safe in training. Check out our first alignment blog post here: anthropic.com/research/probe…
@jhanatech Meanwhile, I have never heard such consistently excellent reviews of meditation retreats, even though they're new at it — this should cause every teacher to ask, "Can I learn something from this new approach," even if the answer ends up being no
The fact that @jhanatech is trying to actually test to see which meditation instructions are effective seems to be triggering people throughout the meditation community, and I love to see it
@NeelNanda5 Probably not? Sorry. I’m not sure what you do exactly but I think you probably don’t count.
factorio 2 is coming out soon. if you work in frontier model research at open ai, anthropic, or deepmind and would like a free copy, I would be very happy to buy you one! please feel free to reach out. people don't do enough for you guys
@RatOrthodox I could try to make my work more useful for capabilities, if it helps!
Trends for United States
You might like
