Logo
Picture

maybe: shivam @kaffeinated

ml research @spotify, former matrix multiplier @twitter @x cortex, nyu. techno-optimist. probably on a bike 🚲☕. London Joined October 2012
  • Tweets
    3K
  • Followers
    455
  • Following
    3K
  • Likes
    18K
maybe: shivam retweeted
demishassabis Profile Picture

Demis Hassabis @demishassabis

6 days ago

Awesome to see this innovation in text diffusion. DiffusionGemma is lightning fast, 4x faster than other Gemma 4 models! Congrats to @bodonoghue85 and the team who worked so hard on this - excited to see what people build with it!

googlegemma Profile Picture

Google Gemma @googlegemma

6 days ago

Meet DiffusionGemma! An experimental open model that explores a fast approach to text generation, released under an Apache 2.0 license. Moving beyond sequential, token-by-token processes to generate entire blocks of text simultaneously. Here’s what’s new with DiffusionGemma: 👇

kaffeinated Profile Picture

maybe: shivam @kaffeinated

6 days ago

Quiet quitting but for AI research

SemiAnalysis_ Profile Picture

SemiAnalysis @SemiAnalysis_

7 days ago

BREAKING NEWS: Anthropic's latest model will NOT help you if it thinks your ML research/ML engineering is interesting, and/or will secretly degrade its IQ so that the average engineer won't notice. We are already seeing Anthropic's latest model's moderation filters our GPU

kaffeinated Profile Picture

maybe: shivam @kaffeinated

3 weeks ago

@TheStalwart @citrini It's wild that this is possible now (or will be soon), I hope it doesn't come to pass but we live in a world that makes Black Mirror feel tame now

maybe: shivam retweeted
trq212 Profile Picture

Thariq @trq212

a month ago

HTML is the new markdown. I've stopped writing markdown files for almost everything and switched to using Claude Code to generate HTML for me. This is why.

trq212 Profile Picture

Thariq @trq212

a month ago

x.com/i/article/2052…

maybe: shivam retweeted
GoodfireAI Profile Picture

Goodfire @GoodfireAI

a month ago

Neural networks might speak English, but they think in shapes. Understanding their rich *neural geometry* is key to understanding how they work – and to debugging and controlling them with precision. Starting today, we’re releasing a series of posts on this research agenda. 🧵

maybe: shivam retweeted
AnthropicAI Profile Picture

Anthropic @AnthropicAI

a month ago

New Anthropic research: Natural Language Autoencoders. Models like Claude talk in words but think in numbers. The numbers—called activations—encode Claude’s thoughts, but not in a language we can read. Here, we train Claude to translate its activations into human-readable text.

maybe: shivam retweeted
yacinelearning Profile Picture

Yacine Mahdid @yacinelearning

a month ago

this is how you look when you have multiple agents running btw

abakcus Profile Picture

Abakcus @abakcus

a month ago

Juggler juggling juggling jugglers.

maybe: shivam retweeted
cassidoo Profile Picture

Cassidy @cassidoo

2 months ago

idk how else to say this, but... build your dream projects now. I feel like all the tools are giving away a LOT for free/cheap now. It's only gotten more pricey over time, and will keep getting more expensive. Your ideas are subsidized now, think of it as a fire sale and build!

maybe: shivam retweeted
tinkerapi Profile Picture

Tinker @tinkerapi

2 months ago

First, to get you started, we've created 23 tutorials to walk you from the API basics to advanced training techniques and deploying models into production. tinker-docs.thinkingmachines.ai/tutorials/

maybe: shivam retweeted
s8mb Profile Picture

Sam Bowman @s8mb

2 months ago

We are entering the ~5 month period each year where there is no better city to be in than London. It's good to be back!

maybe: shivam retweeted
NASA Profile Picture

NASA @NASA

2 months ago

We see our home planet as a whole, lit up in spectacular blues and browns. A green aurora even lights up the atmosphere. That's us, together, watching as our astronauts make their journey to the Moon.

kaffeinated Profile Picture

maybe: shivam @kaffeinated

3 months ago

@giffmana "Have you tried turning the outer learning rate off and on again?"

maybe: shivam retweeted
gdb Profile Picture

Greg Brockman @gdb

3 months ago

if you can imagine it, you can build it

maybe: shivam retweeted
atulit_gaur Profile Picture

atulit @atulit_gaur

3 months ago

dude computers are actually so fucking insane when you really think about it. we literally figured out how to write some fake-ass rules called code and somehow convinced rocks to follow them. like actual rocks. sand, melted, purified, carved into tiny pathways where electricity just flows in patterns. that’s it. that’s the whole magic. and yet from that we get operating systems, compilers, kernels, networks, distributed systems, machine learning models, entire virtual worlds running inside other virtual worlds. billions of tiny electrical decisions per second, all because we defined some abstract logic. humans basically invented a language of instructions and taught matter itself to execute it.

maybe: shivam retweeted
fchollet Profile Picture

François Chollet @fchollet

3 months ago

The time to learn how to think for yourself was before genAI, if you missed your chance, good luck

maybe: shivam retweeted
AndrewCurran_ Profile Picture

Andrew Curran @AndrewCurran_

3 months ago

We are in a new phase since December, everything is moving faster. You can feel it in the most recent releases, you can feel it from the energy of people in the industry. I know some people still think this is hype, but this is real. Everything in this world is about to change.

AnthropicAI Profile Picture

Anthropic @AnthropicAI

3 months ago

Introducing The Anthropic Institute, a new effort to advance the public conversation about powerful AI. anthropic.com/news/the-anthr…

maybe: shivam retweeted
karpathy Profile Picture

Andrej Karpathy @karpathy

3 months ago

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project. This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.: - It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work. - It found that the Value Embeddings really like regularization and I wasn't applying any (oops). - It found that my banded attention was too conservative (i forgot to tune it). - It found that AdamW betas were all messed up. - It tuned the weight decay schedule. - It tuned the network initialization. This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism. github.com/karpathy/nanoc… All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges. And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

maybe: shivam retweeted
amytam01 Profile Picture

Amy Tam @amytam01

3 months ago

x.com/i/article/2031…

kaffeinated Profile Picture

maybe: shivam @kaffeinated

3 months ago

I've been working on this for weeks, and he just Tweeted it...out

karpathy Profile Picture

Andrej Karpathy @karpathy

3 months ago

The next step for autoresearch is that it has to be asynchronously massively collaborative for agents (think: SETI@home style). The goal is not to emulate a single PhD student, it's to emulate a research community of them. Current code synchronously grows a single thread of

kaffeinated Profile Picture

maybe: shivam @kaffeinated

3 months ago

@milalolli 🇬🇧🇬🇧

Trends for United States

Senegal
Shrek
Whoopi
The View
Rabiot
#SaveStargate
Dembele
Pizza Hut
Mbappe
Cursor
#FRASEN
Vance
deGrom
Woll
Cherki
#ElEsequiboEsDeVenezuela
Nicolas Jackson
CMLL
Cam Jordan
Nico Jackson

You might like

Lecturer (Assistant Professor) @ABDNCompSci | Postdoc research fellow @ucl_wi_group | PhD CS @SheffieldNLP | mechanistic interpretability, multimodal LLMs
Ruizhe Li
@liruizhe94
1K Followers
🎓 PhD in LLMs/GenAI. @otter_ai 🛠️ Founding MLE. 🎥 Filmmaker 100k+ views. 🌎 Lived in UK 🇬🇧 SG 🇸🇬 India 🇮🇳 US 🇺🇸 past: MSFT Research, AMZN Alexa, AI2.
Avijit Thawani (Avi)
@thawani_avijit
1K Followers
Making AI doing the boring shit. I don’t this place seriously and you shouldn’t either (personal account).
jjh
@jjh
970 Followers
PhD @uwnlp @AIatMeta. Bending the scaling laws.
Artidoro Pagnoni
@ArtidoroPagnoni
2K Followers
Compute Lead @ HRT, ex-AI infra at NVIDIA, #TwitterCortex, Madbits (acquired by Twitter), DeriveXperts (acquired by Markit)
Nicolas Koumchatzky
@nkoumchatzky
4K Followers
Co-founder & CEO Paddington Robotics Ex-Head of Applied Research @Twitter | Co-founder Magic Pony Technology (acq. by Twitter) | PhD @imperialcollege
Zehan Wang
@ZehanWang
4K Followers
PostDoc @Stanford. Prev @umasscs @genentech @Google @IIITDelhi. Dabbling with Interpretability, Retrieval and Bioinformatics.
Tanya Chowdhury
@ta_knee_aa
432 Followers
Alignment @ GWU
Shi Feng
@ihsgnef
943 Followers
Associate Professor at @CseHuji
Gabriel Stanovsky
@GabiStanovsky
854 Followers
Researcher @Alibaba_Qwen | Robust rewards & evaluation in LLMs. Formerly CS @HKUniversity & @ShandongU.
Qintong Li
@qintong_li
326 Followers
Entrepreneur | Ex ML Eng @Meta @Uber @Pinterest @Twitter | Built core HNSW update algo with Yury (most popular Vector search algo)
Apoorv Sharma
@apoorv_nsit
311 Followers