vinothkumar @vinothkumarDEV

Leaving Meta and PyTorch I'm stepping down from PyTorch and leaving Meta on November 17th. tl;dr: Didn't want to be doing PyTorch forever, seemed like the perfect time to transition right after I got back from a long leave and the project built itself around me. Eleven years at Meta. Nearly all my professional life. Making many friends for life. Almost eight years leading PyTorch, taking it from nothing to 90%+ adoption in AI. Walking away from this was one of the hardest things I've ever done. But I'm leaving with a full heart. PyTorch handles exascale training now. It powers foundation models that are redefining intelligence. It's in production at virtually every major AI company. It's taught in classrooms from MIT to rural India. The tools I dreamed about making accessible? They are. The barrier to entry I wanted to lower? It's almost gone. To be clear, there’s so much more to do. As long as AI evolves at a breakneck pace, PyTorch will continue to play catch up. Obsessing over the yet-to-come sometimes makes us forget how much we’ve already done. To everyone who built this with me—who believed research should be joyful, that tools should be elegant, that open source changes everything—thank you. This wasn't my journey. It was ours. What's next for me? Something small. Something new. Something I don't fully understand yet. Something uncomfortable. I could have moved to something else inside Meta. But I needed to know what's out there. I needed to do something small again. I couldn't live with the counterfactual regret of never trying something outside Meta. It's very hard to leave. I probably have one of the AI industry’s most leveraged seats, I lead the software layer that powers the entire AI industry. Every major AI company and hardware vendor are on a speed dial. This kind of power is really hard to give up. But curiosity ultimately won out in my head. Keep making AI delicious and accessible. I'll be watching. Probably filing issues. Definitely staying involved. Is PyTorch going to be okay? I don't want to be doing PyTorch forever. I don't want to be like Guido or Linus— bound to a single thing for decades. Last November, coinciding with the birth of my daughter, I started planning my exit with Aparna. My goal was to leave PyTorch in a good and stable place. By this August, during the second half of my parental leave, I knew: Edward, Suo, Alban, Greg, John, Joe and Jana were ready. The team faced hard people, product, technical and organizational problems and didn’t feel the need to lean back on me to solve these for them (unlike in the past). The product story they crafted for the PyTorch Conference was coherent—really coherent. The things I'd flagged red were turning healthy. The project didn't need me anymore. Unlike 2020-2022 (when I stepped down to go do robotics and came back when Lin, Dima and Dwarak left), I have strong confidence that this time PyTorch is truly resilient. The most aligned culture carriers of PyTorch – Greg, Alban, Ed, Jason and Joe are at the decision table now, and people with strong value alignment – Suo, John and Jana have joined them at the table. And there’s a long list of equally value-aligned people willing to sit at the table should any of these people leave. There are many little things that make up my confidence on the people – John worked on Julia and open-source for a very long time (in fact we hacked a Torch.jl in 2015), Suo has been the strongest systems builder and strategic partner I’ve had for the past two years, and Jana worked on resilient core systems for a very long time, I’ve had long technical and organizational discussions with her over the past few months that give me confidence. And the product lineup and execution in 2025 should be sufficient evidence for any remaining doubt. I’m confident that this band of PyTorchers are going to do exceptionally well. PyTorch might change in flavor because I no longer impose my own taste from the top, but I’m confident that the values are going to stay intact and the product is going to be awesome. My time at Meta The early years of FAIR were absolutely magical. I was part of a small family of absolutely brilliant people building state-of-the-art AI out in the open. From working on GANs with Emily Denton, Rob Fergus, Leon Bottou, Martin Arjovsky and the (now legendary) Alec Radford to building Starcraft bots with Gabriel Synnaeve, to building the first FAIR Cluster with Howard Mansell, to working on object detection with Adam Lerer and Piotr Dollar, to building PyTorch. It was more fun than I can describe in words. 2015 and 2016 were probably the most productive and professionally enjoyable years of my life. I’ll probably romanticize this period of my life forever. When I joined FAIR, I had massive impostor syndrome, and the first 3 months were very very difficult. I can’t credit Andrew Tulloch enough for being the most thoughtful, kind and welcoming mentor, without whom I wouldn’t have made it. I’m so damn bullish for Meta just from the fact that he’s back. --- My time on PyTorch was special. I loved every part of building it—designing it, managing it, being the PM, TL, comms lead, doc engineer, release engineer, squashing bugs, growth hacking, turning it into a coherent product with hundreds of people, transitioning it to industry stakeholdership – the whole nine yards. To the core PyTorch team at Meta: the engineers, researchers, open-source maintainers, docs writers, CI infrastructure folks, hardware partners, the community builders. To the hundreds more inside and outside Meta—thank you. You turned a library into a movement. There are too many people to credit and thank, but I can't not mention Adam Paszke, Sam Gross, Greg Chanan, Joe Spisak, Alban Desmaison, Edward Yang, Richard Zou, Tongzhou Wang, Francisco Massa, Luca Antiga, Andreas Köpf, Zach DeVito, Zeming Lin, Adam Lerer, Howard Mansell and Natalia Gimelshein. And Schrep. They made the launch happen. And so many more people became centrally important later: Lu Fang, Xiaodong Wang, Junjie Bai, Nikita Shulga, Horace He, Mark Saroufim, Jason Ansel, Dmytro Dzhulgakov, Yangqing Jia, Geeta Chauhan, Will Constable, Briah Hirsh, Jane Xu, Mario Lezcano, Piotr Balecki, Yinghai Lu, Less Wright, Andrew Tulloch, Bruce Lin, Woo Kim, Helen Suk, Chris Gottbrath, Peng Wu, Joe Isaacson, Eli Uriegas, Tristan Rice, Yanan Cao, Elias Ellison, Animesh Jain, Peter Noordhuis, Tianyu Liu, Yifu Wang, Lin Qiao and hundreds more. It’s criminal of me to not take the space to list out everyone else I should be mentioning here. PyTorch is nothing without its people ❤️. The most joyful moments of building PyTorch was meeting users eager to share their happiness, love and feedback. I remember a grad student coming to me at Neurips 2017, in a slurring emotional voice he said he’d been trying to make progress on his research for 3 years but within 3 months of using PyTorch he made so much progress that he was ready to graduate. That moment made it tangible that what we do matters, a lot, to a lot of people, even if you don't constantly hear from them. I do miss the intimacy of the PyTorch community, with a 300 person conference that felt like an extended family gathering, but I feel that’s a small price to pay considering the scale of impact PyTorch is truly having today – yes the Conference is now 3,000 people where market-moving deals get brokered, but it’s helping orders of magnitude more people to do their best AI work. I miss the intimacy, but I'm proud of that growth. --- To Mark Zuckerberg and Mike Schroepfer, who believed that open-sourcing is fundamentally important and is a sound business strategy. This is so hard to understand for most people within the course of business, but we’ve run lock-step on this strategy without ever having to discuss it. Without you two, neither FAIR nor PyTorch would’ve happened. And those mean so much to me. To Yann LeCun and Rob Fergus, for building the magical early FAIR that I so revere. To Aparna Ramani, a leader that I find so rare at Meta in her ability to hold a really high bar for the org, technically brilliant with the span to discuss deep infra systems and industry-strategy within the same conversation and for being an absolute execution-machine! I’ve learned so much from you. To Santosh, Kaushik, Delia, Oldham and Ben for being so welcoming to Infra. For someone coming over from FAIR with a wildly different culture, you all made me feel at home and made me part of the family, and thank you for that. To all my managers who've championed me through the PSC video game – Serkan, Howard, Jerome, Abhijit, Yoram, Joelle, Aparna and Damien – I owe you a lifetime of drinks. --- Signing off for now. —Soumith

Introducing Willow, our new state-of-the-art quantum computing chip with a breakthrough that can reduce errors exponentially as we scale up using more qubits, cracking a 30-year challenge in the field. In benchmark tests, Willow solved a standard computation in <5 mins that would take a leading supercomputer over 10^25 years, far beyond the age of the universe(!).

@sama 🙏 , quantum + AI in the multiverse future to come, congrats on the o1 launches too!

Winning the @NobelPrize is the honour of a lifetime and the realisation of a lifelong dream - it still hasn’t really sunk in yet. With AlphaFold2 we cracked the 50-year grand challenge of protein structure prediction: predicting the 3D structure of a protein purely from its amino acid sequence. Proteins are the building blocks of life, and knowing the structure of a protein is crucial for understanding the function it performs. We then folded all 200 million proteins known to science and made those structures freely available for anyone in the world to use, with the help of our wonderful collaborators at @emblebi. Over 2 million researchers have already used AlphaFold2 and its predictions to advance a huge range of important work - everything from enzyme design, to disease understanding, to drug discovery. But this is only the beginning. Over the next few years AI will help us make great strides towards developing new and more effective therapies for today's most prevalent diseases, and the fantastic team at @IsomorphicLabs are making rapid progress on this mission. I can’t think of a more important or beneficial use of AI. Then of course there is advancing AGI itself, the original and enduring goal, and the vision behind the founding of DeepMind nearly 15 years ago. If AI is built safely and responsibly, I believe it will be one of the most transformative and beneficial technologies ever. I’ve always thought of AI as the ultimate tool to help us accelerate scientific discovery. Congratulations to John Jumper (and David Baker!), the amazing AlphaFold team, and all our incredible colleagues at @GoogleDeepMind and @Google that supported and encouraged us along the way - this award is for all of us! It’s been such an honour and privilege to work with all of you to advance the frontiers of AI and science, and there is so much more to come!

The Nobel Prize @NobelPrize

2 years ago

BREAKING NEWS The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Chemistry with one half to David Baker “for computational protein design” and the other half jointly to Demis Hassabis and John M. Jumper “for protein structure prediction.”

510 8K 19K 9.1M 1K

This is what will matter 1000 years from now. Not your politics. Not your stupid tantrums about who platformed who on some website. Not your incomprehensible desire to send NASA's entire budget to the third world. This guy reignited the Space Age. He spent his own money, hired a bunch of dudes, and reignited the Space Age. And together, they underbid and outdid NASA and its pet dinosaur corporations on every conceivable level. This is history happening before you. If you are a puddlefish, if you think this is a wasteful showpiece or science project, then you don't understand physics, economics, astronomy, or in fact the basic layout of the universe you live in. We live in a tiny puddle at the bottom of a well. Out there is an entire universe, full not only of stuff to explore, but full of stuff to build things out of. Big things. Wonderful things. Things that are going to make all of the cool stuff you have today, all of human civilization to date look like early Assyrians writing stuff down on wet clay with a reed. Infinite resources. Infinite energy. Infinite space. Instead of fighting over little patches of land, we will have an infinite 3d volume. Enclose it in steel, pump it full of air, spin it, and it's a habitat. Instead of scratching tiny scraps of metal out of the crust of one planet, we will break down entire asteroids and smelt them. Instead of drilling for hydrocarbons and turning water wheels, we will harness entire suns, split the atom, and eventually draw our fuel from the substance that makes up 99% of the entire universe. None of your local, temporal Earth politics matter compared to this. This is more important than pride parades and abortions, more important than tribal conflicts in eastern Europe and southwest Asia, more important than tensions with Russia and China. More important, in the long run, than the United States of America. America's most important function, its one most vital purpose, is to serve as an incubator for this. Because this changes everything. All of our arguments about conditions on this planet become obsolete, because the whole planet becomes just one suburban neighborhood. All of our wars over resources and territory become obsolete, because no one has time to brawl when we're all sitting on top of a dragon horde with sacks and shovels. Everyone who was alive at the time remembers where they were when Kennedy died in Dallas. When the towers fell. When the Eagle landed. When the Wall came down. But this... this is the real moment, one of the first of many. They are what every child will know about a thousand years from now, even if they have four arms and are genetically engineered for zero-g, or are sentient blocks of code running on a sphere of computronium enclosing an entire star. You may not live to see that, depending on what we do or don't invent, and when. But it will happen, and you will live to see wonderful things. If the puddlefish don't get in the way. Don't be a puddlefish.

Elon Musk @elonmusk

2 years ago

Good morning

18K 30K 488K 78.3M 6K

Giving a talk at IIT Madras on October 22.

Office of Global Engagement, IIT Madras @ogeiitm

2 years ago

Public Lecture Join us at the Subra Suresh Distinguished #Lecture Series III, featuring @ylecun , VP & Chief AI Scientist at #Meta and Turing Award winner! Discover insights on "How Could Machines Reach Human-Level Intelligence?" on October 22, 2024, from 4:00 PM to 5:30 PM at

35 84 865 373K 102

The Intelligence Age: ia.samaltman.com

here is o1, a series of our most capable and aligned models yet: openai.com/index/learning… o1 is still flawed, still limited, and it still seems more impressive on first use than it does after you spend more time with it.

Still here. 👋 And still more than 15 billion miles (24 billion km) away from Earth. It takes my team about 22.5 hours to send me a command and another 22.5 hours for my response to get back to them. That will continue to increase the further I travel into interstellar space. -V1

How could #AlphaFold 3 transform drug discovery? Most drugs are small molecules known as ligands that bind to proteins to change how they interact in human health and disease. AlphaFold 3 can predict these interactions to atomic accuracy.

Thrilled to announce AlphaFold 3 which can predict the structures and interactions of nearly all of life’s molecules with state-of-the-art accuracy including proteins, DNA and RNA. Biology is a complex dynamical system so modeling interactions is crucial blog.google/technology/ai/…

It is with great sadness that we learned of the passing of Peter Higgs. An iconic figure in modern science, Higgs postulated the existence of the eponymous Higgs boson in 1964. Its discovery at CERN in 2012 was the crowning achievement of the Standard Model of particle physics – a remarkable theory which explains the visible universe at the most fundamental level. Today CERN pays tribute to Peter Higgs. Our thoughts are with his family and friends. home.cern/news/obituary/…

Announcing first end-to-end production of pipeline grade natural gas from sunlight and air.

Humans think about high dimensional objects by using silly mental pictures of 3D spaces and praying the intuitions transfer This is arguably what makes mechanistic interpretability so damn hard—you can’t intuit representational structure However: AI’s don’t intrinsically have this limitation. In theory, one could train an AI to intuitively understand vector spaces of dimension 4+ by giving it the right data/task. In the long term, this could mean A(G)I’s reasoning about the internals of other A(G)I’s far better than people (in addition to solving many other high dimensional problems) Ultimately, this is what I suspect will be the most truly transformative aspect of AGI: the ability to think in terms of useful representations that humans don’t have access to

The psychologist and Nobel winner Daniel Kanheman, dies at age 90. He forever changed Economics. If you haven’t read Thinking Fast and Slow, I suggest you pick up a copy. RIP washingtonpost.com/obituaries/202…

I asked Jensen about his iconic AI prediction: ‘How far are we from a world where every pixel is generated at real-time frame rates?’ Jensen thinks we’re ‘5-8 years away’ and is already seeing signs of traversing the S-curve of innovation. The entire gaming and simulation industry is set to undergo significant changes over the next half-decade. I don’t think we’ve fully grasped what is yet to come.

I got to try this demo in person today at NVIDIA HQ — mind blown 🤯 Immaculate streaming quality, full-res CAD models with real-time ray tracing. Stay tuned for a deep dive video after GTC. I’ll also be interviewing the VP of XR behind these efforts.

@itisprashanth Hi All, I believe lungs has lost its actual capacity/efficiency post covid for most of the guys. Am not sure everybody has awareness about this. So please stop this trekking madness if you or any of your friends/relatives are planning to do.

Very excited to welcome @karpathy as Magic's latest investor!

🚨 BREAKING: Nvidia just released Chat with RTX, an AI chatbot that runs locally on your PC. It can summarize or search documents across your PC's files and even YouTube videos and playlists. The chatbot runs locally, meaning results are fast, you can use it without the internet, and the user's data stays private. New day, new chatbot. Let's go.