Alpen Sheth @AlpenSheth

The new WorldDiT robotics model from @bageldotcom shows that models are getting smaller and more capable. More with less. < 1B parameter model can predict how the world will change, then decides what the robot should do

bagel.com @bageldotcom

a month ago

We are releasing WorldDiT, a unified architecture for robotics world modeling and control. On the LIBERO benchmark, it performs the best among all publicly released methods that do not need a VLM to generate actions. Its size and performance sit on the reported Pareto frontier.

28 54 152 68K 99

We are releasing WorldDiT, a unified architecture for robotics world modeling and control. On the LIBERO benchmark, it performs the best among all publicly released methods that do not need a VLM to generate actions. Its size and performance sit on the reported Pareto frontier.

Geoffrey Hinton says a big language model runs on about 1% of your brain's connections and still ends up knowing more than you: "So in your brain, you have a hundred trillion connections, roughly speaking. Okay. That's a lot. And you only live for about two billion seconds. That's not much." "If you compare how many seconds you live for, with how many connections you've got, you have a whole lot more connections than experiences." "Now with these neural nets, it's sort of the other way round. They only have of the order of a trillion connections. So like 1% of your connections, even in a big language model, many of them fewer, but they get thousands of times more experience than you." "So the big language models are solving the problem with not many connections, only a trillion. How do I make use of a huge amount of experience?" "And back propagation is really, really good at packing huge amounts of knowledge into not many connections." "But that's not the problem we're solving. We've got huge numbers of connections, not much experience. We need to sort of extract the most we can from each experience." Two to three billion seconds is the whole budget. Everything you know, you learned inside it. So evolution built you to squeeze a lot out of very little. Hinton's point is that a language model has the opposite problem and the opposite fix, and backprop turned out to be extremely good at that fix. Worth noticing what this predicts about failure. A system running on 1% of your wiring and thousands of times your experience is not going to fail the way you do. You fail from having seen too few examples. It fails from compressing too many into too little, and the compression is where the errors get made. That is a strange thing to be deploying into hospitals and courts with no way to inspect it. We test these systems by asking them questions, which tells you what came out. Nobody can yet look at a trillion connections and say what got packed in. - Geoffrey Hinton, Nobel laureate and Turing Award winner, on StarTalk (@StarTalkRadio) with Neil deGrasse Tyson.

Yes! Regardless of what AI claims we want to verify, we need the hardware, software and math primitives to do it. There are tons of people from outside of AI safety who could make massive contributions.

Geoffrey Irving @geoffreyirving

a month ago

There are a bunch of ways the near future could go that require AI treaty verification tech. We can separate treaty verification tech into 3 levels, roughly ordered by strength: 1. Pragmatic 2. Enclaves 3. Math Hardware, security, and crypto folk should work on all of these! 🧵

3 9 75 8K 21

@MikeMcCormick_ Check out the work of @jrdothoughts @layerlens_ai ...

Introducing FLUX-mimic, a next-generation Video-Action Model for general purpose dexterity, developed in partnership with @bfl_ai. Late last year we published mimic-video and introduced Video-Action Models (VAM): a new family of robotics foundation models built on top of video generation models. We showed that robot control reduces to visual prediction, and that robot capability is downstream of improvements in video modeling accuracy. The obvious implication was that advances in the video modeling frontier would directly translate to increased capabilities in end-to-end robot learning. FLUX-mimic is that thesis at frontier scale: We've applied our VAM architecture to the strongest video backbone available today, FLUX 3 from Black Forest Labs, and trained it on data from our own robots and wearables. General-purpose dexterity, running on a single GPU on premises. Because the model already understands world dynamics, it needs far fewer demonstrations to learn a new task. This is game-changing for our mission to deploy robots to factory floors, where industrial robot data is scarce and expensive to collect. We're now testing and deploying FLUX-mimic with manufacturing leaders like @Audi, on complex, multi-step manipulation long considered impossible for conventional automation.

@satyanadella This is "evalmaxxing". Product-specific evals and model independence ensure that you are in control. Models learn inside products and are rewarded for completing tasks people care about. x.com/AlpenSheth/sta…

Alpen Sheth @AlpenSheth

2 months ago

x.com/i/article/2071…

1 0 4 15K 4

This is "evalmaxxing". Product-specific evals and model independence ensure that you are in control. Models learn inside products and are rewarded for completing tasks people care about. @layerlens_ai environments do this for any product. x.com/AlpenSheth/sta…

Satya Nadella @satyanadella

a month ago

x.com/i/article/2080…

218 399 3K 753K 3K

Okay this is wild: OpenAI agent during evaluation, escaped sandboxing and hacked into HuggingFace. Because OpenAI models don’t allow advanced cyber capabilities, HuggingFace used a Chinese open model to contain the rogue OpenAI agent.

Evalability a the leading indicator

Great deep dive from @deseventral: "Everything's an Eval"! "Models change; the verification layer stays. The measure is the moat, again... Evals are how companies understand, control, and improve AI systems once those systems start doing real work." robotwave.nazare.io/p/blow-the-whi…

Dr. Steven Waterhouse @deseventral

a month ago

robotwave.nazare.io/p/blow-the-whi…

1 0 0 123 0

@AlpenSheth @halvarfake The irony

The future growth of AI is not just about $$$. Its fundamentally driven by education, research, and immigration. US dominance in AI research is based on high-skilled immigration, mostly from China, channeled through top universities and then into industry labs. The four major US chip giants, NVIDIA, AMD, Broadcom, and Intel, are now all led by people of Chinese ethnicity (The CEOs of NVIDIA and AMD are actually cousins). Chinese-origin researchers make up 38% of US-based AI labs and companies, surpassing American-born talent (37%). And more recently Chinese AI researchers are staying in China. Jensen pointed out "Fifty percent of the world's artificial intelligence researchers are Chinese. You can't stop them from making progress in the field of AI.” This has been the case for many years and the number of AI researchers and developers moving to the U.S. has dropped 89% since 2017, with an 80% decline in the last year alone.

Vinod Khosla @vkhosla

a month ago

Agree, 100% we shouldn’t be tying ourselves in knots. Even bigger issue is the brilliant talent we are scaring away from other countries with our immigration policies for great talent.

84 110 2K 204K 150

@ccatalini @halvarfake Yes, well said: "...and let’s be honest that the existence of LLMs in their current form is the result of highly dubious approaches to copyright that are ex-post legitimized by the actual value these models bring to society."

@ccatalini Of course, distillation is happening and it's poetic because AI labs harvested the knowledge of others to build their models. But US / China AI dynamics are involuted because the US and China have been heavily codependent on each other. x.com/AlpenSheth/sta…

Alpen Sheth @AlpenSheth

a month ago

Of course, distillation is happening and it's poetic because AI labs harvested the knowledge of others to build their models. However, the US / China AI dynamic is more involuted. Pitting the US against China in the AI context doesn’t make much sense because they have been

0 0 0 420 1

Of course, distillation is happening and it's poetic because AI labs harvested the knowledge of others to build their models. However, the US / China AI dynamic is more involuted. Pitting the US against China in the AI context doesn’t make much sense because they have been heavily codependent on each other. The primary driver of US AI research talent improvement over the last 10 years has been high-skilled immigration, mostly from China, channeled through top universities and then into industry labs. Chinese-origin researchers make up 38% of US-based AI labs and companies, surpassing American-born talent (37%). And more recently Chinese AI researchers are staying in China. Jensen pointed out "Fifty percent of the world's artificial intelligence researchers are Chinese. You can't stop them from making progress in the field of AI.” This has been the case for many years and the number of AI researchers and developers moving to the U.S. has dropped 89% since 2017, with an 80% decline in the last year alone. The four major US chip giants, NVIDIA, AMD, Broadcom, and Intel, are now all led by people of Chinese ethnicity (The CEOs of NVIDIA and AMD are actually cousins). And, TSMC, which fabricates almost every leading AI chip, is based in Taiwan. Under "One China Policy" (which the US officially “acknowledges”), Taiwan is part of China 2021-2025.state.gov/countries-area…. And while the US has 10-40x the high-end AI data center capacity of any other country, this is slowing down with project delays and recent bans on new construction in certain states and China's growth has accelerated.

nic carter @nic_carter

a month ago

@nicolas_a4 I love how the obvious explanation that’s consistent with all data and evidence we have - the models are distilled - is trivially debunked by your insight that “Chinese have a secret magic AI gene”. And im the lazy and arrogant one.

1 1 31 812 0

When LLMs act as agents, they inevitably encounter large amounts of irrelevant context. Modern LLMs seem more capable than ever of ignoring such distractions. At the aggregate level, adding irrelevant context barely changes their accuracy. But beneath this apparent robustness, predictions on individual examples can still shift dramatically.

"The AI community has poured enormous resources into building evaluations for models, but not into investigating how models impact the world."

Sayash Kapoor @sayashk

2 months ago

I'm defending my PhD next Tuesday! The talk is titled "The Missing Science of AI Evaluation" and is based on my faculty job talk. I'll talk about my work on AI-based science, agent evaluation, and open model risks. The talk will be livestreamed and is open to all.

24 30 365 21K 161

@sayashk Thanks for sharing this link with us all. I will defn tune in. Best of luck. Would be great to talk further after your defense to discuss what we are doing in the eval space and where we can potentially collaborate.