the1eyecat @champly

🤖 Frontier models like GPT 6 Astra are getting surprisingly good at controlling robots. The results are exciting, but deploying frontier models on robots comes with substantial inference cost and latency. Can we distill these capabilities into compact models that run locally? We open-source 🥑Guava — a framework for distilling frontier models into compact robot agents, using purely simulation interaction data, no real-world data required! With only ~2K simulation trajectories, we finetuned Qwen3.5 into Guava-4B, with: 🤖 90.0% real-world success vs. 93.3% for GPT-5.4 ⚡ 7.1× lower model-call latency with local inference 🌎 Sim-to-real transfer without real-world fine-tuning 🧠 Generalization to unseen tasks and object configurations 🔄 Closed-loop recovery from execution failures Paper + code + project page: guava-harness.github.io 🎥 See Guava in action below. Joint efforts from @haowenssr, @xiruili7_li, @ShaoxiongYao, Peng Shi, @zhoutianyi, @jbhuang0604 , @furongh and @maojiayuan.

ロボットの世界モデルを推論時に動かさず、その表現だけをVLAへ移せる（arxiv.org/html/2609.2468…）。この研究は、世界モデルの画像特徴を事前に計算して保存し、動作を決めるVLAの画像特徴をコサイン類似度で合わせる。学習後は教師モデルと補助層を外すため、推論に追加モデルを積まない。 0.8BのVLAはLIBEROで95.3%から97.9%へ上がり、RoboCasa-GR1でも成功率が48.2%から50.5%へ改善した。RTX 5090での推論は32ms、メモリ使用量は1.86GB。未来を予測する世界モデルが学習中に得た表現を、小さな方策へ移す設計だ。 ただし実機での検証は、果物や卵のピック＆プレースと両腕ロボットの受け渡しなどに限られる。ベンチマークの数字が、そのまま幅広い現場の性能を示すわけではない。

Today, most multimodal LLMs are basically a vision encoder stitched to a language model. The alternative is to drop the vision backbone entirely and project raw image patches straight into the decoder. But which one scales better: a rich visual prior or a simpler architecture? 1/n

A paradox in robot learning: robot policies are spatially dumb and data-inefficient but their image features are spatially robust and semantically rich. Image encoders are spatially smart — features embed semantics + geometry + are multiview-consistent ... but robot policies we build upon them are spatially dumb — hundreds of demos to learn simple pick+place and a small camera bump breaks them To better bridge this gap, check out BIND: a new action head that binds each candidate robot action to its projecting 2D image feature(s) BIND basically offers the network the info of '‘choosing this candidate robot action would move the robot EEF to this image feature." Result: policies much more data efficient and robust to OOD viewpoints and OOD object positions (same rgb input, no 3D sensors, dense EEF trajectories out) 🧵

Fun fact: I had <24 hours to submit this paper to #IROS2026. I had simulation results, but no real-robot demo and I was traveling, far from every robot in my lab. Luckily, I had packed a @LeRobotHF SO-101 in my carry-on. I went to Walmart, bought some metal rods, magic arms, curtains, and toys, and built this rig in my Airbnb 👇 Used my iPhone as the scene camera, trained working policies within a few hours, and submitted just in time. Huge shoutout to @RemiCadene, @ClementDelangue, and everyone at @huggingface for making robotics this accessible.

Benedict Quartey @Benedict_Q

7 days ago

Thanks for highlighting our work @GlenBerseth ! Cc @_ericrosen @roboticseabass @StefanieTellex @rai_inst

0 1 9 10K 2

🤖 Should robots be generalists or specialists? At #IROS2026, @Ken_Goldberg & @andrea_bajcsy moderated one of the best debates in robotics. Matt Mason opened with a better question: “Should animals be humans?” 10 takeaways 🧵👇 1️⃣ @HarryXu12, for generalists: “Don’t build Excel. Build GPT.” We don’t know what we’ll need tomorrow, so buy generality just in case — and amortize it. The twist: GPT didn’t replace Excel. It learned to drive it. 2️⃣ @GeorgiaChal split “general” into 4 axes: Body. Scene. Contact. Task. “Breadth and reliability are measured per axis. One does not imply the other.” Her prediction: “The next task is the test.” 3️⃣ The most honest number of the day, from Georgia’s slides: A general model calling robot tools. Coarse block placement: 19/20 ✅ Precision insertion: 2/20 ❌ “Coarse pick and place works. Physical composition and precise contact not yet.” Tool use ≠ dexterity. 4️⃣ @KaterinaFragiadaki’s architecture: The generalist writes the curriculum. The specialist does the reps. Video → sim → RL on subgoals → specialist policies. No teleop. At deployment: “Fast specialists for the common case. Flexible generalists for the long tail.” 5️⃣ Specialist bench: “Fixture and conveyor are part of the intelligence.” A fixture is a prior. A conveyor is a scheduler. Strip them to look “general” and you’ve moved work onto a pricier learned controller. Optionality that hurts reliability isn’t intelligence. It’s cost. 6️⃣ A surgical robot doing one procedure 10,000 times learns exactly what can go wrong. “Generality is the enemy of certification.” From the floor: Are human drivers generalists or specialists? We certify generalists one task at a time. That’s what a license is. 7️⃣ Toshio Fukuda changed the register: “NON-AI matters.” Kizuki: help offered before it’s asked for. “Awareness comes from the body, not from the model.” Japan: ~29% aged 65+, ~570k care workers short by 2040. “If people live to 120, must our machines last that long?” 8️⃣ The floor fight: “A human is a generalist.” “We’re all human specialists.” The room accidentally re-derived pretrain-then-finetune. Veterans argued from what has shipped. Newcomers argued from what has scaled. Both are data. 9️⃣ The title asked for a side. The slides drew a stack: General planning + recovery on top. Fast, certifiable specialists below. A verifier is a fixture made of code. 🔟 The real question: How thick can the middle get before the body must specialize anyway? Generalist or specialist? Maybe the winning architecture is neither. It’s a stack. #IROS2026 #Robotics #PhysicalAI #RobotLearning #WorldModels

keyboard_arrow_left Previous keyboard_arrow_right Next

Qwen3.8-Flash-Next for a single DGX Spark got a serious upgrade with TensorFold🔥 This is a completely new recipe, optimized and tuned for TesnorFold! Expect further improvements! - KV cache pool is ~1.3M - Default context 256k, with 5 concurrent. - Faster everything compared to vLLM! Performance: Decode prose 62+ tok/s single stream Decode prose 119+ tok/s on 5 streams Prefill is mostly 2500 tok/s across the board! In addition, expect TensorFold recipes for GLM 5.3 Flash and DeepSeek v4.1 Flash - coming soon! Get it here: github.com/MiaAI-Lab/Qwen…

Can we improve VLA execution without real-world RL? VLaRL uses VLM-latent-conditioned residual RL, trained entirely in simulation and transferred directly to real robots through latent alignment. A new preprint is out! arxiv.org/abs/2609.30868

Disaggregated quantization for LLMs 🧵 LLM prefill and decode can be processed by separate weights in different precisions. Boosts speed AND accuracy. Offloading makes it zero-overhead on long sequences for dense models. Applicable to existing quantized checkpoints. Paper: arxiv.org/abs/2609.26333 Code: github.com/IST-DASLab/dis… Qwen3.8 NVFP4 prefillers: huggingface.co/ISTA-DASLab/Qw…

New robot added to LeRobot! 💪 Both DM and RS versions of the reBot B601 from @seeedstudio are now officially supported. The arm features a 2.5kg payload capacity and 6 degrees of freedom + gripper. In the example below, the robot performs towel folding fully autonomously. The ACT model was trained on just 200 examples.

the thing I care about in HomeBody isn't “GPT Astra controls a humanoid.” it's what they refused to make the frontier model do. Astra chooses things like pick(target), navigate(goal) or open_drawer(handle). once a skill starts, the robot handles segmentation, stereo depth, IK, collision checking, visual servoing and bounded retries locally. arm commands run at 250 Hz, locomotion at 50 Hz. Astra only comes back when the skill succeeds or local recovery is exhausted. that separation feels extremely right. LLMs/VLMs are incredible at deciding what should happen next. they're a ridiculous place to put a 250 Hz control loop. and the failure modes on the project page are almost more useful than the demo: cloud reasoning introduces pauses, finger servos overheat on long tasks, Real2Sim adds setup/API cost, and the local stack still needs an RTX 4090 laptop. this is basically the architecture I've been converging toward for physical agents: slow intelligence, fast reflexes. keep semantics/planning replaceable. keep anything that can smash hardware deterministic and local. tml.stanford.edu/homebody/ @Stanford @Caltech

I'll be at IROS2026@Pittsburgh from 9/27-10/1🤖 Happy to meet in person and chat about robot learning from human data/Ego data/WAM, DM me! I'll present HumanEgo at two workshops: 1. WORLDS: worlds-iros2026.github.io, 2. ScaleInfra: scale-infra.github.io/iros2026/

Zhi (Leo) Wang @TX_Leo_Wang

4 months ago

1/ 🧠Humans are the best robot data source! 2/ 👓Human egocentric video is rich in quantity, but poor in quality. 3/ Beyond scaling data, smarter representation and architecture matter just as much. 4/ Want an open-source framework to train your own learn-from-human-data robot

28 77 620 128K 437

I’m in Pittsburgh for #IROS2026! 🤖 Excited to present 🔥 RoboSSM — scaling in-context imitation learning with State Space Models! Happy to chat at my poster session on Monday(9/28)! 🎤 Lightning Talk: 9:35AM @ ROOM 409/410 📍 Poster: 11:00 AM - 12:30 PM @ Kiosk E8

Youngju Yoo @yyoungju_yoo

a year ago

Humans learn new manipulation skills from examples and improve as they see more examples. How can we endow robots with the same ability? 🤖 🚀We introduce RoboSSM, scalable in-context imitation learning that enables robots to learn and improve at test time—robots can improve

7 32 139 21K 92

Awesome-UMIに新しく9件のデバイスを追加しました！ 主にCoRLに採択されているプロジェクトと、商用プロダクトになります。 commissure-inc.github.io/Awesome-UMI/

What can Astra do when given a humanoid embodiment? We built HomeBody to find out. Controlled by GPT Astra, it carries out long-horizon tasks in a previously unseen kitchen—from tidying up across the room to retrieving remembered objects from ambiguous requests—without environment-specific training data or additional policy learning. Here's how we did it 👀: tml.stanford.edu/homebody/

Jev, now open source: Lev A 4B open source System One model based on Qwen backbone The best performance for it's small size huggingface.co/interfaze-ai/l…

We have converted GLiNER2.5-Decide to coreml. ~4× faster, ~5× less Peak RAM, half the size. model: huggingface.co/FluidInference code: github.com/FluidInference

Fastino Labs @fastinoAI

2 weeks ago

Introducing GLiNER2.5-Decide, our new 340M parameter open weight, encoder-based decision model. GLiNER2.5-Decide is built for fast, deterministic classification. The model evaluates a set of user-defined typed questions and rules, and jointly decodes their answers, returning

84 162 1K 162K 1K

Humanoids are coming to LeRobot! 🤖 Learn what you can do with the @UnitreeRobotics G1 in LeRobot, how to use @NVIDIARobotics SONIC, and which open-source humanoid hardware we currently support! Read more in our latest blog: huggingface.co/blog/nepyope/b…

NVIDIA and Stanford just challenged Jev. (their new System 1 architecture runs up to 9x faster.) It is called a Contrastive Language Model, or CLM. Like Jev, CLM is not designed to generate text. It handles the small, repeated decisions inside AI systems, such as choosing a tool, ranking a patch, routing a request, or selecting the next action. But CLM reaches those decisions differently. Instead of generating an answer token by token, it treats decision-making as a retrieval problem. Here is how it works. 1) Encode the state CLM takes the current situation, such as an agent’s context or the state of a game, and converts it into a vector. It uses a frozen Qwen3-8B model with a small trainable state projection head. 2) Encode every possible action A separate action head converts each candidate into the same vector space. In the Mario example, the candidates are left, jump, and right run. CLM does not invent a fourth option. It only evaluates the actions supplied by the application. 3) Learn which states and actions belong together During training, the correct state-action pair is pulled closer while incorrect pairs are pushed apart. A batch of B examples produces a B × B similarity matrix. The matching pairs sit on the diagonal. Every other pairing becomes a negative example. This contrastive training uses InfoNCE, the same general mechanism behind systems such as CLIP and dense retrieval. 4) Turn similarity into a decision At inference, CLM measures the cosine similarity between the state and every candidate action. A softmax converts those scores into a probability distribution. The application can choose the winner, apply a confidence threshold, or escalate an uncertain result. The real speed advantage comes from separating states and actions. Actions can be embedded once and cached. If an agent repeatedly chooses between the same tools, CLM only needs to encode the changing state and compare it with stored action vectors. That replaces repeated generation with one embedding pass and a set of cheap dot products. The researchers report that CLM-8B matches Jev across computer-use, gaming, and tool-calling evaluations while reaching up to 9x lower latency. The improvement is largest when actions repeat or the candidate set grows. CLM still has limits. It cannot generate new actions, its probabilities are relative to the supplied candidates, and its strongest verifier results require task-specific fine-tuning. But its central idea is powerful. The entire research is open-source, including the code. Read more here: contrastive-lm.notion.site When software already knows the possible answers, an AI model should score them instead of generating more words. I also wrote a full breakdown on how system one models like Jev work. The article is quoted below.

Akshay 🚀 @akshay_pachaar

2 weeks ago

x.com/i/article/2100…

127 656 6K 2.4M 11K

My pick for the best model on DGX Spark, as of today 👇 1️⃣ 1 Spark: Qwen 3.8 Flash 2️⃣ 2 Sparks: GLM 5.3 Flash (NVFP4 + DFlash2, 262K context) 3️⃣ 3 Sparks: GLM 5.3 Flash 4️⃣ 4 Sparks: GLM 5.3 Flash (500K context) You could run the full GLM 5.3, but for 2+ Sparks, Flash is still the one I'd run today 🔥 TP2 recipe 👇 github.com/tonyd2wild/GLM…