DENG Lab @ SJTU @SJTUDengLab
https://t.co/kWvEXvQgXu Shanghai Joined November 2024-
Tweets108
-
Followers160
-
Following76
-
Likes55
🚀 Introducing Streaming-WAM: Streaming Your World-Action Model for Real-Time Robot Manipulation. 🧠Streaming-WAM lets the robot act while the WAM thinks ahead, predicting the future during execution through action-conditioned world prediction. 🌐 Project: sjtu-deng-lab.github.io/Streaming-WAM/
🦿 Real-robot evaluation On a single RTX 5090 at 30 Hz, Streaming-WAM achieves 90.0% success (27/30), cuts Chunk Time from 682.1 ms to 122.62 ms (5.6× faster), and reduces rollout time from 90 s to 38 s (2.4× faster).
Streaming-WAM also transfers to RoboCasa, showing the approach is not tied to a single WAM family. It reaches 75.35% average success, comparable to X-WAM at 75.42%, while reducing Chunk Time by 3.2× and Total Time by 1.8×.
On RoboTwin 2.0, Streaming-WAM reaches 91.74% total success, improving over FastWAM-Joint at 87.00%. At the same time, it delivers 12.0× faster Chunk Time and 1.4× faster Total Time—speeding up control without sacrificing task performance.
On LIBERO, Streaming-WAM reaches 98.20% average success while delivering 12.0× faster Chunk Time and up to 3.0× faster Total Time than FastWAM. Without action conditioning, success drops to 96.25%—showing the value of aligning prediction with ongoing actions.
⚡ Faster generation alone is not enough. Streaming-WAM moves prediction into the robot’s execution window. Across LIBERO, RoboTwin 2.0, and RoboCasa, it reduces both Chunk Time and end-to-end rollout time—turning model speed into faster robot execution.
⚙️ Real-time WAM acceleration benefits from full-stack optimization. Combining model-level distillation, system-level asynchronous overlap, and implementation-level optimization yields up to 3.79× cumulative effective-cycle speedup.
🔄 Streaming-WAM overlaps each Stream Update with the execution of the current action chunk. Shared actions provide temporal continuity across adjacent chunks and condition the next visual prediction, aligning world prediction with the robot’s ongoing motion.
⚙️ Real-time WAM acceleration benefits from full-stack optimization. Combining model-level distillation, system-level asynchronous overlap, and implementation-level optimization yields up to 3.79× cumulative effective-cycle speedup.
🖐️ Enabling Robots to Perceive Contact Changes Mid-Action! TacForcing: A streaming action-generation framework that incorporates execution-time tactile feedback into VLA policies. 📄 Paper: arxiv.org/abs/2608.25798 🌐 Project page & demos: 88runaway.github.io/tacforcing/
Robots that continuously perceive contact changes mid-action — introducing TacForcing, a streaming action generation framework built for contact-rich dexterous manipulation tasks. Special thanks to Sharpa Robotics for their systematic support. Paper releasing soon — stay tuned!
Wave Forcing is now open source 🌊 To our knowledge, it is the world’s fastest streaming video generation system: 117.7 E2E FPS (126.5 steady-state), 7.9× faster on 8×H200, and ~7.5× at 14B. Code + preview: sjtu-deng-lab.github.io/WaveForcing/ #videogeneration
Comparison with commercial models Compare with Kling 2.5 Turbo, Hailuo 2.3, Vidu Q3, and Wan 2.7: CineMobile keeps the subject sharp and the camera path stable across the full sequence, while several commercial models drift or blow out the scene at later frames.
CineMobile: On-Device Image-to-Video Generation for Structured Camera Motion 📱 We introduce CineMobile, an efficient on-device image-to-video generation framework for structured camera-motion videos. arxiv: arxiv.org/abs/2607.03803
VBench quantitative comparison At 1.2B params and only 4 steps, CineMobile scores 88–89 on VBench across bullet time, dolly zoom, and slow motion — within ~0.5-0.9 points of the 14B, 20-step Wan2.1 teacher. 10% of the parameters, 20% of the steps, nearly all of the quality.
Qualitative results across effects Sniper stakeouts, guzheng by the lake, street style, motocross jumps, hurdles — CineMobile holds subject identity and camera trajectory steady across wildly different scenes and motion effects, all generated on-device in 4 denoising steps.
Method: SFT warm-up + step distillation pipeline How we got there: depth-pruned DiT → supervised fine-tuning warm-up → adversarial 4-step distillation with a real/fake score pair (DMD) plus a discriminator feeding GRPO rewards.
Efficiency: 40.11x speedup From 97 seconds to 2.4 seconds. We took a 14B teacher DiT and compressed the whole pipeline — pruning, 4-step distillation, hybrid quantization — down to a 1.2B model, while running at ~20s/step on a MediaTek Dimensity 8400 phone.
wuyuan @WuuYuan
1 Followers 97 Following
Owen Ma @owen_ma4573
0 Followers 118 Following
Christian Lin @iamChristianLin
0 Followers 64 Following A ML engineer working on vision-language-action models and world models — systems that build an internal picture of a scene and use it to decide what to do next
lccurious @lccurious0
3 Followers 342 Following
维 @YscbIOxOU0M0F8J
52 Followers 2K Following PhD student @Tsinghua_Uni | Prev. USTC | Intern @Huawei @ByteDanceTalk @MSFTResearch
Mikan Affine @MikanAffine
6 Followers 153 Following
Keyu Zhu @KeyuZhu19
36 Followers 525 Following
Xie Haoling @XieHaoling
0 Followers 14 Following
boyuan zhao @ZhaoBoyuan20091
0 Followers 4 Following
Xxxia @Xxxia20
0 Followers 19 Following
the1eyecat @champly
105 Followers 660 Following
Kewei @Keweihannj
0 Followers 37 Following
robotyj @RobotYinjie
0 Followers 188 Following
Hai @Hai55813852
28 Followers 3K Following
Alex Kotler @realalexkotler
45 Followers 583 Following
Łazzar @LM_22
304 Followers 2K Following Crypto investor, Big Data, BI enthusiast, Digitalisation Consultant, Entrepreneur. #Bitcoin, Ł,
autumn & winter ❄�... @V8DiI
7 Followers 719 Following robotics & embodied intelligence , aviation & space
Kmoto @kuwelz
1K Followers 7K Following Only straight to the point 🍟 Feel free to follow 🔁 and repost Currently in Fukuoka, occasionally back in China, love tech, love robots, love selfies, love BTC
Runnan Xu @xxrrnn2019
2 Followers 242 Following
Wenxin Chen @WenxinChen06
4 Followers 203 Following undergrad cs @umich | ece @sjtu1896 | multimodality, representation learning, embodied ai, and bci.
interstater @interstarter
26 Followers 864 Following I could be bounded in a nutshell, and count myself a king of infinite space
Kyoung Whan Choe @kywch500
1K Followers 2K Following Robot Learning Engineer @ https://t.co/wcLx79rCuW
AngraMainyu @angra_main10235
0 Followers 2 Following
Xiaolin Hu @xiaulinhu
10 Followers 831 Following
Ankur Bohra @AnkurBohra9
66 Followers 6K Following
Yuzheng Zhang @Yuzheng_Z
0 Followers 35 Following
Yuxuan Liu @YuxuanL34893
0 Followers 2 Following
Jiaqi Leng @JiaqiLeng171839
0 Followers 130 Following
Sad Hacker @Sadhac151
13 Followers 1K Following
Tkee @Tkee7491
2 Followers 1K Following
Zhongzhu Zhou @ZhongzhuZhou
76 Followers 353 Following Senior Research Scientist @togethercompute | Ph.D. CS @Sydney_Uni B.Eng.(Hons) CS @SunYatsen_Uni | Ex. @Dolby @Microsoft @Tencent #ComputerScience #AI #MLSys
xdnsgzs5ig6674 @xdnsgzs
0 Followers 1K Following
VegetaAvatar @VeGeTaX29
20 Followers 7K Following
Yu Lei @_OutofMemory_
742 Followers 3K Following PhD student @UTCompSci | Learn to understand ourselves and build intelligence.🤖🧠👁️
Men1scus @Men1scus
243 Followers 5K Following CS @ Nankai University | Founding Engineer @ an AI startup | Side research: ImageGen · Data Infra | Rap enthusiast | 🏃🚴🏊🏋️🧗
Louis @Louis9687221579
106 Followers 4K Following Mainline Economics | Idea page | ramblings of a schizo
Zetong Zhou @ls2kelvin
1 Followers 6 Following An incoming PhD of Shanghai Jiaotong University. I study generative models.
ft.eth @tractusF
65 Followers 2K Following
yingying zhu @yZ31960667
162 Followers 2K Following Professor at Chinese Academy of Sciences. Building Science as next frontier benchmark of general intelligence.
Keyu Li @chlorophyllwzh
19 Followers 258 Following CS-AI Phd @sjtu1896 LLM Post Training, Multi-Agent System
Yicun Yang @maomaocun
9 Followers 490 Following A Research Intern at EPIC Lab, SJTU. 26 fall hkust-gz phd. my Google Scholar is here ! https://t.co/7NnMYheuht
Sharpa @SharpaRobotics
6K Followers 199 Following Sharpa is an AI robotics company dedicated to developing ultra-high performance robots and core components. We Manufacture Time by Making Robots Useful.
Dacheng Li @DachengLi177
2K Followers 1K Following v2 | PhD @BerkeleySky, @berkeley_ai @lmsysorg | Prev: @Nvidia @SCSatCMU
Qubitium @qubitium
2K Followers 4K Following GPT-QModel, QvQ ultra low bit quants. Contributor to stuff you are probably using like sglang, vllm, transformers... Seeking compute sponsorships. Dm me.
ICLR @iclr_conf
62K Followers 57 Following International Conference on Learning Representations #ICLR2027. SPC is @jacobandreas and GC is @BharathHarihar3
Aryan V S @aryanvs_
3K Followers 2K Following
Walter Hugo Lopez Pin... @Warvito
1K Followers 694 Following Senior Research Engineer @synthesiaIO | Ex-Research Fellow @KingsCollegeLon Text-to-Video | Generative Models | Medical Imaging
Robert Scoble @Scobleizer
606K Followers 59K Following The automated life | San Francisco/Silicon Valley AI, robotics, BCIs | Ex-Microsoft, Rackspace, Fast Company | Wrote eight books about the future.
Skywork @Skywork_ai
9K Followers 24 Following Skywork 👉 https://t.co/HyxhwfV5Ji 🎨 Skywork Design: https://t.co/YwPa4sTu49 🎬 Skywork Video: https://t.co/1vP5eY2zrb
Lianmin Zheng @lm_zheng
17K Followers 667 Following Inference @meta | Prev: Engineer @xAI, Ph.D. @UCBerkeley, Co-founder @lmsysorg
Shibo Hao @Ber18791531
2K Followers 1K Following @thinkymachines | Prev. @UCSanDiego @PKU1898 @AIatMeta @Bloomberg Ph.D. Fellow 🥥Coconut, 🎶Reasoning-via-planning, 🛠️ToolkenGPT
Felix Juefei Xu @felixudr
2K Followers 7K Following Research Scientist @GoogleDeepMind ♊️ | Prev. @Meta Superintelligence Labs | PhD @CarnegieMellon | Views are my own | Journey to the Physical Superintelligence!
Shiyi Cao @shiyi_c98
2K Followers 804 Following llm and system | PhD student @UCBerkeley @BerkeleySky, MSc @ETH, B.S @sjtu1896 | Intern @nvidia
Willie Neiswanger @willieneis
2K Followers 298 Following Assistant Professor @USC in CS + AI. Previously @Stanford, @SCSatCMU. Machine Learning, Decision Making, AI-for-Science, Generative Models.
David @DavidSHolz
121K Followers 11K Following founder @midjourney, previously founded leap motion, before that was at nasa and max planck - vibeposting @davidvibesonly
Ben Schulz @schulzb589
2K Followers 8K Following 3D Geospatial Analyst at Vantor Space Operations. Opinions expressed on this site are my own and do not necessarily represent the views of Vantor.
You Jiacheng @YouJiacheng
13K Followers 2K Following
Nicolas Mejia Petit @mejia_petit
745 Followers 238 Following LLM researcher// Made Tested python 22k and 143k datasets, first Mixtral 22b MOE to dense model, open-critic-gpt, code-DPO//https://t.co/bghSLSzZ0j
Koustuv Sinha @koustuvsinha
4K Followers 810 Following Research Scientist @MetaAI; PhD from @mcgillu + @Mila_Quebec; Research in Multimodal World Models; Organizer of ML Reproducibility Challenge (@repro_challenge).
Drew Jin @drew_jin_0827
34 Followers 78 Following Ph.D. Student @SJTUDengLab & @sjtu1896 | Research Intern @Alibaba_Qwen | MLSys & Efficient GenAI Researcher
Lou @lcheviron
224 Followers 3K Following Lead your life, father, entrepreneur. human digital twins builder
Chaoyue He @CYH37
408 Followers 4K Following Research Scientist@Alibaba-NTU Global e-Sustainability CorpLab(ANGEL)@NTUsg🇸🇬🇨🇳|LLM|AGI|ESG|RL|CL|RecSys|KG|AI4X|😌|Xi'an|Bodybuilder|Caregiver
Yixin Lin @yixin_lin_
2K Followers 7K Following something new. prev: embodied AI @GoogleDeepMind, FAIR/@AIatMeta, Google Brain.
Lanxiang Hu @Lanxiang_Hu
440 Followers 364 Following Efficient AI, PhD @UCSDJacobs @haoailab, Intern @NVIDIA Cosmos Lab.
Zhijie Deng @ZhijieDeng2
18 Followers 45 Following
DeepSeek @deepseek_ai
1.1M Followers 1 Following Unravel the mystery of AGI with curiosity. Answer the essential question with long-termism.
Chuang Gan @gan_chuang
10K Followers 501 Following Faculty Member at UMass Amherst; Principal researcher at MIT-IBM Watson AI Lab; Homepage: https://t.co/Pc8WeREfTz
Hao AI Lab @haoailab
8K Followers 380 Following Hao AI Lab at UCSD. Our mission is to democratize large machine learning models, algorithms, and their underlying systems.
Xiang Yue @xiangyue96
7K Followers 937 Following Superintelligence @Meta. Training & evaluating foundation models. Previously @LTIatCMU @osunlp. Opinions are my own.
Unsloth AI @UnslothAI
99K Followers 479 Following Run and train models locally with the Unsloth Desktop app. 🦥 https://t.co/2kXqhhvdCD
Jason Kong @JasonkongLLM
115 Followers 156 Following Researching LLM inference effiency @ucsandiego HW/SW co-design
Haibin @eric_haibin_lin
2K Followers 1K Following On the path to AGIs. Bytedance Seed LLM Systems, @verl_project, Megascale. Prev. @AmazonScience @ApacheMXNet
Hangliang Ding @_foreverpiano
245 Followers 2K Following Code is cheap. Show me the prompt @ByteDance Seed | Grad @Tsinghua_Uni | Opinions of my own
Yichao Fu @FuYichao123
298 Followers 511 Following building efficinet AI @UCSD (PhD) & @Meta (intern) 🚀 | CS undergrad @ZJU_China | chasing ideas that make machines reason
Siqi Zhu @realagi25
670 Followers 524 Following Code is cheap, show me the prompt | 牛来 | 大结果 | CS phd @UofIllinois | prev CS @Tsinghua_Uni, @BytedanceTalk @UCSanDiego @zai_org
Tianyu Pang @TianyuPang1
2K Followers 350 Following Tech Lead of Multimodal RL at @TencentHunyuan | 👨🏻🎓PhD/BS from @Tsinghua_Uni | ex-@SeaAIL
Tri Dao @tri_dao
45K Followers 664 Following Asst. Prof @PrincetonCS, Chief Scientist @togethercompute. Machine learning & systems.
Yuandong Tian @tydsh
48K Followers 952 Following Co-founder of @Recursive_SI. ex-Meta FAIR Director. ex-Google. Reasoning, Optimization and Understanding LLM. Novelist in spare time. PhD in @CMU_Robotics.You might like
