Andrew Rouditchenko ๐บ๐ฆ @arouditchenko
Research scientist at NVIDIA working on speech and multi-modal LLMs. Previously PhD student at MIT CSAIL and intern at @AIatMeta and @Apple MLR. people.csail.mit.edu/roudi/ Joined December 2016-
Tweets224
-
Followers466
-
Following572
-
Likes2K
@adamcohenhillel Nice demo! Lip reading has a lot of potential after being personalized to a specific user. You could consider using audio-visual lip reading methods like Whisper-Flamingo, LLaMA-AVSR, etc... for better recognition when the lips aren't clearly visible.
How do AVSR models balance what they hear and what they see? Introducing Dr. SHAP-AV, the first large-scale Shapley-based analysis of modality contributions in audio-visual speech recognition. 6 sota models 2 benchmarks 3 analyses ๐Project page: umbertocappellazzo.github.io/Dr-SHAP-AV/ ๐งต๐
@rdesh26 Thanks for sharing!
Do you really need audio to fine-tune your Audio LLM? ๐ค Answer below: Introducing Omni-R1, a simple GRPO fineโtuning method for Qwen2.5โOmni on audio question answering. It sets new stateโofโtheโart accuracies on the MMAU benchmark for Audio LLMs. arxiv.org/abs/2505.09439
๐ ๐ซ๐จ๐ฆ ๐ฎ๐ง๐๐ฆ๐ฉ๐ฅ๐จ๐ฒ๐๐๐ฅ๐ ๐ฆ๐๐ญ๐ก ๐ฎ๐ง๐๐๐ซ๐ ๐ซ๐๐ โ ๐ญ๐จ ๐,๐๐๐ ๐๐ข๐ญ๐๐ฎ๐ ๐ฌ๐ญ๐๐ซ๐ฌ & ๐ ๐ซ๐๐ฌ๐๐๐ซ๐๐ก ๐ฌ๐๐ข๐๐ง๐ญ๐ข๐ฌ๐ญ ๐จ๐๐๐๐ซ๐ฌ (๐๐๐, ๐๐ญ๐.) ๐My journey of doing ๐๐ก๐ ๐ข๐ง ๐๐: tinyurl.com/5n7b7v36
@anmol01gulati Congrats!
๐๏ธ Another #MultimodalAI workshop we are organizingโthis one zeroes in on speech & language foundation models! ๐ Dive into #SpeechAI, audio, and language tech. Learn how to build foundation models and hear from both academia and industry experts. ๐Sep 4โ5, 2025 | @TTIC_Connect
๐ข Excited to announce our 2-day workshop on "Foundations of Speech and Audio Foundation Models" at TTI Chicago, happening September 4โ5! ๐ Info & registration: sites.google.com/view/speech-aiโฆ ๐ Poster submissions welcome! Join us for talks, discussions, and community building!
@chenwanch1 Your rebuttal leaves them speechless
If your PhD advisor dressed like this, you probably didn't use neural nets in your thesis
Finally, after all these years of being mocked, ffmpeg enthusiasts win!
๐กBridging speech, sound, & music representations with one universal model? We introduce USAD โ ๐ Distills knowledge from domain-specific SSL models ๐ฏ Matches expert models across speech/audio/music tasks ๐ arxiv.org/abs/2506.18843 ๐งโ๐ป huggingface.co/MIT-SLS/USAD-Bโฆ
Learn to figure out what is worth figuring out: kamperh.com/2025/06/20/knoโฆ
@nizumical +another benchmark that merges them into one benchmark ๐
Congrats to Edson for leading our Contrastive Audio-Visual Masked Autoencoders 2.0 Project (CAV-MAE Sync), accepted at #CVPR2025! Check out Edson's thread for more details โฌ๏ธ
๐ Excited to announce our #CVPR2025 paper: CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment! We introduce a simple yet effective method for improved audio-visual learning. ๐ Project: edsonroteia.github.io/cav-mae-sync/ ๐งต (1/7)๐
Link the MMAU leaderboard (Massive Multi-Task Audio Understanding and Reasoning Benchmark) - it should hopefully be updated soon with Omni-R1 sakshi113.github.io/mmau_homepage/โฆ
Code, models, and datasets on the way! Thank you to my co-authors: Saurabh Bhati, @edsonroteia, Samuel Thomas, @HildeKuehne, @RogerioFeris, James Glass. Collaboration through @MITIBMLab
Granite-speech audio LLM from IBM. The level of data detail here is great especially comparing to ie. Whisper paper
``Granite-speech: open-source speech-aware LLMs with strong English ASR capabilities,'' George Saon, Avihu Dekel, Alexander Brooks, Tohru Nagano, Abraham Daniels, Aharon Satt, Ashish Mittal, Brian Kingsbury, David Haws, Edmilson Morais, Gakuto Kurata, Haโฆ ift.tt/QPsxkH2
Piotr ลปelasko @PiotrZelasko
2K Followers 742 Following AI + Speech @ Nvidia. PhD @ AGH-UST, ex-JHU. My interests: speech processing technology; ML/AI software engineering. Building OSS for Speech AI.
g @g9665926721001
69 Followers 3K Following
Xie Zhifei @XieZhifei14110
586 Followers 217 Following PhD @NUS & NTU, advised by Prof. Shuicheng Yan / Mini-Omni series (1st real-time speech LLM) ยท Audio-Reasoner ยท Mega-ASR / [email protected] Proactive AI Intelligence
Yi Zhong @yiz_be_building
478 Followers 1K Following Building @besimple_ai (YC P25) to make AI listen to you, ex Meta / Dropbox / MSFT Product Lead from MIT
mizero joseph @JosephMizero
80 Followers 628 Following Biotechnologist| Documentary filmmaker|Politics and social affairs Commentator|Service-oriented not profit-motivated|Rwandan Patriot, Pan-Africanist|RWANDA 1st
Liming Wang @LimingWang297
2 Followers 77 Following
Jonah Casebeer @CasebeerJonah
253 Followers 423 Following Senior Researcher @PhysicalAI | Ph.D. @IllinoisCS | personal account
UT_SALT_Lab @utsaltlab
8 Followers 30 Following Speech, Audio, and Language Technologies (SALT) Lab at UT Austin, led by Prof. David Harwath https://t.co/KMu9WAywfW
Vadim Smolyakov @vsmolyakov
265 Followers 489 Following Machine Learning Engineer at @Microsoft. AI/ML Researcher. Author. @MIT_CSAIL and EngSci @UofT alum.
Last Neuron @LastBrainNeuron
5 Followers 384 Following MTech Research student @iiscbangalore Education brain rot maxxing I will treat this space as my diary I'll be logging updates.
jerad @jfields513
117 Followers 5K Following
Sophia roth @roth_sophi35680
68 Followers 503 Following
Philip Schroeder @philip_mit
764 Followers 696 Following PhD student at MIT in Computer Science. @MIT_CSAIL @MITEECS @nlp_mit
Zhengyang Qi @qi_zhengyang
966 Followers 6K Following Research @SnorkelAI | Previously: @Scale_AI Multiturn RL, reward modeling, interactive environments, artificial social intelligence
Chris Glaze @chris_m_glaze
2K Followers 5K Following Principal Research Scientist at @SnorkelAI. PhD in computational neuroscience. Previously: @penn @UofMaryland
Ondลej Plรกtek @oplatk
202 Followers 1K Following #NLProc #speech researcher, father, climber. One thing at a time.
Shih-Lun (Sean) Wu @slseanwu
303 Followers 157 Following music/audio/speech proc, generative models PhD student (now), EECS MIT MSc '24, SCS CMU BSc '21, CS Nat'l Taiwan U casual classical pianist๐น & violist๐ป
Sean Kernitsman @Skernitss
4 Followers 83 Following
Chao Huang @chaooh_
13 Followers 166 Following I like to let model teach me. Research Scientist @Tencent Hunyuan Seattle Been at @Meta RL, @AMD GenAI | PhD @UofR
Ruoujo @Ruoujo641
184 Followers 7K Following The question isnโt whoโs going to let me; itโs whoโs going to stop me.
Robert Scoble @Scobleizer
595K Followers 54K Following San Francisco/Silicon Valley AI | Robots, holodecks, BCIs, analysis of new things | Ex-Microsoft, Rackspace, Fast Company | Wrote eight books about the future.
Noah Ziems @NoahZiems
4K Followers 2K Following MTS @PrimeIntellect. Prev @MIT_CSAIL under @lateinteraction, PhD @NotreDame Pedagogical RL, DSPy
Akshit Singh @akshit_fbd
25 Followers 532 Following To believe in something is to disbelieve in something
Samir Char @Samir_char
109 Followers 470 Following - Data Scientist @Microsoft - MS in Data Science @Columbia University - Runner, weightlifter - I love reading books
ivan @ivillar00
102 Followers 1K Following latent space mechanic | experimenter in deep learning and musical audio | Ableton Live enjoyer | prev. cs @stanford, @StanfordAILab
Saniey Gassama @GassamaSan89620
40 Followers 2K Following hello greetings friend's and family i need help with my family โฅ๏ธ
ๆ็ฟ @xiangli2025cuhk
0 Followers 50 Following
Jason Liu @liuzhan2204
3 Followers 340 Following
Gokul @gokulkarthikk
117 Followers 726 Following
Assel Yermekova @alless_yer
12 Followers 163 Following I am ML Researcher. Have 10+ years in science, 4 years in ML. Interested in: Speech, Generative models.
Anne Wu @anne_youw
249 Followers 220 Following learning from interactions. CS PhD student @cornell_tech
IEEE Speech and Langu... @ieeesltc
157 Followers 62 Following Advancing Language and Speech Processing
Rotnodip Sarkar @rotnodip
11 Followers 794 Following
Hanna Yukhymenko @a_yukh
694 Followers 519 Following post-training apertus @eth_ai_center | ex-๐ค @eth | EEU ambassador @insaitinstitute
RMO Digital @RMODigital
101 Followers 7K Following
EvoAgentX @EvoAgentX
174 Followers 490 Following Building a Self-Evolving Ecosystem of AI Agents. Join discord for discussion: https://t.co/XkaQdgwzFh
Jehanzeb Mirza @jmie_mirza
388 Followers 311 Following i run and do computer vision and play with languages sometimes. postdoc at @MIT_CSAIL
Julian Tsin @JianhuaiQin
4 Followers 505 Following Master of CS @UESTC1956. Looking forward for Ph.D. opportunity in areas of #LLM #Agent #RL in 27 Fall. Feel free to DM me.
Matan Gover @matangover
540 Followers 1K Following Music/audio ML researcher, musician, software engineer
Eduardo Ramson @iarevolutionbr
2 Followers 175 Following
Stephen McConnachie @mcnatch
1K Followers 5K Following Head of Data and Digital Preservation at the BFI National Archive. Working on digital preservation, audiovisual archiving, using AI with care. Opinions my own.
Xuanjun (Victor) Chen @xjchen_ntu
43 Followers 321 Following PhD Candidate, NTU EECS (@NTU_TW). Now: @ntu_spml, advised by @HungyiLee2.
Jom Kuriakose @jomlive
280 Followers 2K Following
Yi Zhong @yiz_be_building
478 Followers 1K Following Building @besimple_ai (YC P25) to make AI listen to you, ex Meta / Dropbox / MSFT Product Lead from MIT
Xie Zhifei @XieZhifei14110
586 Followers 217 Following PhD @NUS & NTU, advised by Prof. Shuicheng Yan / Mini-Omni series (1st real-time speech LLM) ยท Audio-Reasoner ยท Mega-ASR / [email protected] Proactive AI Intelligence
Ruixiang Zhang @onloglogn
717 Followers 880 Following Research @Apple MLR, PhD @Mila_Quebec. Prev @GoogleDeepMind @preferred_jp
Liming Wang @LimingWang297
2 Followers 77 Following
Sebastian Raschka @rasbt
486K Followers 1K Following ML/AI research engineer. Ex stats professor. Author of "Build a Large Language Model From Scratch" (https://t.co/O8LAAMRzzW) & reasoning (https://t.co/5TueQKx2Fk)
Igor Babuschkin @ibab
110K Followers 886 Following CEO & Co-Founder @river_ai_inc. Previously @xAI, Research & Engineering
Maxime Labonne @maximelabonne
33K Followers 543 Following Head of Post-Training @liquidai ๐ค HF: https://t.co/2ECS7Gjhwb ๐ Blog: https://t.co/Dxjebt3GZW
Vadim Smolyakov @vsmolyakov
265 Followers 489 Following Machine Learning Engineer at @Microsoft. AI/ML Researcher. Author. @MIT_CSAIL and EngSci @UofT alum.
Jonah Casebeer @CasebeerJonah
253 Followers 423 Following Senior Researcher @PhysicalAI | Ph.D. @IllinoisCS | personal account
Armin PCM @ArminPCM
2K Followers 6K Following ๐โ๏ธ Javid Shah โ๏ธ๐ PhD | RL, Eval, & Benchmarking
Zhengyang Qi @qi_zhengyang
966 Followers 6K Following Research @SnorkelAI | Previously: @Scale_AI Multiturn RL, reward modeling, interactive environments, artificial social intelligence
Chris Glaze @chris_m_glaze
2K Followers 5K Following Principal Research Scientist at @SnorkelAI. PhD in computational neuroscience. Previously: @penn @UofMaryland
Natasha Jaques @natashajaques
34K Followers 1K Following Assistant Professor leading the Social RL Lab https://t.co/ykwfJG84Bj @uwcse and Staff Research Scientist at @GoogleAI.
Rafael Rafailov @ Neu... @rm_rafailov
8K Followers 914 Following I work on RL at @thinkymachines. Previously at @StanfordAILab @GoogleDeepMind @UCBerkeley
Nathan Lambert @natolambert
96K Followers 927 Following Open model research @ something new. Prev. co-led Olmo at Ai2. Contact via email. Writes @interconnectsai Wrote The RLHF Book, ๐๏ธ๐โโ๏ธ
Ondลej Plรกtek @oplatk
202 Followers 1K Following #NLProc #speech researcher, father, climber. One thing at a time.
Vishaal Udandarao @vishaal_urao
1K Followers 1K Following @ELLISforEurope PhD Student @bethgelab; Currently @Apple; Previously @GoogleAI @GoogleDeepMind @Cambridge_Uni @RutgersU @iiitdelhi
Sean Kernitsman @Skernitss
4 Followers 83 Following
Donald S. Williamson @TheASPIREGrpOSU
112 Followers 134 Following I am an associate professor in the Department of Computer Science and Engineering at The Ohio State University. I lead The ASPIRE Group.
ivan @ivillar00
102 Followers 1K Following latent space mechanic | experimenter in deep learning and musical audio | Ableton Live enjoyer | prev. cs @stanford, @StanfordAILab
Samir Char @Samir_char
109 Followers 470 Following - Data Scientist @Microsoft - MS in Data Science @Columbia University - Runner, weightlifter - I love reading books
Mathias Lechner @mlech26l
2K Followers 221 Following Cofounder/CTO at Liquid AI and Research Affiliate MIT
Luke Whipps @luke_whipps
66 Followers 423 Following Talent Lead at Liquid AIโฆbuilding the team who are building LFMs. Donโt know what LFMs are? click here: https://t.co/M4pd4P14dy
Ioannis (Yiannis) Tsi... @JohnTsiamas
208 Followers 472 Following AI Research Scientist | PhD at UPC Barcelona @la_UPC | ex-intern @AIatMeta, @Apple, @Dolby
Che-Ping Tsai @chepingt
169 Followers 672 Following Building Eval for AI @ @arena | AI PhD @ @mldcmu
Xueyao Zhang @xueyao_98
123 Followers 147 Following Ph.D. student at the Chinese University of Hong Kong, Shenzhen. Amphion co-founder. Interested in speech, music, and multi-modality.
Alexander Wei @alexwei_
25K Followers 214 Following Reasoning @OpenAI. Co-built CICERO @MetaAI | @Berkeley_AI PhD '23 | @Harvard '20
Anne Wu @anne_youw
249 Followers 220 Following learning from interactions. CS PhD student @cornell_tech
IEEE ASRU @ieee_asru
50 Followers 2 Following IEEE ASRU, is focused on bringing together academia and industry to discuss new developments on the field of Automatic Speech Recognition and Understanding.
IEEE Speech and Langu... @ieeesltc
157 Followers 62 Following Advancing Language and Speech Processing
Tae-Hyun Oh @Tae_Hyun_Oh
82 Followers 118 Following Associate professor @ KAIST Former postdoc researcher @ Facebook AI Research Former postdoc associate @ MIT CSAIL
Hanna Yukhymenko @a_yukh
694 Followers 519 Following post-training apertus @eth_ai_center | ex-๐ค @eth | EEU ambassador @insaitinstitute
Yuchen Jin @Yuchenj_UW
230K Followers 812 Following Cooking fun AI systems & products @databricks. Prev: co-founder & CTO @ Hyperbolic, OctoAI (acquired by @nvidia) Apache TVM, PhD @ University of Washington.
Jehanzeb Mirza @jmie_mirza
388 Followers 311 Following i run and do computer vision and play with languages sometimes. postdoc at @MIT_CSAIL
Matan Gover @matangover
540 Followers 1K Following Music/audio ML researcher, musician, software engineer
Atsumoto Ohashi @atsumoto_ohashi
2K Followers 249 Following Postdoc at Kyutai. My research interests lie in spoken dialogue systems, RL, and NLP.
Xuanjun (Victor) Chen @xjchen_ntu
43 Followers 321 Following PhD Candidate, NTU EECS (@NTU_TW). Now: @ntu_spml, advised by @HungyiLee2.
Jesse Engel @jesseengel
9K Followers 65 Following Guitarist, Researcher Google DeepMind. Opinions are my own. @jesseengel.bsky.social @[email protected]
Lingwei Meng @LingweiMeng
70 Followers 169 Following Research Intern @ Microsoft | Ph.D. Candidate @ CUHK | Working on speech things.
Unsloth AI @UnslothAI
79K Followers 479 Following Train and run models locally! ๐ฆฅ https://t.co/2kXqhhvLsbYou might like
