Javier Nistal @latentspaces
Researcher @SonyCSLMusic. Former PhD @TelecomParis_ | @MIDASconsoles | Intern @JukedeckRandD, @SoundCloud. Exploring deep generative models for sound and music linkedin.com/in/javiernista… Paris Joined August 2020-
Tweets146
-
Followers1K
-
Following568
-
Likes442
Final push! We need a few more participants for a 15-minute listening test: golisten.ucd.ie/task/mushra-te… Already helped and coming to #ISMIR? Beer on me :)
Listening test alert 🚨 we need you! 😊 Super simple music denoising test: listen to a few piano clips and rate their closeness to a reference. 15 minutes for you, tons of help for us :) Headphones recommended. Computer needed. lnkd.in/d6t8UzaC Thanks 🫶
Listening test alert 🚨 we need you! 😊 Super simple music denoising test: listen to a few piano clips and rate their closeness to a reference. 15 minutes for you, tons of help for us :) Headphones recommended. Computer needed. lnkd.in/d6t8UzaC Thanks 🫶
🎶 New ISMIR 2025 paper! "Estimating Musical Surprisal from Audio in Autoregressive Diffusion Model Noise Spaces" by Mathias Rose Bjare (JKU), Stefan Lattner (Sony CSL), and Gerhard Widmer (JKU/LIT AI Lab). We explore how surprisal — the unexpectedness of musical events — can be modeled directly from audio using autoregressive diffusion models (ADMs). 💡 What we did: - Compared surprisal from diffusion models vs. Generative Infinite-Vocabulary Transformers (GIVT). - Evaluated across tasks: monophonic pitch surprisal (expectation) & segment boundary detection (structural surprise). - Tested surprisal at different noise levels in diffusion processes to see how musical features emerge at multiple granularities. 🔥 Key take-aways: - Diffusion surprisal beats GIVT in modeling pitch expectation & boundary detection. - Mid-level noise surprisal captures pitch-level expectations while suppressing timbre-related variation. - Surprisal curves align with human-like musical segmentation, showing potential as proxies for perceptual surprise. Why it matters: Understanding musical surprisal links computational modeling with human perception and cognition — with applications in AI composition, real-time music interaction, and brain-music studies. 📜 Paper: arxiv.org/abs/2508.05306 💻 Code: github.com/SonyCSLParis/a… #DiffusionModels #Surprisal #MIR @ISMIRConf @SonyCSLParis @SonyCSLMusic
❌ We don’t need no negative samples ❌ We don’t need no large batches ❌ No modality gap in the classroom Very happy to introduce SLAP, our latest brick in the wall of multimodal SSL 🎶🧠 Joint work with King @Juj_Guinot, accepted at #ISMIR2025! 🇰🇷 1/7
Hey! Our paper, 🌸 “LiLAC: A Lightweight Latent ControlNet for Musical Audio Generation” got accepted as ISMIR 2025! It presents a novel control paradigm for audio diffusion models with greatly reduced parameters, encouraging users to train individual, modular models.
🎭💻 We’re participating in #CultTech Residency — a new program supporting artists and creative teams exploring performance & technology Experience the benefits of working with our AI-powered tool #DiffARiff Feel free to share the news 🔽 Open call: culttech.at/residence-open…
@dadabots I agree that it feels hacky. But imagine it real-time, with many conditioning, each with its own CFG... that should be extremely fun :D
🌟 New @ieeeICASSP Paper Announcement 🌟 We introduce "Music2Latent2: Audio Compression with Summary Embeddings and Autoregressive Decoding", a novel autoencoder achieving higher fidelity at extreme compression rates using consistency models & summary embeddings. 🧵
😃Accepted @ieeeICASSP papers of @SonyCSLMusic: Accompaniment Prompt Adherence: A Measure for Evaluating Music Accompaniment Systems M. Grachten, J. Nistal Estimating Musical Surprisal in Audio M. Bjare, G. Cantisani, S. Lattner and G. Widmer Hybrid Losses for Hierarchical Embedding Learning H. Tian, S. Lattner, B. McFee, C. Saitis Music2Latent2: Audio Compression with Summary Embeddings and Autoregressive Decoding M. Pasini, S. Lattner, G. Fazekas Zero-shot Musical Stem Retrieval with Joint-Embedding Predictive Architectures A. Riou, S. Lattner, A. Gagneré, G. Hadjeres, S. Lattner, G. Peeters Congrats to the authors! @latentspaces @howariou @GiorgiaCanti @tiianhk @marco_ppasini @GeoffroyPeeters @gaetan_hadjeres @SonyCSLParis
💡If you missed it, we released our new AI-tool, #DrumGAN, which allows you to generate flexible drum sounds with ease Here is a full blogpost for a detailed look at it + free download the 1st #AI-drum kits made by #Twenty9 during our collaboration ➡️🎁 tinyurl.com/blpodr
@john_c_vinyard @deeplearnmusic @SonyCSLMusic Perhaps @cyranaouameur can throw some light here
Happy to announce that our legendary DrumGAN, developed 2020 by Javier Nistal @latentspaces is now available as a web application. Free to use and without registration! 🥁 Give it a try at drumgan.csl.sony.fr #DrumGAN @SonyCSLMusic
🥳 New publication announcement! Marco Pasini solved the problem of error accumulation in continuous autoregressive models (CAMs), making it possible to generate sequences without the need for prior tokenization. Say goodbye to RVQ codecs (use music2latent 😉). @SonyCSLMusic
✨ Train language models directly on continuous data - without tokenization ✨ We propose an easy way to train GPT-style autoregressive models on continuous data, without error accumulation. We test it on audio 🔊, but this method can easily work with other modalities 🎆 👇🧵
🌟 Exciting news! The YouTube channel @aisearchio just released a review of #DiffARiff, and it's already making waves—10k views in less than 12 hours and more than 100 comments with fantastic feedback! 🎉 Thanks so much for the incredible exposure!
🎶Current AI music generators like Udio & Suno lack control—they spit out entire songs for you, which isn’t always ideal. But now, there's a game changer for music producers! An AI that generates individual stems for any instrument. Here's a sneak peek! youtu.be/dQJZxPyI5l8
🤩 It's exciting to see how Diff-A-Riff makes it easy to build up a song based on some input audio! 👇
🎉Continue to discover #DiffARiff capabilities New #HipHop demo by @CarliNistal showcasing how #DiffARiff can seamlessly integrate into your workflow, offering flexibility for musical exploration Using #textprompts & #referenceaudio Carli guided the AI to generate this new demo
🚨 New Video Drop! 🚨 My brother, Carli Nistal, used Diff-A-Riff to create this banger! 🎶🤖 Watch it in action!
🎉New #DiffARiff audio demo 🎚️We continue to introduce you to Diff-A-Riff's capabilities, our accompaniment co-creation tool, with this new #Disco Demo produced by @CarliNistal 👏 Find the HQ video & audio on our YouTube channel #aimusic #research #innovation #sonycslparis
🎉New #DiffARiff audio demo 🎚️We continue to introduce you to Diff-A-Riff's capabilities, our accompaniment co-creation tool, with this new #Disco Demo produced by @CarliNistal 👏 Find the HQ video & audio on our YouTube channel #aimusic #research #innovation #sonycslparis
Christian Steinmetz @csteinmetz1
6K Followers 2K Following Research Scientist @ Suno // working on generative music • audio fidelity • signal processing • ML
Pedro Sarmento @umpedronosapato
3K Followers 2K Following AI & Music Data Scientist @moises_ai | prev. @c4dm
Oriol (Uri) Nieto @urinieto
3K Followers 1K Following (INACTIVE ACCOUNT, FIND ME ON LINKEDIN/BLUESKY). Researcher at Adobe Research. Machine learning on audio. Oaklander born in Barcelona. Titan. He/they 🌈
Ilaria Manco @Ilaria__Manco
3K Followers 1K Following Research scientist @GoogleDeepMind working on music • DJ 🎶
dadabots @dadabots
11K Followers 9K Following ∿ Music hackers. Algoraves. Inventing, playing. Neural synthesis. 24/7 ai death metal. Stable Audio team. Open models. Mischief @harmonai_org @artblocks
Keunwoo Choi @keunwoochoi
7K Followers 830 Following AI, LLMs, and music. currently @upstageai @KAIST. prev: @genentech @gaudiolab @tiktok_us @spotify, @c4dm @qmul.
Chris Donahue @chrisdonahuey
6K Followers 1K Following GenAI for *human* creativity in music + more. Assistant prof at CMU CSD, 🎼 G-CLef lab. Part time Google DeepMind, Magenta (views my own)
Sander Dieleman @sedielem
68K Followers 2K Following Research Scientist at Google DeepMind (WaveNet, Imagen, Veo). I tweet about deep learning (research + software), music, generative models (personal account).
Joan Serrà @serrjoa
2K Followers 598 Following Does research on machine learning at Sony AI. Works on audio/multimodal analysis, synthesis, and retrieval. Likes tennis, music, and wine.
Marco Martínez @marcoamaram
1K Followers 370 Following senior research scientist #SonyAI and sometimes i also teach meditation @feralbreath
YCY @yoyolicoris
884 Followers 897 Following PhD Weeeeb @c4dm @qmeecs. Previously @RayarkOfficial @htcvive @AcadSinica @NYCU_official, Intern @SonyAI_global. I love music, anime, and vtubers.
Nao Tokui @naotokui_en
4K Followers 428 Following Surfing on sound, music, AI, and waves 🏄 / Founder of @qosmo_inc & @neutone_ai / @naotokui in 🇯🇵
Yi-Hsuan Yang @affige_yang
2K Followers 679 Following 1 Professor, EE, National Taiwan University 2 Chief Music Scientist, Taiwan AI Labs
Saurjya Sarkar @saurjyasarkar
584 Followers 337 Following
MTG-UPF @mtg_upf
3K Followers 308 Following Research group of @UPFBarcelona (@enginyeria_upf). Audio signal processing, music information retrieval, musical interfaces, computational musicology and more
𝔥𝔢𝔵𝔬𝔯�... @hexorcismos
3K Followers 3K Following ⚫️ Artist, technologist and musician ⚫️ ⚫️ 𝔐ᴜᴛᴜᴀʟɪsᴍ𝔛 out now via @_otherpeople : https://t.co/N11dZNJeYx ⚫️ ⚫️ AI music engineer @semilla_AI ⚫️
Hao-Wen (Herman) Dong... @hermanhwdong
1K Followers 326 Following Assistant Professor at University of Michigan | PhD from UC San Diego | Augmenting Human Creativity with AI | Gen AI for Music, Audio & Video
Pablo Alonso Jiménez @pablo__alonso
499 Followers 468 Following 🔬 Postdoc at @mtg_upf. ⌨️ Developer at @Essentia_MTG. 🎧 A/V artist at @llevolulsumiciu.
Scott H. Hawley @drscotthawley
3K Followers 1K Following Professor of Physics & Senior Data Fellow @BelmontUniv, teaching Audio Engineers. Mostly: ML/AI for music producers. Audio, Rep learning, Tech Ethics.
Felix Brener @VisualInference
48 Followers 441 Following Postmodern Pygmalion Applied AI // Design Nerd // Avid Generalist
Shih-Lun (Sean) Wu @slseanwu
303 Followers 157 Following music/audio/speech proc, generative models PhD student (now), EECS MIT MSc '24, SCS CMU BSc '21, CS Nat'l Taiwan U casual classical pianist🎹 & violist🎻
flenn grant @Kimcrypto26
34 Followers 384 Following I am officerin the United States Army who currently serves as commander of the International Security Assistance Force Joint Command (IJC) and deputy commander.
Mohamed Oussama (Stud... @oussama_ro41983
1 Followers 121 Following
Kyle @Sang_Kyle
0 Followers 188 Following
Sarah @l5ZjsNqX9dLHHw4
186 Followers 6K Following I am not a product of my circumstances. I am a product of my decisions.
Ash @arshiailaty
512 Followers 6K Following PhD @UCIrvine & @SDSU | EX-SWE-intern@Tesla گورستانِ ماه است شب!
Sethu Priya @sethu_priya_
96 Followers 6K Following
RunTimeException @run_time_exe
0 Followers 87 Following
Jennifer R. Rice @Jennifer282896
5 Followers 93 Following Hello I'm Jennifer R.Rice ! I'm professional Digital Markter .We Provide Digital Marketing Related.All Type Service
Jonas vasconceloss @Jonasvasco66460
112 Followers 8K Following
Chao Huang @chaooh_
11 Followers 165 Following I like to let model teach me. Research Scientist @Tencent Hunyuan Seattle Been at @Meta RL, @AMD GenAI | PhD @UofR
Dennis Trujillo, PhD @dptru10
760 Followers 3K Following Founder of @mercurial_ai. Researcher in ML applications for multimodal learning in healthcare. Former researcher at @advancedphoton, @LosAlamosNatLab, @UConn.
유명철 @Minstrel628
0 Followers 13 Following
Richard Foster-Fletch... @RFosterFletcher
2K Followers 2K Following I analyse the fault lines AI creates in judgement, governance, and power. Chair of MKAI.
Neha George @NehaSarahGeorge
220 Followers 664 Following Pianist| Counselor|Educator| #musicscience |
Assel Yermekova @alless_yer
13 Followers 155 Following I am ML Researcher. Have 10+ years in science, 4 years in ML. Interested in: Speech, Generative models.
Joannie 🇨🇦🇬�... @CanadianJoannie
3K Followers 7K Following Wheelchair athlete,PublicSpeaker,Disability Advocate, Mom of 3, mom of 3🐾🐾🐾 views are my own. dm for e-transfers/Paypal #Superfamily
joe_masi @joemasi9
12 Followers 72 Following Full time musician. Part time trader. Sometime vice versa.
Neta Glazer ✈️ AA... @glazer_neta
61 Followers 528 Following PhD candidate @BarIlanU , AI Safety researcher AI researcher @_aiOla
Byung Ha Kang @AILaeto
12 Followers 245 Following AI Researcher | interested Speech, Music, Sound AI
Eric Tchirnhausen @tchirnhaus20039
34 Followers 5K Following @huawei Like to try new things you never know; trying to prove all software can be automated. | ML/AI, | C++/Java/Go | GitHub : Dyl777
Alatau @AlatauMusic
5 Followers 31 Following Building agentic AI music tools with RAG, and acting as intelligent, collaborative producers for creators worldwide.
mankar_camoran @mankar_camoranx
0 Followers 84 Following
Music Non Stop @MusicNonStop_io
175 Followers 342 Following A group of musicians coding a generative music engine. @algomusic, @BenSilver_music and @nickcolemnlive
Seekcore @seekcore
2 Followers 254 Following
malika malikamaria @malika83070
0 Followers 161 Following
Clément Tabary @cbr_tabary
0 Followers 64 Following
Lwandlolubanzi T Ndeb... @lwandlendebele
2K Followers 7K Following Yashua Emmanuel Jesus Christ is Lord of ALL ! Human Rights Activist AI and Inference , Robotics , Aerospace, Quantum Computing Medoola Tecknologia
Subramanyam Sahoo @iamwsubramanyam
320 Followers 5K Following AGI Safety researcher (Independent), MARS Fellow(Cambridge AI Safety Hub), BASIS Fellow @UCBerkeley, Get Published or Die Trying.
Christian Steinmetz @csteinmetz1
6K Followers 2K Following Research Scientist @ Suno // working on generative music • audio fidelity • signal processing • ML
Pedro Sarmento @umpedronosapato
3K Followers 2K Following AI & Music Data Scientist @moises_ai | prev. @c4dm
Oriol (Uri) Nieto @urinieto
3K Followers 1K Following (INACTIVE ACCOUNT, FIND ME ON LINKEDIN/BLUESKY). Researcher at Adobe Research. Machine learning on audio. Oaklander born in Barcelona. Titan. He/they 🌈
Ilaria Manco @Ilaria__Manco
3K Followers 1K Following Research scientist @GoogleDeepMind working on music • DJ 🎶
dadabots @dadabots
11K Followers 9K Following ∿ Music hackers. Algoraves. Inventing, playing. Neural synthesis. 24/7 ai death metal. Stable Audio team. Open models. Mischief @harmonai_org @artblocks
Keunwoo Choi @keunwoochoi
7K Followers 830 Following AI, LLMs, and music. currently @upstageai @KAIST. prev: @genentech @gaudiolab @tiktok_us @spotify, @c4dm @qmul.
AK @_akhaliq
504K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo ,submit papers here: https://t.co/UzmYN5XOCi
ISMIR Conference @ISMIRConf
3K Followers 188 Following ISMIR is the largest research conference in the field of Music Information Retrieval (MIR). #ISMIR2026
Yann LeCun @ylecun
1.2M Followers 788 Following Professor at NYU & Executive Chairman at AMI Labs. Ex-Chief AI Scientist at Meta. Researcher in AI, Machine Learning, Robotics, etc. ACM Turing Award Laureate.
Chris Donahue @chrisdonahuey
6K Followers 1K Following GenAI for *human* creativity in music + more. Assistant prof at CMU CSD, 🎼 G-CLef lab. Part time Google DeepMind, Magenta (views my own)
Sander Dieleman @sedielem
68K Followers 2K Following Research Scientist at Google DeepMind (WaveNet, Imagen, Veo). I tweet about deep learning (research + software), music, generative models (personal account).
Joan Serrà @serrjoa
2K Followers 598 Following Does research on machine learning at Sony AI. Works on audio/multimodal analysis, synthesis, and retrieval. Likes tennis, music, and wine.
arXiv Sound @ArxivSound
7K Followers 38 Following Sound-related articles (https://t.co/dxVYgWJGOw and https://t.co/b90N0Zzvjs) on https://t.co/HHqPequzVU
Marco Martínez @marcoamaram
1K Followers 370 Following senior research scientist #SonyAI and sometimes i also teach meditation @feralbreath
C4DM at QMUL @c4dm
3K Followers 111 Following The Centre for Digital Music at Queen Mary University of London is a world-leading, multidisciplinary research group in the field of music & audio Technology.
Nao Tokui @naotokui_en
4K Followers 428 Following Surfing on sound, music, AI, and waves 🏄 / Founder of @qosmo_inc & @neutone_ai / @naotokui in 🇯🇵
François Chollet @fchollet
694K Followers 825 Following Co-founder @ndea. Co-founder @arcprize. Creator of Keras and ARC-AGI. Author of 'Deep Learning with Python'.
Yi-Hsuan Yang @affige_yang
2K Followers 679 Following 1 Professor, EE, National Taiwan University 2 Chief Music Scientist, Taiwan AI Labs
Isomorphic Labs @IsomorphicLabs
57K Followers 80 Following Solve all disease. Developing and applying frontier AI to unlock deeper scientific insights, faster breakthroughs, and life-changing medicines.
David Autor @davidautor
36K Followers 222 Following
Andrew McAfee 🇺�... @amcafee
80K Followers 826 Following Author of "The Geek Way" and "More from Less;" coauthor of "The Second Machine Age;" proponent of human and planetary flourishing.
Erik Brynjolfsson @erikbryn
232K Followers 5K Following Director @DigEconLab Co-founder, @Workhelix @StanfordHAI @SIEPR @Stanford https://t.co/D2bPyxoFEf
Daron Acemoglu @DAcemogluMIT
365K Followers 329 Following Institute Professor @MIT, @MITEcon. Co-Director of @MITShapingWork. Author of Why Nations Fail, The Narrow Corridor, and Power & Progress.
Philippe Aghion @Ph_Aghion
26K Followers 137 Following Professor at College de France, INSEAD and LSE. 2025 Nobel Prize in Economics. Account managed by my students at @cdf1530 to share news, not opinions.
Jukedeck Research @JukedeckRandD
65 Followers 4 Following
lyra bubbles @_lyraaaa_
3K Followers 999 Following ˈli.ɹə 🏳️⚧️⚢ · 25 · ableton enjoyer · mechinterp researcher · base model appreciator · data farming · lyraaaa_ on discord · ♡ @bubblemoder ♡ 🦔~ ♪❀
Huan @HuanZha80395890
479 Followers 378 Following PhD student in AI+Music @c4dm Expressive piano performance, computational musicology, music information retrieval, music education. I play piano and theremin.
Arnaud Doucet @ArnaudDoucet1
6K Followers 871 Following Senior Staff Research Scientist @GoogleDeepMind. Previously @UniofOxford.
Mansoor Rahimat Khan @marakhmusic
503 Followers 418 Following CEO @beatovenai | Backed by @join_ef & @capital2b | 7th Gen Classical Musician |ex- Product at Great Learning | NIT & @georgiatech alumni | Forbes 30u30 Asia
Prem Seetharaman @pseetharaman
461 Followers 81 Following Researcher in computer audition, machine learning, and HCI. Sr. Research Scientist, @AdobeResearch. Previously @DescriptApp, @Northwestern.
David Chalmers @davidchalmers42
42K Followers 661 Following philosopher@NYU. consciousness, reality+, life, the universe, and everything.
BSC-CNS @BSC_CNS
18K Followers 2K Following BSC-CNS (Barcelona Supercomputing Center-Centro Nacional de Supercomputación) is the National Supercomputing Facility in Spain.
Jascha Sohl-Dickstein @jaschasd
30K Followers 816 Following Member of the technical staff @ Anthropic. Most (in)famous for inventing diffusion models. AI + physics + neuroscience + dynamics.
IEEE ICASSP @ieeeICASSP
5K Followers 1 Following IEEE International Conference on Acoustics, Speech, and Signal Processing. #ICASSP2026 will be held 4-8 May 2026 in Barcelona, Spain.
Hao-Wen (Herman) Dong... @hermanhwdong
1K Followers 326 Following Assistant Professor at University of Michigan | PhD from UC San Diego | Augmenting Human Creativity with AI | Gen AI for Music, Audio & Video
Vivek Ramanujan @RamanujanVivek
283 Followers 111 Following PhD student at the University of Washington with Ali Farhadi. Previously a researcher at Allen Institute for Artificial Intelligence.
Michele Mancusi @michelemancusi
214 Followers 313 Following PhD, Senior Research Scientist @Moises | Former @Musixmatch, @Microsoft, @Sony | Working on #DeepLearning, #SignalProcessing, #GenerativeModels.
Gaudio Lab, Inc. @gaudiolab
598 Followers 384 Following Open a new realm of your world with Gaudio Lab’s AI powered audio technologies. Generative Sound | Source Separation | Spatial Audio
Alexander Kolesnikov @__kolesnikov__
12K Followers 196 Following
André Susano Pinto @ASusanoPinto
610 Followers 102 Following Machine learning research @GoogleAI, Opinions mine.
Michael Tschannen @mtschannen
3K Followers 684 Following Research Scientist @GoogleDeepMind. Representation learning for multimodal understanding and generation. Personal account.
Will Drevo @itsdrevo
401 Followers 631 Following ex-@mit @csail @coinbase @anyscalecompute @stabilityAI. ml & data, systems, audio. the best is yet to come.
Giovanni Bindi @gvnbnd
38 Followers 349 Following PhD in deep generative models for music - ACIDS team @ircam
Château Éphémère @Chateaufmr
775 Followers 709 Following #Château #fabrique #sonore et #numérique #fablabs @VanderLab_, #ateliers #résidences #musique #création #théâtre #danse #objetsconnectés #vieculturelle
Arturia @ArturiaOfficial
77K Followers 410 Following _The Sound Explorers Arturia creates hardware and software instruments to help musicians release their creativity.
Xavier Riley @xavriley
686 Followers 629 Following PhD researcher in AI and music @c4dm. Also guitarist, double bassist, jazz fan, dull bio writer.
Nate Raw @_nateraw
10K Followers 2K Following machine learning hacker @splice. previously @huggingface @lightningai
⚡AI Search⚡ @aisearchio
16K Followers 327 Following Sharing AI stuff 🚀680K subs https://t.co/0Ms1yEzdUN
Julian Lenz @JLenzyy
629 Followers 722 Following Audio AI research engineer w/ Lemonaide. prev. Neutone, Okio. MSc in Audio Computation at UPF. Opinions are a biased interpolation of my training dataset.
John Hopfield @HopfieldJohn
6K Followers 15 Following I work on neural networks and biology, often from the view point of my roots in physics. Professor @PrincetonNeuro.
The Nobel Prize @NobelPrize
1.3M Followers 492 Following The official feed of the Nobel Prize @NobelPrize #NobelPrize
Eric Tarr @hackaudio
519 Followers 291 Following
Mira Murati @miramurati
633K Followers 617 Following Now building @thinkymachines. Previously CTO @OpenAI
Daniel Ek @eldsjal
490K Followers 850 Following Father, Principal @ PM - Trying to build things that matter | Founder of Spotify and Neko Health






































