Dan Roy @roydanroy
ML / AI researcher, emphasis on theory. Research Director and Canada CIFAR AI Chair, @VectorInst Professor, @UofT (Statistics/CS) danroy.org University of Toronto Joined June 2009-
Tweets24K
-
Followers45K
-
Following2K
-
Likes20K
There are a lot of candidates out there choosing where to work. Careful where you choose!
There are a lot of candidates out there choosing where to work. Careful where you choose!
Nice thread!
When we down-scale LLMs (e.g.pruning), what happens to their capabilities? We studied complementary skills of memory recall & in-context learning and consistently found that memory recall deteriorates much quicker than ICL when down-scaling. See us @iclr_conf Session 1 #133 1/8
I may be biased but @gkdziugaite is 🔥🔥🔥
Arxiv link: arxiv.org/abs/2310.04680… . 5 min talk: recorder-v3.slideslive.com/#/share?share=… . Shout-out to my wonderful coauthors, collaborators and advisors: Nolan, @SimonXinDong @_vaishnavh @gkdziugaite @mcarbin @jrk!
@DrCMcMaster @colin_fraser @elonmusk c o n f i d e n c e i n t e r v a l s i n b i o
Vector Institute for the win.
Vector Institute for the win.
I'm as excited as you are about your {lab, company, school}'s research, but perhaps rather than just hype, you can display a bit of scientific humility and tell me also what the challenges, gaps are.
32/36. Is this nonsense?
I think the hardest thing for me the last few years has been seeing so many talented scientists who obviously belong in the academy turn into tech company middle managers or startup founders.
Is Ideogram using SD? No. We have @hojonathanho who came up with denoising diffusion and @wchan212 and @Chitwan_Saharia who led text to image and text to video at Google. We built everything from scratch, and we have a track record in foundational AI research that powers this…
Planning a sabbatical or about to start your first position? Come spend the year as a visiting researcher @VectorInst in downtown Toronto. Access our world-class faculty, facility, engineering team, food and compute. Lots of exciting healthcare collaborations ongoing with…
I have been asking this question since 2013 (but also if new players would enter), and have never heard a satisfactory answer to the question of why supply and demand laws do not seem to apply to GPUs. Why has supply not rushed in? Too technically sophisticated?
I have been asking this question since 2013 (but also if new players would enter), and have never heard a satisfactory answer to the question of why supply and demand laws do not seem to apply to GPUs. Why has supply not rushed in? Too technically sophisticated?
So will AMD and or Intel get their acts together and position themselves as major suppliers of GPUs for AI??
Guess my Uber rating and tell me yours and why it’s what it is.
Always great to see students succeed! Congrats Mufan.
Always great to see students succeed! Congrats Mufan.
I got tenure! It was fitting that I got to celebrate with the lab right after the news. Working together for the last 6.5 years has been a blast.
Heading to Chicago for the Midwest Robotics Workshop at @TTIC_Connect. Drop me a line if you are around and want to chat about AI, CV, and robots. 🦾💬
Clément Canonne @ccanonne_
31K Followers 928 Following Senior Lecturer @Sydney_Uni. Postdocs @IBMResearch, @Stanford; PhD @Columbia. Converts ☕ into puns: sometimes theorems. He/him. @[email protected]
Gautam Kamath @thegautamkamath
44K Followers 507 Following Assistant Prof of CS @UWaterloo, Faculty @VectorInst, Canada @CIFAR_News AI Chair. Co-EiC @TmlrOrg. I lead @TheSalonML. Privacy, robustness, machine learning.
Kosta Derpanis @CSProfKGD
48K Followers 197 Following #CS Associate Prof @YorkUniversity, #ComputerVision Scientist Samsung #AI, @VectorInst Faculty Affiliate, TPAMI AE, #CVPR2024/#ECCV2024 Publicity Co-chair
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Kevin Patrick Murphy @sirbayes
42K Followers 334 Following Research Scientist at Google Brain / Deepmind. Interested in Bayesian Machine Learning.
Rosanne Liu @savvyRL
33K Followers 968 Following Cofounded & running @ml_collective. Host of Deep Learning Classics & Trends. Research at Google DeepMind. DEI/DIA Chair of ICLR & NeurIPS. Writing https://t.co/IbycyGfnDR
Michael Bronstein @mmbronstein
43K Followers 4K Following #DeepMind Professor of #AI @UniofOxford / Fellow @ExeterCollegeOx / ML Lead @ProjectCETI / https://t.co/kZpGpDzYeV
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Zachary Lipton @zacharylipton
59K Followers 2K Following Professor: CMU/@acmi_lab, CTO / CSO: @AbridgeHQ, Creator: @d2l_ai & https://t.co/QQt98VNLUp, Relapsing 🎷
Thomas G. Dietterich @tdietterich
51K Followers 505 Following Distinguished Professor (Emeritus), Oregon State Univ.; Former President, Assoc. for the Adv. of Artificial Intelligence; Robust AI & Comput. Sustainability
Behnam Neyshabur @bneyshabur
18K Followers 690 Following Senior Staff Research Scientist @GoogleDeepMind, Interested in reasoning w. LLMs, traveling & backpacking
Ferenc Huszár @fhuszar
40K Followers 1K Following Secular Bayesian. Associate Professor in Machine Learning @Cambridge_CL. Talent aficionado at https://t.co/RbJkoLguey Alum of @Twitter, Magic Pony and @Balderton
yobibyte @y0b1byte
15K Followers 2K Following Kurin ViTaly, senior research scientist @IsomorphicLabs, ML PhD from @UniofOxford on RL, Multitask learning & Graphs
Sara Hooker @sarahookr
39K Followers 7K Following I lead @CohereForAI. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, @trustworthy_ml. Changing spaces where breakthroughs happen.
Animesh Garg @animesh_garg
21K Followers 1K Following Foundation Models for Generalizable Autonomy. Assistant Professor in AI Robotics @GeorgiaTech + @NvidiaAI. prev @Stanford @berkeley_ai @UofTCompSci
Sam Power @sp_monte_carlo
17K Followers 7K Following Lecturer in Maths & Stats at Bristol. Interested in probabilistic + numerical computation, statistical modelling + inference. (he / him)
Csaba Szepesvari @CsabaSzepesvari
8K Followers 704 Following "If there is not folly in the world, then the world itself is folly. You must understand that mistakes are not always regrets." - Paul Tobin, Bandette🤠
Maxim Raginsky @mraginsky
8K Followers 2K Following father, academic, raconteur, aging wannabe hipster blog: https://t.co/akk6LCvKw6
Eugene Vinitsky @EugeneVinitsky
13K Followers 2K Following Lets make multi-agent learning easy. Anti-cynic. RS at Apple, Asst. Prof at @nyutandon. He/him.
Prof. Anima Anandkuma.. @AnimaAnandkumar
25K Followers 2K Following Bren Professor @caltech, Fmr Sr Director of #AI research @nvidia, Fmr Principal Scientist @awscloud, AI+Science, PDE, Neural operators. Views my own.
Michael Zolotov @mzolotov_alt
19 Followers 133 Following
Dante @liampassion6626
3 Followers 49 Following
Andreas Tinhofer @AndreasTinhofer
24 Followers 218 Following
Nicolás Echániz @nicoechaniz
1K Followers 769 Following hacker-philosopher-spiritual seeker Founder of @AlterMundiNet "Alignment is futile" We need to reason our way to true convergence with AI, not try to tame them
Boris @ToushIFHAQ
34 Followers 259 Following Made in India Believe in yourself Above all have faith in Allah
jenny_yiong @Jennyyiong14
2 Followers 164 Following Beauty, cosmetics & personal care Fairies rely on faces to make a living Girls refuse to admit defeat👧ᵕ̈ ᑋᵉᑊ 🔸ℚ𝔹𝔼𝔼𝔽𝕃𝕐 Brand｜Quick consultation
🩸🛁 @Braillepro
1K Followers 4K Following Of all the bloodbaths in all the towns, in all the world, he ends up in mine. Seeking truth in the lies we tell each other on here.
مضمض @Arwiixv
0 Followers 80 Following
Brookdub @brookdub
1K Followers 5K Following Unwavering support for any political party is asinine. Also, I like the word “asinine.”
BALDEZO @Baldezo1004
137 Followers 979 Following 🗣️Soit à l'écoute, tout parle, tout est parole, tout cherche à nous communiquer une connaissance ✨⚡
Sharon Owino @5b662c44eef949c
2 Followers 23 Following
usertea @user18373bahs
0 Followers 505 Following
Kaushlendra @kaushkay
16 Followers 84 Following Interested in ML & AI. Co-Founder & CTO @ Stealth Startup
Paylz @paylza
141 Followers 2K Following The best online market for digital downloads with best prices.
Gpbhupinder @gpbhupinder
324 Followers 3K Following AI, Generative AI, Web Developer, Electronics Engineer, Space Nerd 🚀
F.Mackenzie @mackenzie85372
6 Followers 142 Following
Ryan D'Orazio @RyanDOrazio
372 Followers 316 Following PhD Student at Mila Quebec AI Institute, and Université de Montréal.
Kiarash Majdi @kia_majdi
1 Followers 36 Following Undergraduate Research Assistant | Machine Learning Researcher | Third Year Statistics @ University of Waterloo
Chad @al0k0
156 Followers 1K Following Insignificant, Self conscious, carbon based life-form. 13.772 billion light years old.
World of Smart Techno.. @WorldSmartTech
1K Followers 4K Following Technology, Energy and Climate Change, AI News. Entrepreneur.
Rafael Bittencourt @rafaelobitten
124 Followers 895 Following Baiano e torcedor do Vitória. Cientista de Dados e amante do uso de IAs Generativas para impulsionar produtividade.
Sheila Schoepp @sheilaschoepp
18 Followers 190 Following
sweetnightmare @chin__20
27 Followers 136 Following
지연 @oYvpgv
22 Followers 87 Following
Nicholas Lourie @NickLourie
154 Followers 313 Following I build things. 🤖 Doing a PhD at @nyuniversity (@CILVRatNYU) on better empirical methods for deep learning and data science. Advised by @kchonyc and @hhexiy.
Milena Moncada @moncadamilena
435 Followers 5K Following 🇻🇪🇨🇴🇨🇦Psicóloga Clínica. Estudiosa de los Trastornos del Espectro Autista y TDAH. Neurodesarrollo. Clinical Psychologist 🧠
Ali Kheirandish @AliKheirandish4
155 Followers 4K Following
Jose Finger @jvfinger
252 Followers 3K Following SQL Server, Oracle, Data warehouse, Alteryx, Tableau. Dogs and Cats. Warhammer 40k Fan.Automation for farms
Eduardo Garrido @vedugarmer
4 Followers 116 Following PhD on Computer Science. AI researcher. Associate Professor at Universidad Pontificia Comillas. Student of knowledge, philosopher's apprentice.
Ramalingam @ramlingamr
141 Followers 4K Following Reader; I share items that interest me in AI, Skills / Talent, Healthcare, Investments and Chess.
mohamed ali @mhmmd_aliiii
17 Followers 1K Following
Jinyuan (Tobias) @JinyuanWang7
86 Followers 772 Following Research Engineer @NUS. Data scientist @Lushair
Clément Canonne @ccanonne_
31K Followers 928 Following Senior Lecturer @Sydney_Uni. Postdocs @IBMResearch, @Stanford; PhD @Columbia. Converts ☕ into puns: sometimes theorems. He/him. @[email protected]
Gautam Kamath @thegautamkamath
44K Followers 507 Following Assistant Prof of CS @UWaterloo, Faculty @VectorInst, Canada @CIFAR_News AI Chair. Co-EiC @TmlrOrg. I lead @TheSalonML. Privacy, robustness, machine learning.
Kosta Derpanis @CSProfKGD
48K Followers 197 Following #CS Associate Prof @YorkUniversity, #ComputerVision Scientist Samsung #AI, @VectorInst Faculty Affiliate, TPAMI AE, #CVPR2024/#ECCV2024 Publicity Co-chair
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Kevin Patrick Murphy @sirbayes
42K Followers 334 Following Research Scientist at Google Brain / Deepmind. Interested in Bayesian Machine Learning.
Rosanne Liu @savvyRL
33K Followers 968 Following Cofounded & running @ml_collective. Host of Deep Learning Classics & Trends. Research at Google DeepMind. DEI/DIA Chair of ICLR & NeurIPS. Writing https://t.co/IbycyGfnDR
Michael Bronstein @mmbronstein
43K Followers 4K Following #DeepMind Professor of #AI @UniofOxford / Fellow @ExeterCollegeOx / ML Lead @ProjectCETI / https://t.co/kZpGpDzYeV
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Zachary Lipton @zacharylipton
59K Followers 2K Following Professor: CMU/@acmi_lab, CTO / CSO: @AbridgeHQ, Creator: @d2l_ai & https://t.co/QQt98VNLUp, Relapsing 🎷
Thomas G. Dietterich @tdietterich
51K Followers 505 Following Distinguished Professor (Emeritus), Oregon State Univ.; Former President, Assoc. for the Adv. of Artificial Intelligence; Robust AI & Comput. Sustainability
Ferenc Huszár @fhuszar
40K Followers 1K Following Secular Bayesian. Associate Professor in Machine Learning @Cambridge_CL. Talent aficionado at https://t.co/RbJkoLguey Alum of @Twitter, Magic Pony and @Balderton
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Richard Sutton @RichardSSutton
26K Followers 37 Following Student of mind and nature, libertarian, chess player, cancer survivor. @ Keen Technologies, UAlberta, Amii, RLAI, The Royal Society, RichSutton.eth
yobibyte @y0b1byte
15K Followers 2K Following Kurin ViTaly, senior research scientist @IsomorphicLabs, ML PhD from @UniofOxford on RL, Multitask learning & Graphs
Petar Veličković @PetarV_93
30K Followers 555 Following Staff Research Scientist @GoogleDeepMind | Affiliated Lecturer @Cambridge_Uni | Associate @clarehall_cam | GDL Scholar @ELLISforEurope. Monoids. 🇷🇸🇲🇪🇧🇦
Sara Hooker @sarahookr
39K Followers 7K Following I lead @CohereForAI. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, @trustworthy_ml. Changing spaces where breakthroughs happen.
Animesh Garg @animesh_garg
21K Followers 1K Following Foundation Models for Generalizable Autonomy. Assistant Professor in AI Robotics @GeorgiaTech + @NvidiaAI. prev @Stanford @berkeley_ai @UofTCompSci
Sam Power @sp_monte_carlo
17K Followers 7K Following Lecturer in Maths & Stats at Bristol. Interested in probabilistic + numerical computation, statistical modelling + inference. (he / him)
Csaba Szepesvari @CsabaSzepesvari
8K Followers 704 Following "If there is not folly in the world, then the world itself is folly. You must understand that mistakes are not always regrets." - Paul Tobin, Bandette🤠
Maxim Raginsky @mraginsky
8K Followers 2K Following father, academic, raconteur, aging wannabe hipster blog: https://t.co/akk6LCvKw6
Håvard Rue @HavardRue1
79 Followers 4 Following
Andriy Burkov @burkov
19K Followers 142 Following Author of 📖 The Hundred-Page Machine Learning Book and the 📖 Machine Learning Engineering book
Sam Cohen @SamCMaths
367 Followers 366 Following Mathematician; Christian; beer, beards and silly hats aficionado
Joshua Saxe @joshua_saxe
3K Followers 982 Following AI+cybersecurity at Meta; past lives in academic history, labor / community organizing, classical/jazz piano, hacking scene
Giannis Daras @giannis_daras
4K Followers 399 Following Ph.D. candidate, Computer Science @UTAustin, working with @AlexGDimakis. Research Scientist Intern @nvidia. Ex: @google, @explosion_ai, @ntua
Avrajit Ghosh @GhoshAvrajit
201 Followers 629 Following Computational math PhD student, Michigan State University. Generalization, optimization, Inverse problems. (No prior) better than (wrong priors).
Christian Andersson N.. @chris_naesseth
2K Followers 317 Following Researcher interested in creativity, reasoning, and uncertainty in machine learning as well as their application to the sciences.
Anca Dragan @ancadianadragan
8K Followers 178 Following AI safety & alignment at Google DeepMind • associate professor at UC Berkeley EECS • proud mom of an amazing 2yr old
Roger Melko @rgmelko
4K Followers 174 Following Professor, University of Waterloo; Associate Faculty, Perimeter Institute for Theoretical Physics
Tim Dettmers @Tim_Dettmers
29K Followers 821 Following PhD Student at @UW. I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.
Omar Sanseviero @osanseviero
32K Followers 2K Following Chief Llama Officer @huggingface 🦙 Founder @AI_Learners. Xoogler (SWE @Google Assistant, 20% PM TF Graphics). 100% Hacker Llama🇵🇪🇲🇽
Desh Raj @rdesh26
3K Followers 2K Following Research Scientist @Meta (AI Speech) | Previously: @jhuclsp, @IITGuwahati
Logan Kilpatrick @OfficialLoganK
92K Followers 2K Following Lead product for @Google AI Studio and working on the Gemini API, helping developers build with AI, my views!
Terezia Zorić @terezia_zoric
2K Followers 4K Following UTFA President re-elected as leader of 'Team Terezia,' a large, diverse & impressive group of colleagues drawn from every corner of our UofT tri-campus
Pushmeet Kohli @pushmeet
10K Followers 97 Following Computer Scientist, VP Research, Leading Science @ Google DeepMind.
Neal Wu @WuNeal
15K Followers 391 Following Building @cognition_labs. Previously @tryramp, @GoogleBrain, @Harvard, competitive programming (featured in @Wired). Created https://t.co/pihw5AGvbV.
Will Knight @willknight
20K Followers 7K Following I write about AI and related stuff for WIRED. signal = wak.01 (no pr pitches pls). newsletter = https://t.co/qG4DExCEbS
Jess Sorrell @JessSorrell
206 Followers 408 Following CS postdoc at Penn. Interested in theory of ML and responsible computing. All cat pictures are my own and do not represent the cats of my employer.
cora @KylerCora
8K Followers 2K Following built @soon_dating . living in cantor's paradise . former PhD student @ucberkeley .
Mathieu Blondel @mblondel_ml
9K Followers 421 Following Research scientist at Google DeepMind. Current research interests: differentiable programming, LLMs, Transformers.
PolymathicAI @PolymathicAI
2K Followers 77 Following The Polymathic AI Collaboration. Shared account.
Gregor Bachmann @GregorBachmann1
234 Followers 274 Following I am a PhD student @ETH Zürich working on deep learning. MLP-pilled 💊. https://t.co/yWdDEV6Z15
Cognition @cognition_labs
123K Followers 19 Following Makers of Devin, the first AI software engineer. We are an applied AI lab focused on reasoning, and code is just the beginning. Join us: https://t.co/tpfZwEwGiq
Wei Ji Ma @weijima01
10K Followers 276 Following Prof of neuroscience and psychology at NYU | Co-Founder, https://t.co/ybdMwoElnI | Founder, https://t.co/92S5O65Vcz | Founding member, @ScientistAction.
Caglar Gulcehre @caglarml
4K Followers 1K Following ML Researcher Prof @ EPFL, PI @ CLAIRE lab Ex: Staff Research Scientist @ Deepmind, MSR, IBM Research Follow me on Mastodon: https://t.co/LZ5sWt7Asj
Daniel Johnson @_ddjohnson
2K Followers 576 Following Researcher at @GoogleDeepMind. PhD student at @VectorInst / @UofT. Building tools to study neural nets and find out what they know. He/him.
Wen-Ding Li @xu3kev
2K Followers 5K Following Program Synthesis & ML. Previously Student Researcher at @google. Previously intern at @theteamatx. Mastodon: [email protected]
Alex Mordvintsev @zzznah
16K Followers 2K Following Mad Scientist, DeepDream creator. Designing Self-Organising Systems and Programmable Artificial Life. https://t.co/rntipHzHW3
Aurelien Lucchi @AurelienLucchi
1K Followers 291 Following Researcher in optimization and theoretical machine learning. Assistant professor at the University of Basel. Past: EPFL, ETH Zurich (Switzerland) 🇨🇭
Eric Novik @ericnovik
1K Followers 474 Following Bayesian Inference, Decision Theory, Stan, R, Viz. CEO/Founder @ Generable. Adjunct faculty, Statistics @ NYU Steinhardt.
Marc Mezard @marc_mezard
3K Followers 927 Following @[email protected] Physicist, Professor @Unibocconi Former director of Ecole normale supérieure -PSL university Retweets not endorsements.
Massachusetts Institu.. @MIT
1.3M Followers 587 Following The Massachusetts Institute of Technology is a world leader in research and education. Related accounts: @MITevents @MITstudents @MIT_alumni
Sara Beery @sarameghanbeery
11K Followers 3K Following Research on computer vision and the environment 🌍 Asst Prof at @MIT_CSAIL #QueerInAI 🏳️🌈 sarabeery on threads @[email protected]
Vincent Sitzmann @vincesitzmann
13K Followers 296 Following Assistant Professor @ MIT, leading the Scene Representation Group (https://t.co/h5gvhLYrtw). Neural scene reps., neural rendering, inverse graphics.
Jack Rae @drjwrae
9K Followers 354 Following Principal Scientist @ Google DeepMind Work on Gemini 💎♊ Compression is all you need LLMs (e.g. Gopher, Chinchilla, Gemini) 💼 Past: OpenAI, Quora
Alex Murphy @Alxmrphi
481 Followers 2K Following 🙋♂️= NeuroAI Researcher. Postdoc @ Amii / UAlberta studying language, vision & (and in) DNNs and brains. PhD & ex-Google Brain intern (🇮🇪 & 🇬🇧)
Olivia Simin Fan @Olivia61368522
583 Followers 849 Following 🎓Ph.D.@EPFL_en-MLO|| https://t.co/QGwaUTkuyY.@UMich. || https://t.co/QGwaUTkuyY.@sjtu1896. ML&LLM research🧐 Interested in being an interesting girl ;)
Marco Ciccone @mciccone_AI
699 Followers 355 Following @ELLISforEurope postdoc @PoliTOnews @ai_ucl Competitions co-chair @NeurIPSConf 2021, 2022, 2023 PhD @polimi • ex @NVIDIA @NNAISENSE
Solomon Kurz @SolomonKurz
9K Followers 697 Following Clinical psychology researcher | adjunct professor | applied statistics geek | so called #RStats influencer
Stephen Wild @stephenjwild
2K Followers 1K Following I try to put straight lines through things but usually fail. Try to be Bayesian when I can. Views my own. RT/like != endorsement. Graduate of YouTube University
A. Jordan Nafa @ajordannafa
6K Followers 783 Following Bayesian Statistician and Data Scientist in the Video Game Industry | Bayes, Causality, Decision Theory, Stan, R, Python | PhD Candidate at @UNT_PSCI
Freda Shi @fredahshi
2K Followers 674 Following Starting July 2024: Asst. Prof. @UWCheritonCS @VectorInst, #nlproc #compling Now: PhD Student @TTIC_Connect Ex-@PKU1898, @MetaAI, @GoogleDeepMind Feeder of 3 🐈
Foundation Agent: a roadmap to build generally capable embodied AI that acts skillfully across many worlds, virtual or real. Project GR00T, the Humanoid robot foundation model, is a cornerstone for Foundation Agent. It's the North Star, the next grand challenge in our quest for…
@roydanroy @gkdziugaite You're biased, but your claim is true.
RL is not the only paradigm for language model alignment or controlled generation! Let’s take a principled probabilistic approach: define a target distribution, perform inference, estimate KLs. Enter our work on twisted SMC for LMs with @brekelmaniac* @AliMakhzani @RogerGrosse
there's always a curious dichotomy between the ability to process memory from pretraining data vs the ability to rely on the current context. visit our poster at #ICLR to learn about how these abilities deteriorate in vastly different ways when you downscale models!
When we down-scale LLMs (e.g.pruning), what happens to their capabilities? We studied complementary skills of memory recall & in-context learning and consistently found that memory recall deteriorates much quicker than ICL when down-scaling. See us @iclr_conf Session 1 #133 1/8
Barbados -> Montreal -> Toronto -> Amsterdam -> Copenhagen -> Norway -> Berlin -> Prague -> **Vienna** 🥳 I have been on a sabbatical recently slowly making my way to ICLR - HMU if you wanna meet!
This will hit arxiv tomorrow night, but you can get the pdf right now: maxim.ece.illinois.edu/pubs/variation…
EECS Rising Stars workshop will be held at MIT this year. Strongly encourage eligible individuals to apply. risingstars-eecs.mit.edu
@karpathy @akbirthko @giffmana Any feedback on how to make it better?
[Raw Thoughts] What I don't like about AI/ML research these days can be explained with the help of Kieran Setiya's (@KieranSetiya) terminology in his "Midlife: A Philosophical Guide" book about the value of activities: 🧵(1/10)
Post a picture YOU took. Just a pic. No description.
Michael and Jie did an amazing job on their first PhD project, by finding and fixing common pitfalls in empirical ML privacy evaluations. It turns out, if you evaluate things properly, DP-SGD is also the best *heuristic* defense when you instantiate it with large epsilon values.
Heuristic privacy defenses claim to outperform DP-SGD in real-world settings. With no guarantees, can we trust them? We find that existing evaluations can underestimate privacy leakage by orders of magnitude! Surprisingly, high-accuracy DP-SGD (ϵ >> 1000) still wins. 🧵
never thought i'd be featured in the financial times, but if i am going to be featured, i'm glad it's because of my crazy commute! 😂 ft.com/content/26b552…
Need help (again!) from experts on #stats learning theory: How does the constant in the L2 minimax rate O(n^-s/(2s+d)) for estimating Holder functions (conditional mean or density) depend on dimension d and Holder order s? What is hidden in the big-O?
🚨 New blog post! Black Box Reductions: Constrained Online Learning From the classic reduction for online learning in the simplex to reductions for arbitrary convex sets, using non-euclidean and Bregman projections Bonus: Simple Regret Matching+ proof parameterfree.com/2024/04/29/bla…
What Bill Fefferman learned from Umesh: “Always listen to physicists, especially when what they are saying sounds completely crazy …” #Umeshfest @SimonsInstitute
With distinguished colleagues. #Umeshfest @SimonsInstitute
@Mathgarden @minilek This sentence makes zero sense
Trends for United States
You might like
