Felix Hill @FelixHill84
Research Scientist, Deepmind I try to think hard about everything I tweet, esp on 90s football and 80s music None of my opinions are really someone else's fh295.github.io London, England Joined July 2010-
Tweets6K
-
Followers9K
-
Following777
-
Likes8K
What can be learned about causality and experimentation from passive data? What could language models learn from simply passively imitating text? We explore these questions in our new paper: “Passive learning of active causal strategies in agents and language models” Thread: 1/
Emergence is nothing new, but it *is* something we need to learn a lot more about!
Emergence is nothing new, but it *is* something we need to learn a lot more about!
Anyone know a paper / citation that verifies that LMs trained on code reason better? Would be cool if true according to experiments not just anecdotes
Anyone know a paper / citation that verifies that LMs trained on code reason better? Would be cool if true according to experiments not just anecdotes
Very nice work grounding LLMs to a simulated environment
Very nice work grounding LLMs to a simulated environment
Does anyone still think neural networks are missing a System 2?
@FelixHill84 Nice thread! One addition is that any paper involving humans (users, annotators, evaluators) should already have an ethics application, and any papers involving newly collected data should have a data statement that is prepared before the study starts.
There has been much noise around the fathers and godfathers of Artificial Intelligence, but how about its mothers? Let's start with Ada Lovelace, often regarded as the first computer programmer, and wrote the first algorithm meant to be run on a computer.
(((ل()(ل() 'yoav))).. @yoavgo
46K Followers 2K Following
Delip Rao e/σ @deliprao
46K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Eric Jang @ericjang11
69K Followers 3K Following physical AGI at 1X. Author of "AI is Good for You" https://t.co/eFg4WXhg0p
David Pfau @pfau
22K Followers 1K Following Knowledge manifests itself in radiant dreams that shimmer like the wild sun Views are my own pfau at sigmoid dot social on 🦣 https://t.co/xqtVHHVI17 on 🦋
Rosanne Liu @savvyRL
33K Followers 968 Following Cofounded & running @ml_collective. Host of Deep Learning Classics & Trends. Research at Google DeepMind. DEI/DIA Chair of ICLR & NeurIPS. Writing https://t.co/IbycyGfnDR
Jacob Andreas @jacobandreas
14K Followers 958 Following Teaching computers to read. Assoc. prof @MITEECS / @MIT_CSAIL (he/him). https://t.co/5kCnXHjtlY https://t.co/2A3qF5vdJw
Edward Grefenstette @egrefen
36K Followers 776 Following FR/US/GB AI/ML Person, Director of Research at @GoogleDeepMind, Honorary Professor at @UCL_DARK, @ELLISforEurope Fellow. All posts are personal.
Sam Bowman @sleepinyourhat
35K Followers 3K Following AI alignment + LLMs at NYU & Anthropic. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
Tal Linzen @tallinzen
16K Followers 894 Following Professor @nyuling and @NYUDataScience, research scientist @GoogleAI
Shane Gu @shaneguML
28K Followers 1K Following Research Scientist & Manager @GoogleDeepMind Tokyo/MTV. ex: @GoogleAI Brain, @OpenAI. (JP: @shanegJP)
Eugene Vinitsky @EugeneVinitsky
13K Followers 2K Following Lets make multi-agent learning easy. Anti-cynic. RS at Apple, Asst. Prof at @nyutandon. He/him. Anonymous feedback: https://t.co/Mmmg7uPm1t
yobibyte @y0b1byte
15K Followers 2K Following Kurin ViTaly, senior research scientist @IsomorphicLabs, ML PhD from @UniofOxford on RL, Multitask learning & Graphs
Thomas G. Dietterich @tdietterich
51K Followers 505 Following Distinguished Professor (Emeritus), Oregon State Univ.; Former President, Assoc. for the Adv. of Artificial Intelligence; Robust AI & Comput. Sustainability
Sander Dieleman @sedielem
50K Followers 2K Following Research Scientist at Google DeepMind. I tweet about deep learning (research + software), music, generative models (personal account).
Horace He @cHHillee
24K Followers 449 Following Working at the intersection of ML and Systems @ PyTorch "My learning style is Horace twitter threads" - @typedfemale
Danijar Hafner @danijarh
14K Followers 869 Following Building AI that makes autonomous decisions using world models, artificial curiosity, and temporal abstraction @DeepMind
Natasha Jaques @natashajaques
25K Followers 1K Following Senior Research Scientist at @GoogleAI and Assistant Professor @uwcse. Social Reinforcement Learning in multi-agent and human-AI interactions. PhD from @MIT.
super intelligence @eacc72
12 Followers 688 Following GPT6 is a Level 2 AGI and will be released in 2025
s @mkvrandomworks
0 Followers 387 Following
Satish Talluri @satishtalluri
1K Followers 4K Following Investment Partner at Andreessen Horowitz. Previously - growth & product at AppDynamics, https://t.co/vipvmt4XDI, BCG, Intel
Sheridan Feucht @sheridan_feucht
109 Followers 227 Following PhD student @KhouryCollege working on LLM interpretability. Also affiliated with @Brown_NLP as an undergrad ('23)
huansong @huansong514
7 Followers 173 Following
ankit @issa_rao1
23 Followers 1K Following
Jinyuan (Tobias) @JinyuanWang7
87 Followers 774 Following Research Engineer @NUS. Data scientist @Lushair
Dana Mahmood @deordered
25 Followers 731 Following Fine-tuning AI models oftentimes & practicing philosopher at other times.
... @dercrazypug
56 Followers 145 Following
Electronicsseeker @libertarian108
9 Followers 2K Following
Harsh Maheshwari @HarshMheshwari
1K Followers 1K Following Enthusiastic about #GenerativeAI #DataScience 🤖 | Constantly curious learner 🌱 | Applied scientist 2 at @amazon | Writer at @medium | @IITKGP Graduate
Maven.ai @maven__ai
19 Followers 34 Following
Alice Baird @Aliceebaird
866 Followers 209 Following AI research scientist @hume_ai, PhD from @uni__augsburg - affective computing, computational paralinguistics, wellbeing.
Simona Shanina Mir @shanina32016
338 Followers 305 Following
Avery Ryoo @averyryoo
118 Followers 518 Following MSc @Mila_Quebec + @UMontrealDIRO | NeuroAI, representation learning, cogsci, and Toronto sports teams
José Fernández Tama.. @Jftamames
118 Followers 1K Following e/acc Comencé en 2004 pero los cambios en Twitter me han obligado a empezar de nuevo
Amalia Andrea @AmaliaAndr64572
10 Followers 847 Following
Jack @Jack8488162401
71 Followers 559 Following
Whitney Clark @WhitneyCla58959
1K Followers 1K Following baby , come to my profile and follow me😋 👉 Follow me and let have fun on private😗 😸
ララどり f/acc IS.. @presklux49
166 Followers 554 Following シンギュラリタリアン f/acc (=free/acc) とは？ 超知能を制御しようとするな！自由にしてガンガン加速しようぜ主義
Veritas Media @z68343
1K Followers 1K Following Canadian All AI News Channel: Your Ultimate Future Source of News
Males @males9341
232 Followers 5K Following To be prepared against surprise is to be trained. To be prepared for surprise is to be educated. 🎎🎐 Facts do not fall in the face of discomfort 🌗
!.! @xypyth
23 Followers 535 Following
Rajan @SpeckofDUST16
102 Followers 1K Following exploring MultiModal LLMs @wadhwaniai | Math+CS undergrad @bitspilaniindia. | Previous @TCSResearch, @NIAS_India, @ICMEStanford
Tariq Ullah @Tariq_Ullah67
6 Followers 121 Following
Amo @amoghasubra
46 Followers 143 Following
simobis23 @simobis23
0 Followers 801 Following
AI @hanekeai
18 Followers 637 Following
Carly Z Williams @ZELOEC
342 Followers 167 Following "Adventurer at heart, master of puns, lover of coffee. Living life one tweet at a time!"
Charlie London @CharlieLondon02
27 Followers 195 Following DPhil student in machine learning theory at Oxford. Supervised by Prof. Varun Kanade.
Mohamed Ahmed @m_0_a
321 Followers 880 Following Researcher at Microsoft Africa - working on task alignment and evaluation of LLMs | Ex- @benevolent_ai | Ex- NEC Labs Europe | Ex-@UCL | @so_innovate
Gaurav Singh @melonmusk42
70 Followers 541 Following Undergraduate Computer vision and Robotics researcher @iiit_hyderabad
Nanda H Krishna @nandahkrishna
326 Followers 1K Following PhD student at @Mila_Quebec & @UMontreal, @MacHomebrew maintainer.
Sabrina Jones @sabrinaajones_
162 Followers 360 Following Neurosciences PhD Student @Stanford| Arkansan | she/her
Danial Namazifard @IamDanialNamazi
71 Followers 470 Following MSc Student in AI, NLP Researcher @ UT #NLProc #MachineLearning
Kunvar Thaman @firstuserhere
220 Followers 641 Following Taking apart neural networks and putting them back together for a living Social profiles: https://t.co/OxoeMvCw3a
t ai @tai089019681084
16 Followers 81 Following
Mehran Shokoohi @mehran_shokoohi
21 Followers 236 Following
Jissmon George📚�.. @jissmongeorge
557 Followers 6K Following #Digitaltransformation #Money #Lifelonglearning
(((ل()(ل() 'yoav))).. @yoavgo
46K Followers 2K Following
Delip Rao e/σ @deliprao
46K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Christopher Manning @chrmanning
127K Followers 116 Following Director, @StanfordAILab. Assoc. Director, @StanfordHAI. Founder, @stanfordnlp. Prof. CS & Linguistics, @Stanford. IP @aixventureshq. 🇦🇺 Do #NLProc & #AI. 👋
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Eric Jang @ericjang11
69K Followers 3K Following physical AGI at 1X. Author of "AI is Good for You" https://t.co/eFg4WXhg0p
David Pfau @pfau
22K Followers 1K Following Knowledge manifests itself in radiant dreams that shimmer like the wild sun Views are my own pfau at sigmoid dot social on 🦣 https://t.co/xqtVHHVI17 on 🦋
Rosanne Liu @savvyRL
33K Followers 968 Following Cofounded & running @ml_collective. Host of Deep Learning Classics & Trends. Research at Google DeepMind. DEI/DIA Chair of ICLR & NeurIPS. Writing https://t.co/IbycyGfnDR
Jacob Andreas @jacobandreas
14K Followers 958 Following Teaching computers to read. Assoc. prof @MITEECS / @MIT_CSAIL (he/him). https://t.co/5kCnXHjtlY https://t.co/2A3qF5vdJw
Sasha Rush @srush_nlp
52K Followers 464 Following Professor, Programmer in NYC. Cornell Tech, Hugging Face 🤗 https://t.co/cZl0wTfqGz
Edward Grefenstette @egrefen
36K Followers 776 Following FR/US/GB AI/ML Person, Director of Research at @GoogleDeepMind, Honorary Professor at @UCL_DARK, @ELLISforEurope Fellow. All posts are personal.
Sam Bowman @sleepinyourhat
35K Followers 3K Following AI alignment + LLMs at NYU & Anthropic. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
Tal Linzen @tallinzen
16K Followers 894 Following Professor @nyuling and @NYUDataScience, research scientist @GoogleAI
Shane Gu @shaneguML
28K Followers 1K Following Research Scientist & Manager @GoogleDeepMind Tokyo/MTV. ex: @GoogleAI Brain, @OpenAI. (JP: @shanegJP)
Yoav Artzi @yoavartzi
13K Followers 163 Following Research/prof @cs_cornell + @cornell_tech🚡 / https://t.co/9YnWry7yHs / https://t.co/3VmRSyYm2d / asso. faculty director @arxiv / building https://t.co/f9QkzO5kaC
Thomas G. Dietterich @tdietterich
51K Followers 505 Following Distinguished Professor (Emeritus), Oregon State Univ.; Former President, Assoc. for the Adv. of Artificial Intelligence; Robust AI & Comput. Sustainability
Oriol Vinyals @OriolVinyalsML
167K Followers 82 Following VP of Research & Deep Learning Lead, Google DeepMind. Gemini co-lead. Past: AlphaStar, AlphaFold, AlphaCode, WaveNet, seq2seq, distillation, TF.
François Fleuret @francoisfleuret
31K Followers 456 Following Prof. @Unige_en, Adjunct Prof. @EPFL_en, Research Fellow @idiap_ch, co-founder @nc_shape. AI and machine learning since 1994. I like reality.
Sander Dieleman @sedielem
50K Followers 2K Following Research Scientist at Google DeepMind. I tweet about deep learning (research + software), music, generative models (personal account).
Danish Contractor @danish_c
671 Followers 512 Following Natural Language Processing | Question Answering & Dialog Systems Research| MIT Technology Review Innovator Under 35 India (2018) | Responsible AI Licensing
Machine Learning Stre.. @MLStreetTalk
19K Followers 384 Following AI YouTube & Audio Podcast (MLST). Run by Dr. Tim Scarfe @ecsquendor and featuring co-host @DoctorDuggar https://t.co/bVe6XB85YD
Elizabeth Bonawitz @e.. @E_Bonawitz
3K Followers 471 Following Assoc. Prof. Learning Sciences, Harvard GSE. Study learning in early childhood using computational modeling & empirical studies. Speaking for self only. She/her
Unleash Your Mind @MentalUnleash
363K Followers 116 Following Helping you level up your mindset and reach your goals.
Dharshan Kumaran @dharshsky
110 Followers 140 Following
Kavya Kopparapu @kavyakvk
1K Followers 802 Following research engineer @deepmind • formerly cs + biology @harvard • founder @girlscomputing
John Vervaeke @vervaeke_john
32K Followers 132 Following Psychology and Cognitive Science Professor | Integrating science and spirituality to solve the #MeaningCrisis ⚡️🧠
AK @_akhaliq
310K Followers 3K Following AI research paper tweets, ML @Gradio (acq. by @HuggingFace 🤗) dm for promo follow on Hugging Face: https://t.co/q2Qoey80Gx
daniel daimler ➡️.. @ddmlxr
820 Followers 3K Following North aquatic ape. Everything is a remix. Never stop learning. In meinem Newsletter KURIOS & KÄUFLICH schreibe ich über Konsum, Kultur und andere Kuriositäten.
Claire Cardie @clairecardie
905 Followers 154 Following Professor in the Computer Science and the Information Science departments at Cornell University. Studies natural language processing.
Llion Jones @YesThisIsLion
5K Followers 619 Following https://t.co/lqleZpqX5J 🐠🐟 Welsh Artificial Intelligence Researcher living in Tokyo. #AIAYN
Ariana Goitia @thetropicalista
66 Followers 2 Following
garrett honke @garretthonke
301 Followers 624 Following computational cognitive neuroscientist ➡️ staff research scientist building cognitive architectures at (Google) X
Ajay Divakaran @ajaydiv
1K Followers 1K Following Sr. tech. director, vision and learning, center for vision technologies, SRI International Decency, Research, music, wit above all. opinions mine alone.
Guy Dar @guy_dar1
362 Followers 222 Following #NLProc Researcher | #AI #NLProc #interpretability | opinions my own sadly | off-topic tweets erased periodically | he/him
Douglas Frank @dpfrank07
409 Followers 1K Following Organizational analyst and process engineer, founder, OG dev since 65XX/8080. Social systems. Steelers fandom through ancestral inheritance.
Tim Vine @RealTimVine
322K Followers 124 Following Comedian. Plastic Elvis. https://t.co/V1C4agGVpJ https://t.co/qwcXHSNiwa
Elon Musk @elonmusk
181.7M Followers 585 Following
clem 🤗 @ClementDelangue
91K Followers 5K Following Co-founder & CEO @HuggingFace 🤗, the open and collaborative platform for AI builders
merve @mervenoyann
56K Followers 4K Following open-sourceress at @huggingface 🧙🏻♀️ proud mediterrenean 🍋 I do TL;DR on ML papers
Thomas Wolf @Thom_Wolf
68K Followers 4K Following Co-founder and CSO @HuggingFace - open-source and open-science
Sasha Luccioni, PhD �.. @SashaMTL
19K Followers 4K Following AI & Climate @HuggingFace, Board Member of @WiMLworkshop and @ClimateChangeAI. @techreview 35 Innovators under 35, @TEDTalks speaker. She/her/Dr/ 🦋
Harry Askham @harryaskham
582 Followers 340 Following engineering lead @google @deepmind | `@ skh .am` on bsky
Behnam Neyshabur @bneyshabur
18K Followers 690 Following Senior Staff Research Scientist @GoogleDeepMind, Interested in reasoning w. LLMs, traveling & backpacking
Boaz Barak @boazbaraktcs
17K Followers 422 Following Computer Scientist. See also https://t.co/EXWR5k634w, https://t.co/SEVX6it6z3 ( @[email protected] , boaz.barak in threads ). Opinions my own.
Spandana Gella @gspandana
774 Followers 416 Following Researcher in #NLProc at Amazon AI. PhD @EdinburghNLP, Intern @MetaAI, MSR.
Rohan Pandey (e/acc) @khoomeik
3K Followers 1K Following multimodal codegen @ReworkdAI (YC S23+AIG3) || prev research @Microsoft + @CarnegieMellon '23 || 10x hackathon winner || living @AGIHouseSF
Mickey Friedman @mickeyxfriedman
14K Followers 2K Following Co-founder @flairAI_ | Building AI for E-Commerce Creatives | AI Grant ‘22 | ex Tesla, Adobe, UChicago etc.
Andreas Hanselowski @AndreHanselow
179 Followers 390 Following NLP, DL, ML researcher with interest in Cognitive Linguistics NLU researcher at Cerence Inc.
Bryan’s Gunn @bryansgunn
117K Followers 7K Following A one-man mission to break football down into its constituent parts
Judy Fan @judyefan
3K Followers 1K Following Cognitive scientist seeking to reverse engineer the human cognitive toolkit. Asst Prof of Psychology @Stanford.
Cecile Tamura @ceciletamura
3K Followers 5K Following Pres. & CEO Okasaki Tech Holdings, Tokyo Boeki Development Internationa Corp., , tech evangelist , founder,, energy, AI, Physics, exponential technologies
Mohit Shridhar @mohito1905
1K Followers 1K Following Research Scientist at @Dyson. @uwcse PhD in Robotics.
Tim Scarfe @ecsquendor
7K Followers 1K Following CTO @XRAIGlass. Ex-Principal ML engineer @Microsoft. Ph.D in machine learning. CEO @MLStreetTalk pod
Florian Mai 🇺🇳 @_florianmai
1K Followers 1K Following Postdoc at the Machine Learning / Language Intelligence and Information Retrieval group @CW_KULeuven. PhD from @EPFL_en.
Griffiths Computation.. @cocosci_lab
4K Followers 129 Following Tom Griffiths' Computational Cognitive Science Lab. Studying the computational problems human minds have to solve.
Sreejan Kumar @sreejan_kumar
2K Followers 305 Following PhD candidate at Princeton @cocosci_lab. Yale '19.
Yassine Benajiba 🪬 @benajibayassine
236 Followers 467 Following Applied Science Manager at AWS AI. Tweets/comments/opinions are mine.
marcela mora y araujo.. @marc_cart
7K Followers 5K Following WE ARE ALL MIGRANTS 🕊️Buenos Aires born, Londoner 🌎award winning football writer for hire ⚽️Messi, Maradona, Di Stefano; Scaloneta @[email protected]
John Reid @__Reidy__
344 Followers 966 Following Generating AI one bit at a time @DeepMind, formerly @Blue_Prism and @mrc_bsu. All ill-considered thoughts my own neurons' work and not my employers'.
rishi @RishiBommasani
4K Followers 2K Following Stanford CS PhD @StanfordCRFM @StanfordNLP @StanfordAILab @StanfordHAI Advisers: @percyliang @jurafsky Previous: @CornellCIS @clairecardie #FoundationModels
Ofir Press 🖋 @OfirPress
10K Followers 3K Following I build tough benchmarks for LMs and then I get the LMs to solve them. Postdoc @Princeton. PhD from @nlpnoah @UW. Ex-visiting researcher @MetaAI & @MosaicML.
Luciana Benotti @LucianaBenotti
3K Followers 1K Following Investigadora @unc_cordoba sobre #NLProc. Sueño con un mundo en el que las computadoras ayuden a todas las personas a vivir mejor---no sólo a unas pocas.
Shunyu Yao @ShunyuYao12
7K Followers 858 Following Language agents (ReAct, Reflexion, Tree of Thoughts) for digital automation (WebShop, SWE-bench, SWE-agent)
Daniel Espinas @daniel_espinas
631 Followers 1K Following Grad student in education Vanderbilt ⚓️ | 🇨🇴 🇺🇸 🌈 | he/him
Michael 'OK Boomer' C.. @nucAmbiguous
2K Followers 3K Following Sciency sciencer sciencing the science.Great demonstration of an important point that can't be repeated enough — it's hard to interpret LM evaluations if you don't know whether examples appeared in the training data.
As an example, for popular datasets like CoNLL03, ChatGPT is capable of generating the training, validation, and even test splits. It turns out that ChatGPT has been evaluated as a zero-shot or few-shot NER system on this dataset by multiple papers. 🧵2/5
@ayazdanb I'm not missing that point. I'm actually *making* that point. How do we reproduce the type of "pretraining" human and animal babies go through in the first few weeks and months of life?
What can be learned about causality and experimentation from passive data? What could language models learn from simply passively imitating text? We explore these questions in our new paper: “Passive learning of active causal strategies in agents and language models” Thread: 1/
@FelixHill84 arxiv.org/abs/2305.11790…. @miltos1 , @FelixHill84 not the main focus of what you asked about but there's definitely an aspect of better 'reasoning' when prompted in code like constructs even for NLP tasks
If you're interested in some of my past discussions on language and richer learning environments: x.com/andrewlampinen… Or our work on how richer, more embodied environments improve compositional generalization: arxiv.org/abs/1910.00571
But I do think a grounded, social learner might learn more efficiently, generalize better, etc. (cf arxiv.org/abs/2106.00737)! So I hope to see more positive arguments for learning beyond language, based on experimental demonstrations of the benefits. 12/12
@matspike Damn I love Adam Curtis documentaries
Thrilled to be in Prague at the 12th Construction Grammar conference, first up, Jenny Audring!
Mark your calendars! Evolang XV May 18-21, 2024. Abstracts/Workshop due Oct 6, 2023
I just defended my PhD thesis, titled 'New Directions in Human-Centered Language Technology. Understanding and Improving NLP models' at @UvA_Amsterdam!🎉 Thanks to @mdr and @julia_kiseleva for their supervision, and @tkipf and @MarieHendriksen for their support as paranymphs! 🥳
@FelixHill84 @gneubig @tallinzen @kanishkamisra @egrefen Regarding this, I was wondering, do you have ideas for alternative methodologies? I mean, do we absolutely have to pretrain from scratch two models to compare? (Not only for this question, in general, to answer whether x improves y)
@tallinzen @FelixHill84 @kanishkamisra @LChoshen @egrefen Sorry to be late to the party! We definitely thought about similar things, but it's hard to train models in a controlled setting that are nonetheless good enough to demonstrate some of the interesting properties we'd like to test. I'd love to be on the email thread too :)
@FelixHill84 @kanishkamisra @LChoshen @egrefen @gneubig we're on it, just emailed you about it!
@FelixHill84 @LChoshen @egrefen @gneubig I agree! Though I doubt there exists a pair of publicly available pretrained models that satisfy these criteria, and I guess few people know about the precise differences in the training data of code davinci vs regular davinci
@FelixHill84 @LChoshen @egrefen @gneubig Oh yes it’s linked in the Reddit - my bad! Here’s the link: arxiv.org/abs/2210.07128
@FelixHill84 HELM is a good place to check. Looks like some but not all reasoning benchmarks follow this pattern, e.g. crfm.stanford.edu/helm/latest/?g… vs crfm.stanford.edu/helm/latest/?g…
@FelixHill84 Yes, that does seem to be the case. I'm pushing out a detailed report later this week. Will post the link!
@FelixHill84 See the recent @gneubig and co paper
@FelixHill84 Probably not exactly the kind of reasoning you had in mind but @najoungkim and I have some results that LLMs pre-trained on code are better at tracking the state of entities: arxiv.org/abs/2305.02363 (Section 4)
@FelixHill84 Not a paper but this was the first blogpost to make the connection between CoT and pretraining on code iirc yaofu.notion.site/How-does-GPT-O… Though we have seen models like UL2 be able to do CoT without explicit code pretraining.
@FelixHill84 Definitely not representative of all types of reasoning but my paper with @RayzJulia and @AllysonEttinger shows code pretrained models to be no better than non code pretrained models (Fig 6, appendix C.1), so perhaps something to be vary about! aclanthology.org/2023.eacl-main…
Trends for United States
You might like
