Ido Rosen @idorosen
#ai #ml #economics #neuroscience #trading #pilot #xoogler👨💻🤖🧠📈🛩🏳️🌈 (tweets my own. not advice. no endorsement implied.) ido.ai San Francisco, CA Joined February 2008-
Tweets483
-
Followers1K
-
Following2K
-
Likes215
I often hear that evals are the most confusing part of creating LLM AI products. It's a shame b/c IMO, domain-specific evals are the most important part of an AI product! I've written a detailed blog post with real examples on how to do this (1/3) hamel.dev/blog/posts/eva…
There appears to be a mismatch between publishing criteria in AI conferences and "what actually works". It is easy to publish new mathematical constructs (e.g. new models, new layers, new modules, new losses), but as Apple's MM1 paper concludes: 1. Encoder Lesson: Image…
I'm releasing all the lectures and notes for an introductory course on Statistical Detection and Estimation I used to teach. The core material hasn't changed - it was an EE course, but it's as relevant today to AI researchers as ever before. Hope you find it useful. Covers: *…
I put up a mini-site today at citydensity.com to help you compare the world’s cities. You can see how many people live near a city, or use my favourite metric: "population weighted density" to get an accurate measure of how dense a city is to live in.
@karpathy is perhaps the most talented deep learning teacher out there, and his video lectures are always worth watching. Some minor addenda on the history of tokenization: While GPT-2 used sub-word tokenization pretty early, it was really shown to be important for handling…
@karpathy is perhaps the most talented deep learning teacher out there, and his video lectures are always worth watching. Some minor addenda on the history of tokenization: While GPT-2 used sub-word tokenization pretty early, it was really shown to be important for handling…
@Tim_Dettmers Yeah exactly. The trap is that the original creator has actually built it piece by piece and over time molded it, which creates an unintuitively large disparity between how easy they perceive it, and how easy fresh eyes perceive it, even when controlling for technical level.
Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish colors to hyperparameters for which training diverges.
Folks, you don't need anything special to have python automatically drop you into the debugger if there's an exception -- this capability is already built in! Here's a complete demonstration of how to do it, including an alias you might find pretty handy:
The skills needed for developing new ML techniques have little overlap with the skills needed for applying ML effectively. A bit like how chip design has little overlap with software engineering.
I'd like to start 2024 with some thoughts on the state of optimization theory. To begin with, let me quote Nesterov's 2003 book: "The main fact, which should be known to any person dealing with optimization models, is that, in general, the optimization problems are unsolvable" 🧵
@dharmesh Vector embeddings *do not* magically solve search. In fact, the heavy lifting is in the step before you re-rank with semantic similarity search. Making a genuine improvement over BM25 or full-text search is hard.
What people think production ML is vs. what it actually is
The fact that most individual neurons are uninterpretable presents a serious roadblock to a mechanistic understanding of language models. We demonstrate a method for decomposing groups of neurons into interpretable features with the potential to move past that roadblock.
Interesting, I'd missed that Python gets a whole new generic syntax in 3.12 via the `type` statement. Looks very TypeScript-y. I think Ruff should be able to do automatic upgrades to this new syntax.
Folks, the time to run debirdify.pruvisto.org or fedifinder.glitch.me is now. You don't need to have an account elsewhere yet. Download the CSVs while you can, and you can import them later. go go go go
Tomorrow (Wednesday 6th) at @BlackHatEvents, we are presenting with @martinralbrecht, @DowlingBJ and @djwj_ our work on finding practically exploitable vulnerabilities in Matrix. Join us!! blackhat.com/eu-22/briefing… (and check our paper: nebuchadnezzar-megolm.github.io)
The 6 different ways of using a language transformer / LLM 1 Train from scratch 2 Feature-based: Train new model on embeddings 3 Finetuning I: Freeze all but output layer weights 4 Finetuning II: Update all weights 5 Zero-shot learning 6 Few-shot learning Anything missing? 1/6
Someday aliens are going to land their saucers in a field somewhere in New Jersey and everything is going to go just fine right up until we try to explain our calendar to them
Recently @twilio, which provides SMS verification services for Signal, suffered a phishing attack. Via Twilio, attackers may have accessed phone numbers & SMS registration codes for 1,900 Signal users. 1/
I am delighted to announce that the camera-ready version of my new book, "Machine Learning: Advanced Topics", is finally available online for free at probml.github.io/book2 (@mitpress will publish the hard copy in 2023.)
rbit @crypto_hades
7K Followers 2K Following Quant trading. In crypto since 2011. HFT, statarb. C++, Python.
Sheslea @Sheslea445585
1 Followers 55 Following
Annemarie Bhatia @AnnemarieB12537
86 Followers 5K Following
Era Donald @DonaldEra46045
40 Followers 5K Following
Earlean Seifts @EarleSeif
42 Followers 5K Following
Miah Rhorer @miah_rhor
56 Followers 5K Following
Cora Wieben @CoraWieben58768
49 Followers 5K Following
Bonnie-mae Vizcarrond.. @MaeVizcarron
56 Followers 5K Following
Bella Lakey @BellaLakey55141
70 Followers 5K Following
Alessandro Krause @Alessandro63773
64 Followers 3K Following
Kiara Deisch @KDeisc
23 Followers 5K Following
Keiko Schueren @kei_schuer
57 Followers 5K Following
Nannie Baraban @NBaraban66160
13 Followers 2K Following
Marcelene Bouley @marcelene12505
79 Followers 5K Following
Sabrina Conder @ConderSabri
72 Followers 5K Following
Josh Young @Josh_Young_1
95K Followers 2K Following Contrarian value investor. O&G. Not advice, do not rely, past performance not indicative, positioning may change w/out notice. Likes & r/ts are not endorsements
Carlee Adil @AdilCarl
88 Followers 5K Following
Anisa Sheu @AnisaSheu24387
84 Followers 5K Following
Tonita Pressman @PressmanTo55537
70 Followers 5K Following
Emeli Eflin @EflinEme
44 Followers 5K Following
Georgann Monje @MoGeorga
32 Followers 5K Following
Breanna Kalin @brean_kal
50 Followers 5K Following
Cassaundra Vaquero @CVaquero42677
37 Followers 5K Following
Haylee Bronzo @BronzHayle
51 Followers 5K Following
Genevieve Binstock @GenevieveB90132
73 Followers 5K Following
Felicidad Strackbein @FelicidadS46081
70 Followers 5K Following
Jenna Packineau @JennaPacki38362
47 Followers 5K Following
Mila-rose Vandermeer @MilaVanderm
27 Followers 2K Following 📈Mila-rose , 23 , Earn your own Crypt$ casino👇🐋
Jenice Stoss @JeniceStos64779
65 Followers 5K Following
Cyrstal Schiltz @CyrstalSch74545
36 Followers 5K Following
Alannah Moats @alann_moa
37 Followers 5K Following
Lakisha Funn @FunnLakish13781
92 Followers 5K Following
Ellia Tackette @tacket_elli
73 Followers 5K Following
Coco Coggins @CogginsCoc15136
16 Followers 3K Following
A_JaneD_SJ2 @sj2_a62524
6 Followers 510 Following
Elisabeth Allamon @AllamElisab
47 Followers 5K Following
k_zer0s @k_zer0s
751 Followers 2K Following VC, Quantum Computing, AGI, AI, SDXL, FPGA, Startup Consulting, Senior Venture Architect at Financial Institution.
Lucille Sprang @LucillSpran
31 Followers 2K Following 💰Lucille ~ 25 ~ Biggest crypto casino presale👇🔗
io🏄🏽♂️ @Astro_Erik
3K Followers 765 Following CEO, Maverick, comedian, lover and human person. 🤔🤦♂️ Tweets/RTs are not Endorsements! 👀🗑🔥 😎🤙
Marilyn Launt @LauntMaril16937
82 Followers 5K Following
mendy @mufaasa883416
181 Followers 3K Following Am living with my siblings our both parents has passed away few years ago we are looking for help at all over the world. Out mum live a 2 year old young baby🍲
Cristie Picken @CristiePic64424
72 Followers 5K Following
Stevie Harmeson @HarmeStev
16 Followers 3K Following
Logan @McShine466606
96 Followers 1K Following 아주 아름다운 나라예요. 당신 나라의 남자들은 매우 신사적입니다. 당신과 친구하고 싶어요 https://t.co/eKEuMQXc9P LiYY1225543
Paul Graham @paulg
1.9M Followers 772 Following
Yann LeCun @ylecun
710K Followers 718 Following Professor at NYU. Chief AI Scientist at Meta. Researcher in AI, Machine Learning, Robotics, etc. ACM Turing Award Laureate.
Eric Jang @ericjang11
69K Followers 3K Following physical AGI at 1X. Author of "AI is Good for You" https://t.co/eFg4WXhg0p
@levelsio @levelsio
417K Followers 1K Following 🦄https://t.co/sQ0aiU7v02 $202K/m 💆https://t.co/AoNP9BW2Dp $2K/m ✨https://t.co/BmbkrX4Zyf $0.1K/m 📸https://t.co/lAyoqmSBRX $57K/m 🖼https://t.co/1oqUgfD6CZ $44K/m 🌍https://t.co/BjTozWAXwG $27K/m 🛰https://t.co/ZHSvI2wjyW $51K/m
cephalopod @macrocephalopod
45K Followers 568 Following At Octopus Capital our passion is providing best-in-class liquidity in the marketplace of ideas
clem 🤗 @ClementDelangue
90K Followers 5K Following Co-founder & CEO @HuggingFace 🤗, the open and collaborative platform for AI builders
Erik Bernhardsson @bernhardsson
38K Followers 3K Following Building @modal_labs when I'm not posting bangers about data and software. Previously built the music rec sys at Spotify and ran the eng team at Better.
Sebastian Raschka @rasbt
266K Followers 906 Following Machine learning & AI researcher writing at https://t.co/A0tXWzG1p5. LLM research engineer @LightningAI. Previously stats professor at UW-Madison.
Andrej Karpathy @karpathy
978K Followers 904 Following 🧑🍳. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥
Ethan Mollick @emollick
210K Followers 551 Following Professor @Wharton studying AI, innovation & startups. Democratizing education using tech Book: https://t.co/CSmipbJ2jV Substack: https://t.co/UIBhxu4bgq
(((ل()(ل() 'yoav))).. @yoavgo
46K Followers 2K Following
François Chollet @fchollet
469K Followers 770 Following Deep learning @google. Creator of Keras. Author of 'Deep Learning with Python'. Opinions are my own.
Kyle Cranmer @KyleCranmer
16K Followers 3K Following Director Data Science Institute @UWMadison @datascience_uw. EiC @MLSTjournal. Physics, stats/ML/AI, open science. same handle @sigmoid.social and bsky
Patrick McKenzie @patio11
164K Followers 796 Following I work for the Internet and am an advisor to @stripe. These are my personal opinions unless otherwise noted.
Camille Fournier @skamille
50K Followers 831 Following Distributed systems, dysfunctional programming, and all that management gobbledegook. Author, “The Manager’s Path." she/her. https://t.co/Uk9L1X9GeE
martin_casado @martin_casado
50K Followers 2K Following GP @ a16z ... questionable heuristics in a grossly underdetermined world
Balaji @balajis
1.0M Followers 4K Following Immutable money, infinite frontier, eternal life. #Bitcoin
Austin Rief ☕️ @austin_rief
145K Followers 1K Following Co-founder & CEO @MorningBrew | Owner/Investor @oceans_xyz | Opinions Are My Own
Yao Fu @Francis_YAO_
13K Followers 2K Following PhD @EdinburghNLP on LLMs and Machine Reasoning. Ex. @Columbia @PKU1898 @MITIBMLab @allen_ai AGI has yet to come, so keep running
Emre Gucer @aegucer
728 Followers 335 Following building fume (actually fume builds itself, i just help) - yc w24
Jesse Lyu @jessechenglyu
28K Followers 290 Following founder and ceo @rabbit_hmi board @jugendingenieur any crypto relates to @rabbit_hmi or r1 is a scam.
Elon Musk @elonmusk
181.3M Followers 584 Following
Emery Wells @emerywells
11K Followers 1K Following Founder of a unicorn, @Frame_io (acquired by Adobe). Video Pro. Apple Design Award winner.
Danielle Fong 💁�.. @DanielleFong
49K Followers 9K Following physics / AI / energy / art / realposting. 🏳️🌈/ally. cofounder, inventor, CEO, https://t.co/oSXgT0x7Mm "extraordinary alien" - US govt HIGH VOLUME ACCOUNT
Matt Shumer @mattshumer_
51K Followers 1K Following CEO @HyperWriteAI, @OthersideAI - I make AIs do the impossible.
Sebastian Völkl @basti_vkl
4K Followers 937 Following Building AI software for space/defense to reimagine how complex hardware systems are built. @1517fund fellow. Founded @hackerBCI. e/acc
Daniel Dennett @danieldennett
299K Followers 75 Following I'm an author and philosopher of mind and cognitive scientist @philosophytufts.
Aston Zhang @astonzhangAZ
5K Followers 92 Following Research Scientist at the #llama team of Meta Generative AI, designing and training large language models. Opinions are my own.
Mike Lewis @ml_perception
6K Followers 227 Following Llama3 pre-training lead. Partially to blame for things like the Cicero Diplomacy bot, BART, RoBERTa, kNN-LM, top-k sampling & Deal Or No Deal.
Teknium (e/λ) @Teknium1
29K Followers 3K Following Cofounder @NousResearch, prev @StabilityAI Github: https://t.co/LZwHTUFwPq HuggingFace: https://t.co/sN2FFU8PVE Support me on Github Sponsors
Junyang Lin @JustinLin610
5K Followers 1K Following Chief Evangelist Officer of Qwen Team & OpenDevin, building LLM and LMM. Now @Alibaba_Qwen . Previously @PKU1898 LANCO group. ❤️ 🍵 ☕️ 🍷 🥃
Yangqing Jia @jiayq
12K Followers 263 Following Founder @leptonai. @UCBerkeley alumni. ex @google & @facebook. ex vp @AlibabaGroup. Open source work on caffe, @pytorch, @tensorflow, & @onnxai.
Pessimists Archive @PessimistsArc
91K Followers 67 Following Exploring technophobia and moral panic through the ages. A litany of shameful cynicism and spite. Curated by @louisanslow
Ross Taylor @rosstaylor90
6K Followers 869 Following Something new 🥷. Previously: @paperswithcode, reasoning lead @metaai, Galactica LLM lead, Atlas ML (acq by Meta)
Naftali Bennett נפ�.. @naftalibennett
610K Followers 0 Following ראש הממשלה ה-13 של מדינת ישראל • 13th Prime Minister of Israel
Hank Green @hankgreen
1.6M Followers 989 Following You should get some socks: https://t.co/IMXCkqywav
Ezra Klein @ezraklein
2.6M Followers 1K Following Columnist, @NYTOpinion Author, "Why We're Polarized" Host of "The Ezra Klein Show" podcast
Daniel Feldman❗ @d_feldman
18K Followers 11K Following Security engineering at Stealth Co. Always trying new things. Good takes are mine alone. Bad takes are someone else's.
Answer.AI @answerdotai
1K Followers 81 Following A new kind of AI R&D lab which creates practical end-user products based on foundational research breakthroughs
Tal Linzen @tallinzen
16K Followers 894 Following Professor @nyuling and @NYUDataScience, research scientist @GoogleAI
austin petersmith @awwstn
9K Followers 869 Following co-founder https://t.co/SVWqiSF4Se // prev: founder @capiche (acq by @vendrhq) // pre-launch investor in Lattice, Mercury, Superhuman, Varda + 50 more
stephen balaban @stephenbalaban
9K Followers 1K Following Co-founder, CEO - Lambda. We’re hiring: https://t.co/n0yFaq0ZDn
Alex Clemmer 🔥🔥.. @hausdorff_space
4K Followers 1K Following Brexit, Britney Spears, and Buffalo Wild Wings. if there aint no ring on my finger you aint goin on my gram wasq'u descendent.
Chris Paxton @chris_j_paxton
8K Followers 1K Following Mostly posting about robots. Embodied AI @hellorobotinc, formerly @AIatMeta, @NVIDIAAI, @zoox. All views my own.
Sebastian Ruder @seb_ruder
80K Followers 1K Following Multilingual LLMs @cohere • Prev: @GoogleDeepMind • Newsletter: https://t.co/7JGh2qpG98
Alexander Osipovich @aosipovich
8K Followers 142 Following Reporter covering exchanges, HFT and market structure, and occasionally crypto, at The Wall Street Journal
Yishan @yishan
79K Followers 355 Following I run Terraformation, and I was once the CEO of Reddit. Both are very interesting challenges. Views are mine alone, but also yours if I do my job right.
AssemblyAI @AssemblyAI
37K Followers 403 Following Access powerful AI models to transcribe and understand speech via a simple API. Try our no-code playground for free 👉 https://t.co/YPCK9mq5Qy
PicoCreator (🇸🇬.. @picocreator
2K Followers 164 Following Builds Attention-Free Transformer (https://t.co/YL7CbNYKBs) from scratch - CEO @ https://t.co/kQHiGtzJWr Also built k8s tools, uilicious & GPU.js (https://t.co/OIfnI1EPU7)
Santiago @svpino
352K Followers 444 Following I tell stories about technology and teach hard-core Machine Learning at https://t.co/iZifcK7n47. YouTube: https://t.co/pROi08OZYJ
edgartools @dwightwgunning
177 Followers 2K Following
Talin @IamTalin
468 Followers 259 Following Ex- Strategy & Finance @_superagi & @ContloHQ, Ex-@summitpartners // @chiefaioffice is my alt
Sholto Douglas @_sholtodouglas
15K Followers 856 Following Scaling Gemini @Deepmind - working towards intelligence too cheap to meter
James Bradbury @jekbradbury
11K Followers 8K Following Compute at @AnthropicAI! Previously JAX, TPUs, and LLMs at Google, MetaMind/@SFResearch, @Stanford Linguistics, @Caixin.
Noam Brown @polynoamial
34K Followers 611 Following Researching reasoning @OpenAI | Co-created Libratus/Pluribus, the first superhuman no-limit poker AIs | Co-created CICERO | PhD from @SCSatCMU
Magic.dev @magicailabs
10K Followers 3 Following Magic is working on frontier-scale code models to build a coworker, not just a copilot. Come join us: https://t.co/hGZKtUzsR3
Eric Steinberger @EricSteinb
7K Followers 478 Following Writing code that writes code on a mission to build safe superintelligence | CEO/cofounder @magicailabs
Yaroslav Bulatov @yaroslavvb
6K Followers 698 Following ex-Google Brain, OpenAI, Meta Scholar: https://t.co/iVycFw5dSX New Blog: https://t.co/SLix8HqVeY Old Blog: https://t.co/Ur3GWKoOzy
derek guy @dieworkwear
798K Followers 964 Following Menswear writer. Editor at @putthison. Creator of @RLGoesHard. Bylines at The New York Times, The Washington Post, The Financial Times, Esquire, and Mr. Porter
I installed @comma_ai in my car Their tagline is “make driving chill” which is accurate! I have pretty good confidence relying on it to drive most of the time, feels way “chiller” than Tesla autopilot (I haven’t driven the new FSD) It’s awesome that you can self install…
# scheduling workloads to run on humans Some computational workloads in human organizations are best "run on a CPU": take one single, highly competent person and assign them a task to complete in a single-threaded fashion, without synchronization. Usually the best fit when…
THE TECHNO-OPTIMIST MANIFESTO part 1 “You live in a deranged age — more deranged than usual, because despite great scientific and technological advances, man has not the faintest idea of who he is or what he is doing.” — Walker Percy “Our species is 300,000 years old. For the…
A few new CUDA hacker friends joined the effort and now llm.c is only 2X slower than PyTorch (fp32, forward pass) compared to 4 days ago, when it was at 4.2X slower 📈 The biggest improvements were: - turn on TF32 (NVIDIA TensorFLoat-32) instead of FP32 for matmuls. This is a…
95% of what we are doing in AI is stuff that's simple enough to explain to a child, made way over-complicated by mathematical-looking notation and unclear thinking. A short rant 🧵 using attention as an example.
Why Google Deepmind's Mixture-of-Depths paper, and more generally dynamic compute methods, matter: Most of the compute is WASTED because not all tokens are equally hard to predict
Should I proceed?
Thrilled to share we've raised $21.7m for @ColumnTax to scale up our fundamentally new tax filing product forbes.com/sites/igorbosi… It's been so much fun building the first-ever IRS authorized tax filing API with the amazing team here
i think it's a very good sign that the last three people to join Column Tax are all former founders
Apple says its latest AI model ReALM is even “better than OpenAI’s GPT4”. It likely is as GPT4 has regressed because of “alignment”. The ReALM war begins at WWDC 2024. Paper: arxiv.org/pdf/2403.20329…
@idorosen Thanks, Ido. We have yet to meet anyone who is envious of our minivan!
WTH! @Rainmaker1973 blocked me after I asked him to delete a post of MY video on X by someone who ripped it off TikTok. I didn't want to have to do it, but if @elonmusk's platform is this bad at protecting its creators I'll work up a little more action.
I often hear that evals are the most confusing part of creating LLM AI products. It's a shame b/c IMO, domain-specific evals are the most important part of an AI product! I've written a detailed blog post with real examples on how to do this (1/3) hamel.dev/blog/posts/eva…
3 paper deadlines, 3 awards nominations, one video deadline, one lecture to make and give, work to do for a group meeting, and policy feedback to give all by eod tmr thursday 😵💫 (and this is not an unusual amount of work). all for a chance at a non-contractor position someday...
TIL about binary vector search... apparently there's a trick where you can take an embedding vector like [0.0051, 0.017, -0.0186, -0.0185...] and turn that into a binary vector just reflecting if each value is > 0 - so [1, 1, -1, -1, ...] and still get useful cosine similarities!
There appears to be a mismatch between publishing criteria in AI conferences and "what actually works". It is easy to publish new mathematical constructs (e.g. new models, new layers, new modules, new losses), but as Apple's MM1 paper concludes: 1. Encoder Lesson: Image…
Trends for United States
You might like
