Databricks Mosaic Research @DbrxMosaicAI
We remove the barriers to state-of-the-art generative AI model development and make data + AI available to all. databricks.com/research/mosaic San Francisco, CA Joined December 2020-
Tweets965
-
Followers29K
-
Following115
-
Likes666
Our team is incredibly proud to partner with @allen_ai and thrilled to see them cook! Achieving such a massive improvement in MMLU, while reducing the compute budget, is a fantastic win. And doing it fully open? Everyone wins. Congrats! Can't wait to see what's next 👀
Our team is incredibly proud to partner with @allen_ai and thrilled to see them cook! Achieving such a massive improvement in MMLU, while reducing the compute budget, is a fantastic win. And doing it fully open? Everyone wins. Congrats! Can't wait to see what's next 👀
Part of what makes #DBRX special is the performance we deliver when we serve it. In Daya Khudia's talk at @nvidia #GTC24, he shared some insights on how we do it. Watch the replay here: nvidia.com/en-us/on-deman…
Ready to use a programmatic approach to prompting #LLMs and building #RAG applications? The @stanfordnlp #dspy repo includes support for @databricks Model Serving and Vector Search! Details: databricks.com/blog/dspy-data…
📢 New blog post from our @databricks Mosaic AI researchers @mvpatel2000 and @vitaliychiley announcing the integration of MegaBlocks open source library into #LLM foundry, our open source #training stack! 🙌
📢 New blog post from our @databricks Mosaic AI researchers @mvpatel2000 and @vitaliychiley announcing the integration of MegaBlocks open source library into #LLM foundry, our open source #training stack! 🙌
📢TOMORROW! Join some of our amazing research team (@bandish @abhi_venigalla @davisblalock @ajaysaini725) online for a deep dive on #DBRX - hosted by @databricks DevOps guru @dennylee. Register now!
📢TOMORROW! Join some of our amazing research team (@bandish @abhi_venigalla @davisblalock @ajaysaini725) online for a deep dive on #DBRX - hosted by @databricks DevOps guru @dennylee. Register now!
DBRX is the top open-source model on the latest WildBench Leaderboard on HuggingFace! Thanks to our friends @allen_ai for this benchmark of LLMs with challenging tasks from real users in the wild. #DBRX huggingface.co/spaces/allenai…
Thank you @JuliaANeagu for recognizing the accomplishments of our @databricks Mosaic AI research and engineering teams in building our highly performant open source #DBRX model. 🙌🙌🙌
Thank you @JuliaANeagu for recognizing the accomplishments of our @databricks Mosaic AI research and engineering teams in building our highly performant open source #DBRX model. 🙌🙌🙌
Happy to announce that we're launching the DBRX-coin! Every new variant of the model creates a node on the chain, so train more, make more $$! Let's make crypto and AI actually correlate!
Curious about #DBRX and how it was trained? Join @abhi_venigalla and @ajaysaini725 to learn about the model and the @databricks platform that trained it! Hosted by our own Eric Peter, and the AI Alliance's @TimBonnemann and @ChiefScientist! lu.ma/kiidiyeb
#DBRX sets a new standard for efficient open source LLMs. While it has 132B total parameters, with its fine-grained MoE architecture, DBRX only uses 36B at any given time. Learn more about how we built #DBRX & benchmarked its performance. dbricks.co/3J13hed
Meet #DBRX: a general-purpose LLM that sets a new standard for efficient open source models. Use the DBRX model in your RAG apps or use the DBRX design to build your own custom LLMs and improve the quality of your GenAI applications. dbricks.co/43xaCMj
ICYMI: earlier this month, @NaveenGRao, @matei_zaharia, @jefrankle, and guest speakers from @Accenture and @ADP dropped some knowledge on enterprise #GenAI implementation methods.
ICYMI: earlier this month, @NaveenGRao, @matei_zaharia, @jefrankle, and guest speakers from @Accenture and @ADP dropped some knowledge on enterprise #GenAI implementation methods.
In this @mlopscommunity episode, MosaicML's @davisblalock and @bandish share war stories and lessons learned from pushing the limits of #LLM training and helping dozens of customers get LLMs into production. 🤝 👀 Watch the full episode: home.mlops.community/public/videos/… #mlops #LLMs
Training LLMs is tough work and lots can go wrong. Scale 📈is hard and things break 💔, often. Listen to @davisblalock and me break it down and get some insights💡into why developing custom GenAI on @databricks + @DbrxMosaicAI is the best solution for enterprises!
Training LLMs is tough work and lots can go wrong. Scale 📈is hard and things break 💔, often. Listen to @davisblalock and me break it down and get some insights💡into why developing custom GenAI on @databricks + @DbrxMosaicAI is the best solution for enterprises!
The @databricks Mosaic Research team is committed to building the best possible training stack for #LLMs and #genAI models. @mvpatel2000, @davisblalock, Saaketh Narayan, and Cheng Liang write about our latest benchmark results and training speedup methods here:…
Since becoming part of @databricks last July, the MosaicML team has continued its mission to optimize and improve #GenAI model training. Our rigorous science leads to real-world results. Visit our new research hub to discover what we've working on: databricks.com/research/mosaic
Jim Fan @DrJimFan
229K Followers 3K Following @NVIDIA Sr. Research Manager & Lead of Embodied AI (GEAR Lab). Creating foundation models for Humanoid Robots & Gaming. @Stanford Ph.D. @OpenAI's first intern.
Soumith Chintala @soumithchintala
186K Followers 877 Following Cofounded and lead @PyTorch at Meta. Also dabble in robotics at NYU. AI is delicious when it is accessible and open-source.
Horace He @cHHillee
23K Followers 449 Following Working at the intersection of ML and Systems @ PyTorch "My learning style is Horace twitter threads" - @typedfemale
Eric Jang @ericjang11
69K Followers 3K Following physical AGI at 1X. Author of "AI is Good for You" https://t.co/eFg4WXhg0p
Percy Liang @percyliang
49K Followers 408 Following Associate Professor in computer science @Stanford @StanfordHAI @StanfordCRFM @StanfordAILab @stanfordnlp | cofounder @togethercompute | Pianist
Jeremy Howard @jeremyphoward
222K Followers 5K Following 🇦🇺 Co-founder: @AnswerDotAI & @FastDotAI ; Hon Professor: @UQSchoolITEE ; Digital Fellow: @Stanford
Jonathan Frankle @jefrankle
16K Followers 685 Following Chief Scientist, Neural Networks @Databricks via MosaicML. PhD @MIT_CSAIL. BS/MS @PrincetonCS. DC area native. Making AI efficient for everyone at @DbrxMosaicAI
Rosanne Liu @savvyRL
33K Followers 966 Following Cofounded & running @ml_collective. Host of Deep Learning Classics & Trends. Research at Google DeepMind. DEI/DIA Chair of ICLR & NeurIPS. Writing https://t.co/IbycyGfnDR
Delip Rao e/σ @deliprao
46K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈
Naveen Rao @NaveenGRao
28K Followers 785 Following VP GenAI @Databricks. Former CEO/cofounder MosaicML & Nervana/IntelAI. Neuro + CS. I like to build stuff that will eventually learn how to build other stuff.
rohan anil @_arohan_
12K Followers 2K Following Principal Engineer, @GoogleDeepMind Gemini. prev PaLM-2. Tinkering with optimization and distributed systems. opinions are my own.
Tim Dettmers @Tim_Dettmers
29K Followers 819 Following PhD Student at @UW. I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.
Davis Blalock @davisblalock
12K Followers 165 Following Research scientist + first hire @MosaicML. @MIT PhD. I write + retweet threads about machine learning papers. Paper summaries newsletter: https://t.co/xX7NIpsIVZ
Dan Roy @roydanroy
45K Followers 2K Following ML / AI researcher, emphasis on theory. Research Director and Canada CIFAR AI Chair, @VectorInst Professor, @UofT (Statistics/CS)
Ethan Caballero is bu.. @ethanCaballero
8K Followers 2K Following ML PhD student @Mila_Quebec ; previously @GoogleDeepMind
Kyunghyun Cho @kchonyc
61K Followers 2K Following a combination of a mediocre scientist, a mediocre manager, a mediocre advisor & a mediocre PC at @nyuniversity (@CILVRatNYU) & @genentech (@PrescientDesign).
Sara Hooker @sarahookr
39K Followers 7K Following I lead @CohereForAI. Formerly Research @Google Brain @GoogleDeepmind. ML Efficiency at scale, LLMs, @trustworthy_ml. Changing spaces where breakthroughs happen.
Omar Sanseviero @osanseviero
31K Followers 2K Following Chief Llama Officer @huggingface 🦙 Founder @AI_Learners. Xoogler (SWE @Google Assistant, 20% PM TF Graphics). 100% Hacker Llama🇵🇪🇲🇽
Cameron R. Wolfe, Ph... @cwolferesearch
21K Followers 622 Following Director of AI @RebuyEngine • Writer @ Deep (Learning) Focus • PhD @optimalab1 • I make AI understandable
Shane Gu @shaneguML
28K Followers 1K Following Research Scientist & Manager @GoogleDeepMind Tokyo/MTV. ex: @GoogleAI Brain, @OpenAI. (JP: @shanegJP)
ROCK EMPEROR 🪨🪨.. @Rockorites
732 Followers 4K Following I Like Collecting Rocks, I love collecting heavy Rocks, I LOVE ROCKS.🪨🪨🪨🪨🪨🪨🪨🪨🪨🪨🪨🪨
Abhishek 🇮🇳 @abhishekbujju
198 Followers 965 Following
Rohan Bhasin @the_rohanbhasin
16 Followers 144 Following
Russ Thompson @platformpilot89
240 Followers 1K Following Chief #Engineer by day, #AI enthusiast by night. If it's not automated, I haven't touched it yet!
albert @albert55281844
0 Followers 25 Following
Parmanad khatik Parma.. @khatik_parm6028
49 Followers 308 Following
Mbaku @DCSwann1
51 Followers 3K Following
jackiechannelchannel @lance_tupperman
0 Followers 60 Following
Reza Sayar @iamRezaSayar
167 Followers 655 Following 👨🏻🎓Life-long Learner👨🏻🎓 Kindness❤️, Helpfulness🫂 , AI🧠 & Reggaetón💃🏻
Vidhya @whaats_that
3 Followers 61 Following
Gentium @GentiumAI
9K Followers 153 Following We are empowering AI innovation through decentralization. Join Gentium to create, collaborate, and earn with AI agents. #DecentralizeAI #GentiumAI
jr @jamesrichmanx
16K Followers 138 Following
dig8italX @dig8italX
134 Followers 2K Following dig8italX, the leading artificial intelligence firm that specializes in creating customized AI solutions for businesses.
Alok Shah @alokshah1504
9 Followers 24 Following
coffee & AI @realcoffeeAI
41 Followers 579 Following
MOHAMED MASKITTOU @MMaskittou
2 Followers 62 Following
Matt Heap @matt_heap
132 Followers 597 Following
Akshay Sankar @sankarakshay1
133 Followers 3K Following
B Barrett @brossbarrett
149 Followers 844 Following Managing Director of Registered Investment Advisory. For informational purposes only and is not intended to be personal financial advice.
Paolo V. @pfvaldez1
24 Followers 985 Following
Fei Wu @YangFei1990
0 Followers 9 Following
niccolasmunoz @niccolasmunoz
1K Followers 2K Following Father and husband. #buildinpublic Azomland. Technology, media, investments and people.
thethinker @tworoadsdivergd
157 Followers 5K Following “A child of five would understand this. Send someone to fetch a child of five.”
Mira Kwak (Irma Snow) @aleph0
1K Followers 5K Following Head, Art Chaosmos/ Phaidalos, Mystral, Mystral Andel/ mind, transhumanism, cybernetics, semiotics, truth, freedom, metaverse/ Velvet Goldmine, Remy Martin
Debidatta Dwibedi @debidatta
821 Followers 2K Following Senior Research Scientist @GoogleDeepMind, Previously Robotics @CarnegieMellon, EE @IITKanpur, StudApps (https://t.co/iDVr86IjhA)
Brite_Future @BriteFuture21
41 Followers 499 Following We should always have gratitude for all the blessings we have been given and take them for granted.
Michael @MichaelLuJY
24 Followers 285 Following
Hegel Angel @HegelAngel
71 Followers 481 Following #HegelAngel "Acid-Mom&RockBaby escaped blood & madness at Rolling Stones gig, Altamont". Raised neo-lacancartesian sophistofeliner * Jehovah Whitney, Houston,TX
Jeff Xu @jjx003
61 Followers 1K Following
bren @OtherscallmeB
1 Followers 482 Following
the_sven @the___sven
1 Followers 92 Following Sr. Fullstack developer with an interest in Data Engineering and Machine Learning.
Thabiso Moleko @ThabisoMol46642
407 Followers 2K Following
Wenzhao Qiu @WenzhaoQiu
3 Followers 138 Following
Duyen V. Mai @_negordyh_
5 Followers 140 Following 'But only dead fish go with the flow!' - Salmon thought, swam away.
Jim Fan @DrJimFan
229K Followers 3K Following @NVIDIA Sr. Research Manager & Lead of Embodied AI (GEAR Lab). Creating foundation models for Humanoid Robots & Gaming. @Stanford Ph.D. @OpenAI's first intern.
Soumith Chintala @soumithchintala
186K Followers 877 Following Cofounded and lead @PyTorch at Meta. Also dabble in robotics at NYU. AI is delicious when it is accessible and open-source.
Jeremy Howard @jeremyphoward
222K Followers 5K Following 🇦🇺 Co-founder: @AnswerDotAI & @FastDotAI ; Hon Professor: @UQSchoolITEE ; Digital Fellow: @Stanford
Jonathan Frankle @jefrankle
16K Followers 685 Following Chief Scientist, Neural Networks @Databricks via MosaicML. PhD @MIT_CSAIL. BS/MS @PrincetonCS. DC area native. Making AI efficient for everyone at @DbrxMosaicAI
Rosanne Liu @savvyRL
33K Followers 966 Following Cofounded & running @ml_collective. Host of Deep Learning Classics & Trends. Research at Google DeepMind. DEI/DIA Chair of ICLR & NeurIPS. Writing https://t.co/IbycyGfnDR
Hugging Face @huggingface
343K Followers 189 Following The AI community building the future. https://t.co/VkRPD0VKaZ #BlackLivesMatter #stopasianhate
Naveen Rao @NaveenGRao
28K Followers 785 Following VP GenAI @Databricks. Former CEO/cofounder MosaicML & Nervana/IntelAI. Neuro + CS. I like to build stuff that will eventually learn how to build other stuff.
Davis Blalock @davisblalock
12K Followers 165 Following Research scientist + first hire @MosaicML. @MIT PhD. I write + retweet threads about machine learning papers. Paper summaries newsletter: https://t.co/xX7NIpsIVZ
Cameron R. Wolfe, Ph... @cwolferesearch
21K Followers 622 Following Director of AI @RebuyEngine • Writer @ Deep (Learning) Focus • PhD @optimalab1 • I make AI understandable
Oriol Vinyals @OriolVinyalsML
166K Followers 82 Following VP of Research & Deep Learning Lead, Google DeepMind. Gemini co-lead. Past: AlphaStar, AlphaFold, AlphaCode, WaveNet, seq2seq, distillation, TF.
Stella Biderman @BlancheMinerva
15K Followers 748 Following Open source LLMs and interpretability research at @BoozAllen and @AiEleuther. My employers disown my tweets. She/her
Mosiac Research @MosaicResearch
4 Followers 0 Following
shreya rajpal @ShreyaR
6K Followers 767 Following ML, systems, and everything in between. Building @guardrails_ai. Previously founding eng @predibase, @Apple SPG, @driveai_, @IllinoisCS, @iitdelhi.
swyx @swyx
91K Followers 3K Following Anti-ego ideas for anti-ergodic life. Founder, @smolmodels ▹ Listen: @latentspacepod ▹ Read: @coding_career ▹ Join: @aiDotengineer
Zack Ankner @ZackAnkner
487 Followers 306 Following Junior @MIT. President of AI@MIT. Research Scientist Intern @MosaicML. A(CL)verage Embargo enjoyer.
Linden Li @lindensli
1K Followers 534 Following CS @Stanford, @StanfordSVL. Research/Eng @MosaicML, previously @NVIDIA.
Allen Institute for A.. @allen_ai
53K Followers 361 Following AI for the Common Good. › Join us: https://t.co/DqTs1G4bGO › Get our newsletter: https://t.co/tvb1VpySfL
Daya @dskhudia
175 Followers 113 Following
Kristen Richardson @butwhyevernot
2K Followers 2K Following Book: The Season: A Social History of the Debutante (W.W. Norton: 2019). Inquiries via the Wylie Agency.
Dan Biderman @dan_biderman
596 Followers 871 Following Final-year PhD student at @cu_neurotheory building ML systems for neuroscience. Also NLP research @DbrxMosaicAI
Databricks @databricks
70K Followers 1K Following Databricks is the data and AI company, helping data teams solve the world’s toughest problems.
Hagay Lupesko @hagay_lupesko
243 Followers 88 Following VP of Software Engineering at MosaicML, making ML efficient and accessible for the masses. DM me to learn more!
Matei Zaharia @matei_zaharia
39K Followers 1K Following CTO at @Databricks and CS prof at @UCBerkeley. Working on data+AI, including @ApacheSpark, @DeltaLakeOSS, @MLflow, https://t.co/94gROE5Xa0. https://t.co/nmRYAKG0LZ
Shahin Farshchi @Farshchi
9K Followers 894 Following @lux_capital, invested @zoox, @planet, @relativityspace, @vardaspace, @epsilon3inc, @nervanasys, @mosaicml, @CovariantAI, @goformic, $AEVA, Dad/Bear/Bruin/Pilot
Replit ⠕ @Replit
122K Followers 1K Following Idea to software, fast. Build and deploy software collaboratively with the power of AI without spending a second on setup. Need help? @ReplitSupport
Gradient Flow @GradFlowTech
30 Followers 7 Following Official Account of Gradient Flow (https://t.co/xh4UsbDEgS) and The Data Exchange podcast (https://t.co/23gJEo92zo)
Perplexity @perplexity_ai
132K Followers 28 Following Our mission is to serve the world’s curiosity. https://t.co/BBZ1kG0TVG
Stability AI @StabilityAI
189K Followers 31 Following We are building the foundation to activate humanity's potential.
Sally Ward-Foxton @sallywf
2K Followers 1K Following Senior Reporter at @eetimes, reporting mainly on AI accelerators.
Chase Lochmiller @ChaseLochmiller
3K Followers 2K Following CEO and Co-Founder of @CrusoeEnergy Former @polychain, @jumptrading, @Stanford, @MIT
"nicole" @ninklefitz
1K Followers 517 Following master of decorum @alpacaml. prev: @MicrosoftResearch, @MosaicML, @Mila_Quebec
Barry Dauber @barrydauber
694 Followers 468 Following VP of Mosaic AI GTM @DbrxMosaicAI / @Databricks, DC Native, Texas Longhorn
Jesse Dodge @JesseDodge
3K Followers 2K Following Senior Research Scientist at AI2 @ai2_allennlp. Responsibly open work on the science of AI and AI for science. Environmental impact of AI. he/him 🏳️🌈
Vitaliy Chiley @vitaliychiley
2K Followers 607 Following Head of NLP Pretraining @Databricks / @MosaicML | Former @CerebrasSystems | What do we want? FLOPS! When do we want it? TOKENS!
OpenAI @OpenAI
3.4M Followers 0 Following OpenAI’s mission is to ensure that artificial general intelligence benefits all of humanity. We’re hiring: https://t.co/dJGr6LgzPA
Oracle Developers @OracleDevs
118K Followers 702 Following Oracle Developers is a community for developers by developers
Oracle @Oracle
820K Followers 825 Following Leading the cloud. We help people see data in new ways, discover insights, unlock endless possibilities.
Manish Kapur @kapmani
370 Followers 236 Following Tech guy | Pensive thinker | Sports fan | Views expressed are my own.
DavidLinthicum @DavidLinthicum
38K Followers 4K Following Cloud Computing visionary. CTO, CEO, blogger, speaker, best selling author. RT≠endorsements, all opinions are my own.
Song Han @songhan_mit
6K Followers 144 Following Assoc. Prof. @MIT, Distinguished Scientist @NVIDIA, cofounder of DeePhi (now part of AMD) and OmniML (now part of NVIDIA). PhD @Stanford. Efficient AI computing
Nina da Hora - tweets.. @ninadhora
70K Followers 12K Following Master’s student in Ethics in AI @unicamp. 2024 GlobalFellow @ fordfoundation. Director @ institutodahora. Decolonize science 🌈🏳️⚧️
Sahil Khose @SahilKhose
569 Followers 1K Following Incoming PhD @ Gatech @ICatGT | MSCS GaTech '24 🇺🇸| BTech MIT Manipal '22 🇮🇳
Sharon Zhou @realSharonZhou
23K Followers 1 Following Building the future of LLMs | Cofounder & CEO, @LaminiAI | Prev: CS Faculty & PhD @Stanford. Product @Google. @Harvard | @MIT 35 under 35. Angel investor.
Climate Change AI @ClimateChangeAI
12K Followers 362 Following Tackling climate change with machine learning. We facilitate cooperation and provide resources for those working in this area. RT is not endorsement.
David Rolnick @david_rolnick
4K Followers 371 Following Assistant Professor in Computer Science, @mcgillu / @Mila_Quebec. Co-Founder and Chair, @ClimateChangeAI. MIT @techreview Innovator Under 35. he/him/his
Priya L. Donti @priyald17
5K Followers 819 Following Asst Prof @MITEECS & LIDS. Co-founder & Chair @ClimateChangeAI. MIT @techreview 35 Innovators Under 35. she/they
Lynn Kaack @LynnHKaack
2K Followers 931 Following Assistant Prof @thehertieschool working on climate & energy policy and ML, co-founder and chair @ClimateChangeAI, previously @eth_epg & @CMU_EPP
ICML Conference @icmlconf
70K Followers 17 Following Int'l Conf on ML • July 21-27, 2024 (Vienna, Austria) • #icml2024 • Contact: https://t.co/6saHKWV01y • https://t.co/sFwmcQNWkE
John Maeda @johnmaeda
390K Followers 340 Following VP AI & Design at Microsoft / How To Speak Machine (2019) https://t.co/eb6gj2wf1bLlama3-70B has settled at #5. With 405B still to come next... I remember when GPT-4 released in March 2023, it looked like it was nearly-impossible to get to the same performance. Since then, I've seen @Ahmad_Al_Dahle and the rest of the GenAI org in a chaotic rise to focus,…
Exciting update -- Llama-3 full result is out, now reaching top-5 on the Arena leaderboard🔥 We've got stable enough CIs with over 12K votes. No question now Llama-3 70B is the new king of open model. Its powerful 8B variant has also surpassed many larger-size models. What an…
There is no question that AI will eventually reach and surpass human intelligence in all domains. But it won't happen next year. And it won't happen with the kind of Auto-Regressive LLMs currently in fashion (although they may constitute a component of it).…
just realized how powerfully Google has owned the primary colors as a brand. I mindlessly scrolled past this chart and assumed it was a Google-related tweet even though Google never appears here. nice eval results for @MistralAI and @DbrxMosaicAI btw!
HumanEval benchmark results for the new Mixtral 8x22B, DBRX-base, Qwen1.5-72B, and Mixtral 8x7B:
@jefrankle Super cool. Thanks so much for the commitment to open-source from both a model/weights and code perspective!
AI21 and Databricks show open source can radically slim down AI Two new large language models, Jamba and DBRX, dramatically reduce the compute and memory needed for predictions, while meeting or beating the performance of top models such as GPT-3.5 and Llama 2.…
I hit an error in my notebook and the @databricks Assistant politely told me what the cause was, how to work-a-around it, but it also told me that if I used another function, I would not have to use the work-a-round. Wow. (that compensated for all the times it was plain wrong!)
Open models FTW:
@DbrxMosaicAI DBRX outperforms @OpenAI GPT-4 on realistic, domain-specific benchmark datasets. For example, on a customer support summarization use-case👇👇👇 Still neck and neck but it shows that open models can be the no-brainer choice for actual enterprise applications.
#DBRX democratizes training + tuning of custom, high-performing LLMs so enterprises don't need to rely on a handful of closed models. Now, every organization can efficiently build production-quality GenAI applications while having control over their data. dbricks.co/3x8pxjK
Things are changing weekly but this seems to be the best open LLM for now.
It’s finally here 🎉🥳 In case you missed us, MosaicML/ Databricks is back at it, with a new best in class open weight LLM named DBRX. An MoE with 132B total parameters and 32B active 32k context length and trained for 12T tokens 🤯
We built a new model! 🧱 It's called DBRX 🧱 * mixture of experts * 16 choose 4 experts * 36B active, 132B total * trained on 12T tokens * built e2e in 2 months * using 3072xH100 * served up to 150 tok/s on @databricks * open weights :)
Congrats @abhi_venigalla and Mosaic Databricks team. You’re moving the AI industry forward, brick by brick 🧱 😘
We built a new model! 🧱 It's called DBRX 🧱 * mixture of experts * 16 choose 4 experts * 36B active, 132B total * trained on 12T tokens * built e2e in 2 months * using 3072xH100 * served up to 150 tok/s on @databricks * open weights :)
12T tokens?? 132B params??? you spent HOW MUCH on H100s?!
@code_star @jefrankle I can feel him vibrating through slack
What does it look like to knock a million dollars off the cost of training huge models? For us, it looked like this:
🚨New🌟blog✍️ on ⏩ maximizing🌙 FLOPS 🚀 Training large models requires maximizing flops/GPU, especially at scale. Excited to share a few of the cool tricks in thread👀. 1/N
@DbrxMosaicAI @databricks Love the new logo!
In this @mlopscommunity episode, MosaicML's @davisblalock and @bandish share war stories and lessons learned from pushing the limits of #LLM training and helping dozens of customers get LLMs into production. 🤝 👀 Watch the full episode: home.mlops.community/public/videos/… #mlops #LLMs
Training LLMs is tough work and lots can go wrong. Scale 📈is hard and things break 💔, often. Listen to @davisblalock and me break it down and get some insights💡into why developing custom GenAI on @databricks + @DbrxMosaicAI is the best solution for enterprises!
In this @mlopscommunity episode, MosaicML's @davisblalock and @bandish share war stories and lessons learned from pushing the limits of #LLM training and helping dozens of customers get LLMs into production. 🤝 👀 Watch the full episode: home.mlops.community/public/videos/… #mlops #LLMs
Wonderful to see @abhi_venigalla light up my feeds today. This man knows what he's talking about :)
Advancing AI: @databricks NLP Architect, Abhinav Venigalla, discusses the hardware and software advantages from AMD.
@databricks is the best platform to customize model behavior, and that includes grounding to feedback. We are constantly looking for ways to make Gen AI more reliable to make it usable in an enterprise context. 👇read below about some of the fundamental innovations in this area!
New blog post! @zeqiuwu1, @huyushi98, and @rajammanabrolu share a recent highlight from their work in #LLM finetuning research: Fine-Grained Reinforcement Learning from Human Feedback (RLHF) databricks.com/blog/fine-grai…
A recap of how to get better rewards for RLHF and a view into what I've been working on Scaling to production levels at Mosaic. We have so much more exciting work to show y'all vv soon
New blog post! @zeqiuwu1, @huyushi98, and @rajammanabrolu share a recent highlight from their work in #LLM finetuning research: Fine-Grained Reinforcement Learning from Human Feedback (RLHF) databricks.com/blog/fine-grai…
Trends for United States
You might like
