Apache TVM @ApacheTVM
Open deep learning compiler stack for CPUs, GPUs and specialized accelerators. Join us for the TVM and Deep Learning Compilation Conference https://t.co/i6MTbWYt87 tvm.apache.org Joined January 2018-
Tweets580
-
Followers3K
-
Following945
-
Likes543
#LLama3 on #SteamDeck ! All thanks to the #MLC_LLM library that enables you to effortlessly compile #llms for Cuda, ROC and Vulcan and since Steam Deck is using AMD GPU it wasn’t a problem at all. I managed to run Llama-3-8B-Instruct-q4f16 with almost 15 tokens/s! #localllms
Run #Llama3 locally on your android device using MLC-LLM! Download the mlc-llm APK from github.com/mlc-ai/binary-…. For more details, check out llm.mlc.ai/docs/
#Llama3 🦙🦙 running fully locally on iPad without internet connnection. credits to @ruihanglai and the team
📢We're thrilled to announce that Kurt Keutzer will give the keynote speech for MLSys 2024 Young Professionals Symposium. Welcome to join us for exciting invited talks by @Azaliamirh, Xupeng Miao, @jiawzhao , @ying11231 , @tri_dao on cutting-edge MLSys research! The full…
#MLSys2024 Student Travel Grant just get announced The deadline for applications is 4/24/24. Checkout Young Professional symposium chaired by @BeidiChen and @guanh01 ! See mlsys.org for further details.
#MLSys2024 early registration deadline is coming up in six days. The conference will happen May 13th through Thu the 16th at the Santa Clara Register today at mlsys.org/Register and looking forward to seeing everyone there!
WebGPU just got more powerful 🔥 in Chrome 124: ✅ Read-only and read-write storage textures ✅ Service workers and shared workers support ✅ New adapter information attributes ✅ Bug fixes Check out developer.chrome.com/blog/new-in-we… to learn more.
If you want to learn more about the latest advances in AI and systems, such as systems for diffusion models, multi-LoRA serving, MOE, efficient quantization and systems, and more AI and systems topics. Check out this year’s #MLSys2024 program. The conference will happen May 13th…
🌟 My fully-local chat with PDF repo doubled in stars this week - now at more than 1.2k! So I revisited WebLLM and was able to add browser-only mode! Here, the entire @LangChainAI ingest + RAG flow runs with the model weights cached in browser storage! webml-demo.vercel.app
More updates from WebLLM coming soon!
We are excited to announce the technical program for MLSys 2024! The provisional set of accepted papers is now available on the website at mlsys.org/Conferences/20…. Register for MLSys now at mlsys.org/Register/
🚀Exciting news! Join us at MLSys 2024 Young Professionals Symposium on May 13th in Santa Clara. 🎓Dive into discussions on large model training, industry vs. academia, entrepreneurship, and more. Don’t miss this chance to connect with experts & peers in the field! #MLSys2024 🔥
Please spread the words, #MLSys2024 will feature a full day single track-event young professional symposium with invited talks, panels, round tables, and poster sessions. Submit your 1-page abstract by April 1st & present your work at our poster session. sites.google.com/view/mlsys24yps
Checkout accepted papers from #MLSys2024, register today! 👉
Checkout accepted papers from #MLSys2024, register today! 👉
Please help spread the words📢 If you are in the field of AI and interested in the latest innovations in machine learning and systems. You should checkout #MLSys24!
Please help spread the words📢 If you are in the field of AI and interested in the latest innovations in machine learning and systems. You should checkout #MLSys24!
Wow that was fast. Google Gemma running in the browser locally - look ma, no server!
Wow that was fast. Google Gemma running in the browser locally - look ma, no server!
Besides Android, iOS, and web browsers, Gemma is also supported in MLC LLM on various GPUs! A single model definition does it all -- thanks to the ML compiler infra lead by @junrushao and many others! Try it in Google Colab: github.com/mlc-ai/noteboo…
PyTorch @PyTorch
379K Followers 77 Following Tensors and neural networks in Python with strong hardware acceleration. PyTorch is an open source project at the Linux Foundation. #PyTorchFoundation
Horace He @cHHillee
23K Followers 449 Following Working at the intersection of ML and Systems @ PyTorch "My learning style is Horace twitter threads" - @typedfemale
Soumith Chintala @soumithchintala
186K Followers 877 Following Cofounded and lead @PyTorch at Meta. Also dabble in robotics at NYU. AI is delicious when it is accessible and open-source.
Tianqi Chen @tqchenml
15K Followers 973 Following AssistProf @mldcmu and @CSDatCMU. Chief Technologist @OctoML. Creator of @XGBoostProject, @ApacheMXNet, @ApacheTVM. Member https://t.co/QYyfjQNp4p, @TheASF.
Song Han @songhan_mit
6K Followers 144 Following Assoc. Prof. @MIT, Distinguished Scientist @NVIDIA, cofounder of DeePhi (now part of AMD) and OmniML (now part of NVIDIA). PhD @Stanford. Efficient AI computing
Vinod Grover @vinodg
2K Followers 1K Following Sr Distinguished Engineer @nvidia. Compilers, CUDA C++, PL, Machine Learning and Systems. tweets and opinions are personal.
Talia Ringer 🟣 �.. @TaliaRinger
26K Followers 6K Following Professor, @plfmse, @IllinoisCS! Proof Automation. @SigplanM & CCF Founder. Israeli-American for peace, equality, & justice. They/היא, ND, bi. די לכיבוש
Edward Z. Yang @ezyang
10K Followers 971 Following I work on PyTorch at Meta. Chatty alt at @difficultyang. Mastodon @[email protected]
Luis Ceze @luisceze
3K Followers 2K Following computer architect. marveled by biology. professor @uwcse. ceo @OctoAICloud. venture partner @madronaventures.
Jeff Dean (@🏡) @JeffDean
296K Followers 6K Following Chief Scientist, Google DeepMind and Google Research. Co-designer/implementor of things like @TensorFlow, MapReduce, Bigtable, Spanner, Gemini .. (he/him)
Jeremy Howard @jeremyphoward
222K Followers 5K Following 🇦🇺 Co-founder: @AnswerDotAI & @FastDotAI ; Hon Professor: @UQSchoolITEE ; Digital Fellow: @Stanford
Aart Bik @AartBik
1K Followers 800 Following 🇳🇱🇺🇲 Dutch-American computer scientist Utrecht (MSc), Leiden (PhD) @Google @Intel #MLIR #LLVM #astronomy #chess #compilers #simd #sparse #vectorization
Nadav Rotem @nadavrot
4K Followers 427 Following Engineering director at Facebook. Interested in systems, compilers, ML, performance, and other stuff. 🇮🇱
Delip Rao e/σ @deliprao
46K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈
Michel Steuwer @MichelSteuwer
527 Followers 193 Following Mastodon: @[email protected] | https://t.co/jzWdXuCM7V | Professor | Chair of #Compilers and #ProgrammingLanguages at @TUBerlin
Neural Magic @neuralmagic
5K Followers 2K Following Deploy the fastest ML on CPUs and GPUs using only software. GitHub: https://t.co/99a5S2627M #sparsity #opensource
Ross Wightman @wightmanr
18K Followers 1K Following Computer Vision @ 🤗. Ex head of Software, Firmware Engineering at a Canadian 🦄. Currently building ML, AI systems or investing in startups that do it better.
Angel Madrueno @MadruenAng
72 Followers 5K Following
Michaela Redstone @MichaeRedston
50 Followers 5K Following
Science🧪Art🖌️.. @SciArtMagic
78 Followers 421 Following (and a whole lot of math I still don't understand) Senior developer but AI/ML noob. Let this be a forcing function for me to build in public.
Agatha Dudman @AgathaDudm
42 Followers 5K Following
Celestina Luedke @CelestinLued
96 Followers 5K Following
Libby Ernspiker @LErnspiker60159
83 Followers 5K Following
Camila Metoxen @CamilaMeto8072
47 Followers 5K Following
Jeff Sullivan @JeffSul27156104
229 Followers 2K Following
Rose Cryar @cryar_ro
65 Followers 5K Following
Soulana @DreamPC380705
493 Followers 3K Following saga monke with heart and sol and a samoyed || Labor market expert 🇪🇺 || Translator 🇵🇱🇩🇪 || AI & Blockchain enthusiast || Solana ecosystem adventurer
Elizebeth Horwich @EHorwi
33 Followers 5K Following
Test @BakoczRagtag
0 Followers 57 Following
Eileen Pignataro @PignatEile
44 Followers 5K Following
王毅 @Tougerwy
2 Followers 91 Following
Armida Bauerlein @ABauerlein65951
63 Followers 5K Following
Mingheng Wu @wmhst7
31 Followers 131 Following MLSys | Master at Tsinghua and UW | Bachelor at Tsinghua
SenaBeren @findingmerit
296 Followers 3K Following
Verena Dolven @v_dolve
82 Followers 5K Following
Sandee Stuber @SandeeStub47243
71 Followers 5K Following
Harper Lavole @LavHarp
42 Followers 5K Following
AgentC @AgentC825479
2 Followers 33 Following
Macie Finck @FincMa
79 Followers 5K Following
Abdurrahman Guner @abdrrhmnguner
34 Followers 127 Following
Aravind Choutpally @achoutpally
49 Followers 820 Following #DistributedSystems #VLDB #Data #Databricks #Streaming #ApacheFlink #Kafka
Hayley Rhinehardt @HayleyRhin44364
94 Followers 5K Following
Eric Auld @AuldEric
313 Followers 688 Following AI, math, CS. Former @uclamath. I’ll let you be in my dream if I can be in yours
Shiyi Cao @shiyi_c98
396 Followers 361 Following PhD student @UCBerkeley, MSc @ETH, B.S @sjtu1896, systems, ml, and hpc
Eugene Terentev @eugenvector
16 Followers 119 Following Founder @soedged — we make machines intelligent. I tweet about AI, the future, and whatever else catches my fancy.
Tony @liutuo1111
18 Followers 171 Following
vibha @vibhamasti
461 Followers 1K Following she/her. Master’s @LTIatCMU. Prev: ML @ Apple India. Bachelor’s @PESUniversity. Not a professional account.
zk @_zk67
1 Followers 504 Following Software Engineer of Search Infra/ML Infra/Ads Infra/Distributed Systems
Keeley Sibilio @KeelSibi
42 Followers 5K Following
Lanxiang Hu @Lanxiang_Hu
34 Followers 159 Following PhD Student @UCSDJacobs. AI & Systems. Prev. @UCBerkeley.
@[email protected].. @climate_dad
1K Followers 4K Following Husband & dad. Technologist-activist. Thinks about #IrreversiblePlanetaryDegradation. Knows ML/AI/SWE, education, dataviz. 🇻🇳🇺🇸
Andrej Karpathy @karpathy
978K Followers 904 Following 🧑🍳. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥
Yann LeCun @ylecun
711K Followers 718 Following Professor at NYU. Chief AI Scientist at Meta. Researcher in AI, Machine Learning, Robotics, etc. ACM Turing Award Laureate.
PyTorch @PyTorch
379K Followers 77 Following Tensors and neural networks in Python with strong hardware acceleration. PyTorch is an open source project at the Linux Foundation. #PyTorchFoundation
François Chollet @fchollet
469K Followers 770 Following Deep learning @google. Creator of Keras. Author of 'Deep Learning with Python'. Opinions are my own.
Horace He @cHHillee
23K Followers 449 Following Working at the intersection of ML and Systems @ PyTorch "My learning style is Horace twitter threads" - @typedfemale
Soumith Chintala @soumithchintala
186K Followers 877 Following Cofounded and lead @PyTorch at Meta. Also dabble in robotics at NYU. AI is delicious when it is accessible and open-source.
Tianqi Chen @tqchenml
15K Followers 973 Following AssistProf @mldcmu and @CSDatCMU. Chief Technologist @OctoML. Creator of @XGBoostProject, @ApacheMXNet, @ApacheTVM. Member https://t.co/QYyfjQNp4p, @TheASF.
MIT CSAIL @MIT_CSAIL
298K Followers 22K Following MIT's Computer Science & Artificial Intelligence Laboratory (CSAIL). Media Inquiries: [email protected]
AI at Meta @AIatMeta
531K Followers 255 Following Together with the AI community, we are pushing the boundaries of what’s possible through open science to create a more connected world.
Song Han @songhan_mit
6K Followers 144 Following Assoc. Prof. @MIT, Distinguished Scientist @NVIDIA, cofounder of DeePhi (now part of AMD) and OmniML (now part of NVIDIA). PhD @Stanford. Efficient AI computing
elvis @omarsar0
189K Followers 486 Following Building with LLMs @dair_ai • Prev: Meta AI, Galactica LLM, PapersWithCode, Elastic, PhD • Creator of the Prompting Guide (~4M learners)
Tim Dettmers @Tim_Dettmers
29K Followers 820 Following PhD Student at @UW. I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.
Satnam Singh @satnam6502
14K Followers 3K Following Punjabi-Scottish-American Haskell hacker at @GroqInc, cook, cyclist, lost in music. ∃🇮🇳 ∧ ∀🇬🇧 ∧ ∃🇪🇺 ∧ ∀🇺🇸 #celiac ex-{Microsoft, Google, Facebook}
Vinod Grover @vinodg
2K Followers 1K Following Sr Distinguished Engineer @nvidia. Compilers, CUDA C++, PL, Machine Learning and Systems. tweets and opinions are personal.
Andy Pavlo (@andy_pav.. @andy_pavlo
29K Followers 205 Following Associate Prof. of Databases @CarnegieMellon. Co-Founder @OtterTuneAI
Talia Ringer 🟣 �.. @TaliaRinger
26K Followers 6K Following Professor, @plfmse, @IllinoisCS! Proof Automation. @SigplanM & CCF Founder. Israeli-American for peace, equality, & justice. They/היא, ND, bi. די לכיבוש
Edward Z. Yang @ezyang
10K Followers 971 Following I work on PyTorch at Meta. Chatty alt at @difficultyang. Mastodon @[email protected]
Science🧪Art🖌️.. @SciArtMagic
78 Followers 421 Following (and a whole lot of math I still don't understand) Senior developer but AI/ML noob. Let this be a forcing function for me to build in public.
Zhijing Jin @ZhijingJin
3K Followers 1K Following Final-year PhD @MPI_IS & @ETH_en w/ @bschoelkopf. Research on (1) @CausalNLP and (2) NLP4SocialGood @NLP4SG. Mentor and mentee @ACLMentorship.
Rui Pan 潘瑞 @ruipeterpan
403 Followers 2K Following 2nd-yr Ph.D. student @PrincetonCS working on systems for ML, interning @aws this summer, previously @maxplanckpress @WisconsinCS, fan of @fcbarcelona
hazyresearch @HazyResearch
7K Followers 1K Following A research group in @StanfordAILab working on the foundations of machine learning & systems. https://t.co/JHK58TDorG Ostensibly supervised by Chris Ré
Christopher De Sa @chrismdesa
409 Followers 24 Following
Shiyi Cao @shiyi_c98
396 Followers 361 Following PhD student @UCBerkeley, MSc @ETH, B.S @sjtu1896, systems, ml, and hpc
Zephyr Project @ZephyrIoT
10K Followers 1K Following An #opensource project that builds a safe, secure & flexible RTOS for resource-constrained devices. #ZephyrRTOS #ZephyrDevSummit
Guangxuan Xiao @Guangxuan_Xiao
1K Followers 513 Following Ph.D. student at @MITEECS Prev: CS & Finance @Tsinghua_Uni
Sanjoy Das @_sanjoydas
959 Followers 468 Following DL Compiler lead @NVIDIA, Ex-@Cruise, Ex-@Google, Ex-@AzulSystems.
Pavel Larionov @pa1ar
205 Followers 407 Following ｢ healthtech and digital things ｣ orchestrating inception of digital constructs
Rulin Shao @RulinShao
617 Followers 396 Following PhD @UWNLP | MS @SCSatCMU | ex-Applied Scientist @AWS
Ben Holfeld @BenHolfeld
89K Followers 32K Following SF AI Studio Lead @Accenture, partnering with @OpenAI @Google @Microsoft. Pianist. German Quantum Physicist. Creator of the Nth Floor. Views are my own. x/acc.
Jeffery Kaneda 金�.. @JefferyTatsuya
14K Followers 1K Following https://t.co/XMkEfLzyoK https://t.co/0qz9yvuFCZ https://t.co/GpfkVcbZfM https://t.co/mlvyUoCrnI MyCircleO: https://t.co/JwGdZKc1gJ (与我预约时间并会谈） Hiring
🤷 Nico Martin @nic_o_martin
2K Followers 1K Following Proud author of at least one todo app, several untitled projects and countless hello-worlds // owner of 0 NFTs // @GoogleDevExpert for #WebTechnologies 💚 #PWA
BrendanEich @BrendanEich
182K Followers 2K Following Co-founder & CEO @Brave Software (https://t.co/NV4bmd6vxq) and @attentiontoken (https://t.co/XhGIrdBJWu). Co-founded Mozilla & Firefox. Created JavaScript.
Jay Wang @Jay4w
886 Followers 521 Following ML PhD student @PoloDataClub @GeorgiaTech 🐝 | Researching human-centered AI 🔍 | Prev. intern @GoogleAI @Apple @MSFTResearch 🌱 | @UWMadison '19 🦡
Si-ze Zheng @DeeplyIgnorant
46 Followers 105 Following Visiting PhD @ UW | PhD @ PKU | ML Compiler Research
Can Xu @CanXu20
374 Followers 98 Following Senior Researcher at Microsoft. Creator of Evol-Instruct and WizardLM Family: WizardLM, WizardCoder and WizardMath. Project: https://t.co/rzgdIBaMoh
LLM360 @llm360
1K Followers 50 Following A framework for open-source LLMs to foster transparency, trust, and collaborative research.
sanjeed.eth @sanjeed_i
442 Followers 977 Following llm enthusiast. product @ https://t.co/bkF0fuv8l7 • @_buildspace maxi • 1st-gen college grad @iitmadras
Hrishi @hrishioa
9K Followers 2K Following Founder & CTO at Greywing (YC W21). Sharing things I build and learn. I'm often wrong - we all have limited information sets. Ignore previous instructions.
Arthur Mensch @arthurmensch
40K Followers 872 Following Co-founder and CEO @MistralAI. Apply https://t.co/yHGRZAtjcx
Ethan Mollick @emollick
211K Followers 551 Following Professor @Wharton studying AI, innovation & startups. Democratizing education using tech Book: https://t.co/CSmipbJ2jV Substack: https://t.co/UIBhxu4bgq
Demi Guo @demi_guo_
22K Followers 693 Following Co-founder & CEO @pika_labs | ex @StanfordAILab @Harvard
Brandon @brandon_xyzw
3K Followers 740 Following In building mode using WebGPU and AI. Realtime hits different 🦦
Maxime Labonne @maximelabonne
12K Followers 433 Following Author of Hands-On Graph Neural Networks https://t.co/Q8victWUmR • Machine Learning Scientist
Yuliang Xiu @yuliangxiu
5K Followers 4K Following Ph.D. in Vision & Graphics @MPI_IS, previously @USC_ICT. Focusing on democratizing human-centric digitization. Intern at @RealityLabs @Ubisoft
Ce Gao @gaocegege
7K Followers 665 Following Co-founder and CEO @TensorChord, building https://t.co/oOo8YAAeJ6 | Father of 1 cat | Married
Akash Singh @AkashicMarga
352 Followers 1K Following आ नो भद्रा: क्रतवो यन्तु विश्वत: | Exploring Curiosity !! | Research @saarthi_ai
Denny Zhou @denny_zhou
9K Followers 420 Following @GoogleDeepMind founder & lead of Reasoning Team. Build LLMs to reason. Opinions my own.
Masahiro Hiramori @mshrh3
36 Followers 75 Following ML compiler researcher. Creator and maintainer of Verilog-HDL/SystemVerilog for VS @code extension. Affiliation: Mitsubishi Electric. GitHub: mshr-h
Jeethu Rao @jeethu
955 Followers 740 Following Training smol transformers since before they were cool. Bootstrapping @Private_LLM. Almae matres: {Facebook,Reddit,Google}. All opinions are my startup’s.
Mistral AI @MistralAI
90K Followers 0 Following Fast, open-source and secure language models. Join us https://t.co/INALdNGvCP
Babylon.js @babylonjs
10K Followers 130 Following Powerful, beautiful, simple, and open 3D for everyone on the web. https://t.co/EC8VPmwYsM blog: https://t.co/si4tsmNUNe #webgl #webxr #pbr #gltf #webgpu
David Pissarra @davidpissarra
105 Followers 114 Following Research Intern @CSDatCMU | MSc @istecnico | Prev: @Tsinghua_Uni
Zeyuan Allen-Zhu @ZeyuanAllenZhu
8K Followers 273 Following physics of language models @ Meta / FAIR IOI - USACO - MCM - ACM/ICPC - Codejam Tsinghua - MIT - Princeton/IAS - MSR - FAIR
Tianle Cai @tianle_cai
5K Followers 4K Following ML PhD @Princeton. Life-long learner, hacker, and builder. Tech consultant & angel investor. Prev @togethercompute @GoogleDeepMind @MSFTResearch @citsecurities.
nisten @nisten
10K Followers 5K Following fullstack-dev democratizing intelligence @skunkworks_ai | 🦝.ai | prev https://t.co/68jAlAVBKR |
Xinyun Chen @xinyun_chen_
4K Followers 840 Following Research Scientist at @GoogleDeepMind. PhD from @Berkeley_EECS.
Kirito (e/acc) 🏴�.. @bronzeagepapi
3K Followers 5K Following engineer scientist artist –– moloch disrespectoor // qualia connoisseur // tensor whisperer // epistemology enjoyer // kardashev mechanic // bounty hunter
Rick Lamers @RickLamers
2K Followers 866 Following 👨💻 AI Research & Engineering @GroqInc. I publish a weekly update about LLM Engineering on Substack, it’s free. Opinions are my own.❓Wanna host a Llama2-7B-128K (14GB weight + 64GB KV cache) at home🤔 📢 Introducing TriForce! 🚀Lossless Ultra-Fast Long Seq Generation — training-free Spec Dec! 🌟 🔥 TriForce serves with 0.1s/token on 2 RTX4090s + CPU – only 2x slower on an A100 (~55ms on chip), 8x faster…
We're having a big event on agents at CMU on May 2-3 (one week from now), all are welcome! cmu-agent-workshop.github.io It will feature: * Invited talks from @alsuhr @ysu_nlp @xinyun_chen_ @MaartenSap and @chris_j_paxton * Posters of cutting edge research * Seminars and hackathons
On May 2-3, we're going to have a big event in Pittsburgh about LLM Agents. We have invited talks from great speakers inside and outside CMU, student research presentations and posters, tutorials and discussions! Come join us at CMU campus, and register at cmu-agent-workshop.github.io
@shawiz @petergyang @ollama @private_llm Thanks! It’s faster because it’s built on a completely different inference engine: mlc-llm, which in turn is based on @ApacheTVM. The performance advantage comes from the compilation based approach. Each model architecture gets compiled into a highly tuned and optimised library.
WebGPU just got more powerful 🔥 in Chrome 124: ✅ Read-only and read-write storage textures ✅ Service workers and shared workers support ✅ New adapter information attributes ✅ Bug fixes Check out developer.chrome.com/blog/new-in-we… to learn more.
🌟 My fully-local chat with PDF repo doubled in stars this week - now at more than 1.2k! So I revisited WebLLM and was able to add browser-only mode! Here, the entire @LangChainAI ingest + RAG flow runs with the model weights cached in browser storage! webml-demo.vercel.app
More updates from WebLLM coming soon!
🌟 My fully-local chat with PDF repo doubled in stars this week - now at more than 1.2k! So I revisited WebLLM and was able to add browser-only mode! Here, the entire @LangChainAI ingest + RAG flow runs with the model weights cached in browser storage! webml-demo.vercel.app
The first post gives a general overview of the main components and design of DNN64 (including libdragon and MicroTVM @ApacheTVM ): gibsonic.org/blog/2024/03/1…
We are excited to announce the technical program for MLSys 2024! The provisional set of accepted papers is now available on the website at mlsys.org/Conferences/20…. Register for MLSys now at mlsys.org/Register/
.@antmicro introduces Kenning to the @ZephyrIoT ecosystem for a uniform API for various #EdgeAI DNN frameworks & unified AI runtime / model/hardware benchmarking. Learn more: hubs.la/Q02p9SzG0 @TensorFlow @renodeio @ApacheTVM #ZephyrRTOS #opensource #RTOS #blog
Wow that was fast. Google Gemma running in the browser locally - look ma, no server!
webllm.mlc.ai now adds Gemma from @GoogleDeepMind! The 2b model is perfect for building in-browser agents with @WebGPU acceleration -- everything local! Here is a 1x speed demo of 4-bit quantized gemma-2b-it on @GooglePixel_US 7 Pro with @googlechrome.
@GoogleDeepMind @WebGPU @GooglePixel_US @googlechrome Performance wise: 40 tok/sec on M3 Macbook
webllm.mlc.ai now adds Gemma from @GoogleDeepMind! The 2b model is perfect for building in-browser agents with @WebGPU acceleration -- everything local! Here is a 1x speed demo of 4-bit quantized gemma-2b-it on @GooglePixel_US 7 Pro with @googlechrome.
Google's Gemma model is now supported on Android using MLC LLM. Here is a demo of 4-bit quantized Gemma-2b model running on Samsung S23. Thanks to @ruihanglai and many others for bringing Gemma support to MLC! github.com/mlc-ai/mlc-llm llm.mlc.ai/docs/deploy/an…
Gemma 2B runs on phones.
Run Gemma model locally on iPhone - we get blazing fast 20 tok/s for 2B model. This shows amazing potential ahead for Gemma fine-tunes on phones, made possible by the new MLC SLM compilation flow by @junrushao from @OctoAICloud and many other contributors. github.com/mlc-ai/mlc-llm
I'm happy to share the release of gemma.cpp - a lightweight, standalone C++ inference engine for Google's Gemma models: github.com/google/gemma.c… Have to say, it’s one of the best project experiences of my career.
@srush_nlp @haozhangml Cascade Inference is what you need: flashinfer.ai/2024/02/02/cas… APIs are available at: docs.flashinfer.ai/api/python/cas… Looking forward to hearing your feedback :)
CodeLlama 70B is now on MLC LLM -- local deployment everywhere! Thanks to JIT compilation, running on different platforms (even w/ multi-GPU) is made easy -- see how M2 Mac (left) and 2 x RTX4090 (right) have almost the same code. llm.mlc.ai/docs/ huggingface.co/mlc-ai
@monnef @gmonsooniii @ollama Have you tried #mlcllm (uses @ApacheTVM)? ❤️💔 MLC is where I have most luck so far. Cf. blog.mlc.ai/2023/10/19/Sca…
@charlie_ruan @googlechrome @WebGPU @quicksave2k @BrendanEich you may want to check this out! Amazing progress!
Trends for United States
You might like
