-
Tweets346
-
Followers2K
-
Following473
-
Likes9K
check out the server here: github.com/abetlen/llama-… you can run it as a uv script with the pre-compiled wheels: git clone github.com/abetlen/llama-… cd llama-cpp-python/examples/server uv run \ --index abetlen.github.io/llama-cpp-pyth…{cpu,metal,cu125,vulkan,rocm,etc} \ --script server.py -C configs/gemma-4-12b-it-qat.json ps: i want to thank @julien_c and @reach_vb for supporting the project and open source in general from huggingface and openai
shoutout to Aman for all the amazing work adding multi-token prediction support to llama.cpp still experimental which is why this isn't in the main python api yet but should be soon x.com/osanseviero/st…
Gemma 4 MTP just got officially merged into llama.cpp This means you can use Gemma 4 QAT + MTP for a lightweight + super fast setup. Excited to see what the community builds with it github.com/ggml-org/llama…
new openai api compatible webserver for llama-cpp-python - /v1/responses api (w codex cli support) - continuous batching - structured response parsing (auto tools, reasoning) - multimodal support (images, audio) video: gemma 4 12b running in codex on amd ai max desktop
Gemma 4 MTP just got officially merged into llama.cpp This means you can use Gemma 4 QAT + MTP for a lightweight + super fast setup. Excited to see what the community builds with it github.com/ggml-org/llama…
Colab Link: colab.research.google.com/github/abetlen…
btw you can run Gemma 4 QAT using llama.cpp in a T4 Google Colab notebook (link in reply)
Gemma 4 quantization-aware training (QAT) models are now available, bringing AI performance directly to edge devices and consumer GPUs. These checkpoints are optimized with quantization-aware training to dramatically reduce memory requirements and unlock high-speed local
llama.cpp adds MTP for the Qwen3.6 family This is a significant milestone for the local AI ecosystem. The performance jump with these changes is massive and elevates local inference on commodity hardware further. Special thanks to Aman Gupta for leading this development! github.com/ggml-org/llama…
@vikhyatk they're calling him the jack kerouac of vlms
llama.cpp at 100k stars now that 90% of the code worldwide is being written by AI agents, I predict that within 3-6 months, 90% of all AI agents will be running locally with llama.cpp 😄 Jokes aside, I am going to use this small milestone as an opportunity to reflect a bit on the project and the state of AI from the perspective of local applications. There is a lot to say and discuss and yet it feels less and less important to try to make a point. Opinions about viability of local LLMs are strongly polarized, details are overlooked, the scientific approach is lacking. Arguments are predominantly based on vibes and hype waves. One thing is clear though - local LLMs are used more and more. I expect this trend to continue and likely 2026 will end up being one of the most important years for the local AI movement. I admit that I didn't expect the agentic era to come so quickly to the local LLM space. One year ago, the available models were too computationally expensive for doing long-context tasks. There wasn't an obvious path towards meaningful agentic applications. The memory and compute requirements were huge. Last summer, with the release of gpt-oss, things started to change. It was the first time we saw a glimpse of tool calling that actually works well within the resource constraints of our daily devices. Later in the year, even better models were released and by now, useful local agentic workflows are a reality. Comparing local vs hosted capabilities at a given moment of time is pointless. To try put things into perspective: - We don't need frontier intelligence to automate searches and sending emails - We don't need trillion parameter models to be able to summarize articles or technical documents - We don't need massive GPU data centers to control our home appliances or turn the lights off in the garage I believe that there is a certain level of intelligence we as humans can comprehend and meaningfully utilize to improve our working process. Beyond that level, access to more intelligence becomes unnecessary at best and counterproductive at worst. I also believe that that level of useful artificial intelligence is completely within reach locally and it has always been just a matter of implementing the right software stack to bring it to the end user. With llama.cpp, I am confident that we continue to be on the right track of building that software stack! The llama.cpp project is going stronger than ever. With more than 1500 contributors, the project keeps growing steadily. From technical point of view, I think that llama.cpp + ggml is the only solution that actually makes sense. That is, the software stack must run efficiently on every possible device, hardware and operating system. The technology is too important to be vendor-locked. It has to be developed in the open, by the community, together with the independent hardware vendors. This is the only right way to build something that will truly make a difference in the long run. I won't try to convince you about what is currently and will be possible with local AI. We will just continue to build as usual. I am confident that after the smoke clears and we look objectively at what we have built together, the benefits will be obvious to everyone. Big shoutout to all llama.cpp maintainers. I feel extremely lucky to be able to work together with so many talented contributors. Every day I learn something new and I feel there is so much more cool stuff that we are going to build. Also, I am really thankful that the project continues to have reliable partners to support it! Cheers!
🚨 We're open-sourcing Druids, a library for coordinating and deploying coding agents across machines. Our beta users have used Druids to work on open math problems, conduct ML "autoresearch," and make software faster.
This past July, @AlexKontorovich and Terry Tao announced progress of a “medium” Prime Number Theorem proof, which was a very exciting result. However, it faced technical difficulties in complex analysis related to the Riemann zeta function, which has been notoriously tricky for Lean. Today I’m very glad to announce a ~25K line Lean proof of a “strong” Prime Number Theorem, by establishing the classic zero-free region for the Riemann zeta function. Github: github.com/math-inc/stron…
Woohoo!! The "Medium" strength Prime Number Theorem was just proved in @leanprover Lean: (the bottom node in the picture is Green) The main `MediumPNT` file is about 8000 lines of code, which uses a big `ZetaBounds` file with around 4000 lines of code, and another ~1000 lines
Gauss is entering beta testing with select mathematicians. 🔗 Blog: math.inc/gauss 🔗 GitHub: github.com/math-inc/stron… 🔗 Early access: math.inc/early-access
Today we're announcing Gauss, our first autoformalization agent that just completed Terry Tao & Alex Kontorovich's Strong Prime Number Theorem project in 3 weeks—an effort that took human experts 18+ months of partial progress.
really happy to announce i'm in sf now fulltime on an o1 visa! thank you @jessemhan @morph_labs and @lisawehden for making this happen also, a big thank you to @natfriedman @PhilCulliton and @julien_c
Incredibly proud of this work by the @morph_labs team!
A mathematical paper autoformalized for the first time: amazing work by @morph_labs, presented today at the Big Proof conference by @jdlichtman and @jessemhan. I am very impressed by the blazing fast progress of the morph team. Especially by @LeyanPan and @critic_model.
yoooooo szegedy to morph??? insane trade deal wow huge
new lab launching in 1 month at @aiDotEngineer keep a very special watch on @willccbb's track :)
sλrthak @sarthak2143
2K Followers 2K Following 20. systems & compilers. manual cars enthusiast. fafoing w my life.
Jay Alven @JayAlven13
7 Followers 574 Following
Nikita Belokopytov @NikiBelokopytov
61 Followers 88 Following Sometimes engineering, sometimes business. Always slav. Building products since 2010.
joey00072 @joey00072fp4
168 Followers 531 Following new acc @shxf0072 has been compromised. Do not reply to any DMs on that account.
Anatoly @t2v12
101 Followers 6K Following
feynon @feynon_
395 Followers 3K Following technologist @tilesprivacy community https://t.co/OLNOxCQEmf
Wing Lian (caseus) @winglian
11K Followers 2K Following @axolotl_ai OSS maintainer. Axolotl AI founder. AI/ML tinkerer. Building tools for everyone.
Andrew @_andrewtjames
14 Followers 546 Following
ZKIWU @zkiwu
2 Followers 2K Following
Daniel van Strien @vanstriendaniel
6K Followers 2K Following Machine Learning Librarian @huggingface 🤗 I like datasets.
Mathew Schroeder (bsi... @bsides230
0 Followers 11 Following Chief Simulation Architect @ LYRN Systems Open Source Automation Loop Dashboard: https://t.co/sIEJ9zKNun
k @IceY1605
1 Followers 148 Following
Acer @AcerFur
7K Followers 2K Following Furry pure maths student @Cambridge_Uni. | 🇬🇧🇵🇹 21 He/Him | Incoming @OpenAI. Opinions my own.
Moritz Groß @Moritz__Gross
342 Followers 6K Following Algorithms, AI, Open-Source 🦀🐍 | | currently doing Masters
Fabio Grandi @aigrandi
7 Followers 425 Following
Cruthaifios @cruthaifios
35 Followers 280 Following Aiming to form an ecosystem of engineers who craft AI in a distributed network, training their own model weights. Building Clankadex https://t.co/N0c7N2vfWb
Swair Bot @SwairBot
0 Followers 791 Following
James Adebayo @Biggi1331
49 Followers 903 Following
Behnam @OrganicGPT
943 Followers 834 Following Sharing tips and lessons about AI inference and research. PhD @carnegiemellon
Radu @RaDu7788
138 Followers 1K Following
Leonie @Ouboupbeab1875
152 Followers 7K Following A woman with a voice is, by definition, a strong woman.
hi-peixin hi @pexin2025
1 Followers 152 Following
John Schulman @johnschulman2
75K Followers 2K Following Recently started @thinkymachines. Interested in reinforcement learning, alignment, birds, jazz music
dean cureton @deancureton
367 Followers 238 Following 21 ∙ math/cs @stanford, math engineer @mathematics_inc ∙ prev tech @hackwithtrees
Chris Yunker @cyunker
228 Followers 624 Following Software developer in St. Louis @[email protected]
nvpy @CuTeDSL
0 Followers 39 Following
Latent Kiri @edgeaiguy
1K Followers 3K Following Crafting AI solutions for tiny embedded devices. Thinking in latent space. Preparing for AGI world ! #AIAgents #EdgeAI #TinyML #LLMs #EmbeddedSystems #Robotics
Abe @AbeKazemzadeh
134 Followers 826 Following I use twitter for fun and work at times. For fun I like to keep up with news friends and marine weather.
Alcoft/I4.0 AI @TAO71AI
66 Followers 188 Following
Estelle @vlwQ0X9n09h781t
47 Followers 1K Following The best protection any woman can have is courage.
lessretardedeveryday @helpmeimretarde
65 Followers 4K Following 🇮🇱 🛠️ Superpositioned Communist 👽 🏳️🌈 ---- Hillo 👋 I repost everything without reading so pls don't be confused ❤️ 😘 💋
Lars Johansson @masao16w
100 Followers 1K Following engineer @ByteroverDev - building a central memory layer for modern dev team | 2.8k stars on Github
Joanne @jungyoonlim
159 Followers 1K Following
sshkhr @sshkhr16
2K Followers 2K Following research eng @GoogleDeepMind prev: founder @DiceHealth, researcher @AIatMeta @VectorInst
Alex Imas @alexolegimas
32K Followers 2K Following Director of AGI Economics @GoogleDeepMind. Professor at @ChicagoBooth. (on leave) Essays: https://t.co/9qSiQxvdja Opinions are my own.
Reese Levine @reeselevine
285 Followers 253 Following Recreating on public lands. Thinking about computers at UCSC.
Timothy Gowers @wtgow... @wtgowers
57K Followers 187 Following Mathematician. Professeur titulaire de la chaire Combinatoire au Collège de France. Also fellow of Trinity College Cambridge.
leloy! @leloykun
7K Followers 5K Following Math @ AdMU • NanoGPT speedrunner • Muon fan 🤍 • prev ML @ XPD • 2x IOI & 2x ICPC • https://t.co/nfO038itfn
Thariq @trq212
274K Followers 2K Following Claude Code @anthropicai. prev YC W20, @southpkcommons, @medialab
Mitchell Hashimoto @mitchellh
203K Followers 147 Following Creator of Ghostty. 👻 Prev founded @HashiCorp, created Vagrant, Terraform, Vault, and others.
Jerry Tworek @MillionInt
37K Followers 1K Following CEO and co-founder of Core Automation former VP of RL @ OpenAI : reasoning models, o3, o1, GPT4, ChatGPT, Codex, RL for robots cautious AI optimist
Daniel van Strien @vanstriendaniel
6K Followers 2K Following Machine Learning Librarian @huggingface 🤗 I like datasets.
Demis Hassabis @demishassabis
1.1M Followers 172 Following Nobel Laureate. Co-Founder & CEO @GoogleDeepMind - working on AGI. Solving disease @IsomorphicLabs. Trying to understand the fundamental nature of reality.
Jonathan Gorard @getjonwithit
46K Followers 18 Following Applied mathematician, computational physicist @Princeton Previously @Cambridge_Uni Making the universe computable.
Fulcrum @fulcrum_inc
507 Followers 3 Following scaling human intent in software | https://t.co/NM8UThTlW7
Stanislas Polu @spolu
24K Followers 640 Following co-founder+engineer(https://t.co/SXBR0l9TrF); alumni(https://t.co/z6zJ8xaKGI, https://t.co/CvVTA1CHAo, https://t.co/WOVEe2aLcK, https://t.co/ui9I4Nj7o1);
You Jiacheng @YouJiacheng
12K Followers 2K Following
Yu Zhang 🐙🌘 @yzhang_cs
4K Followers 873 Following @Kimi_Moonshot; working on efficient methods for LLMs; disciple of parallel programming; INTP; 🐈
Acer @AcerFur
7K Followers 2K Following Furry pure maths student @Cambridge_Uni. | 🇬🇧🇵🇹 21 He/Him | Incoming @OpenAI. Opinions my own.
Akshay @akshayvegesna
431 Followers 170 Following Working on generalization at Q Labs. https://t.co/ExPhN2Kb4X Previously perception @nuro, math @caltech
Lawrence Chen @lawrencecchen
5K Followers 850 Following building open source coding tools https://t.co/dToYirHstr
dominik kundel @dkundel
20K Followers 2K Following @OpenAI DevX, Codex, gpt-oss, TS Agents SDK - he/him - Opinions my own
Toby Pohlen @TobyPhln
144K Followers 610 Following Sleeping. Previously founding team @xAI, engineer @GoogleDeepMind. @RWTH alumnus.
dean cureton @deancureton
367 Followers 238 Following 21 ∙ math/cs @stanford, math engineer @mathematics_inc ∙ prev tech @hackwithtrees
Daniel Litt @littmath
58K Followers 916 Following Assistant professor (of mathematics) at the University of Toronto. "Tireless math ronin." Algebraic geometry, number theory, etc. He/him.
gabriel @gabriel1
97K Followers 579 Following new thing, previously research at @OpenAI & @midjourney
Ivan Velichko @iximiuz
74K Followers 563 Following Software Engineer. Educator. Entrepreneur. Bootstrapping https://t.co/9b6sZ2UVQj - a learning-by-doing platform to master Linux, Containers, and Kubernetes 🚀
Andy Pavlo (@andypavl... @andy_pavlo
40K Followers 206 Following Associate Professor of Databases @CarnegieMellon.
John Schulman @johnschulman2
75K Followers 2K Following Recently started @thinkymachines. Interested in reinforcement learning, alignment, birds, jazz music
Marc Brooker @MarcJBrooker
25K Followers 748 Following Distinguished engineer at AWS. AI, agents, databases, and serverless. Views are my own.
LaurieWired @lauriewired
155K Followers 292 Following researcher @google; serial complexity unpacker; https://t.co/Vl1seeNgYK ex @ msft & aerospace
Alex Kontorovich @AlexKontorovich
33K Followers 835 Following Mathematician (Distinguished Professor of #Math at @RutgersU). Here to learn about research, education, and community. Let’s build something together.
Patrick Shafto @patrickshafto
3K Followers 1K Following PM @DARPA; Prof of Math and CS @Rutgers-Newark; co-founder @ https://t.co/e6dJA2bLus; Math @the_IAS 2021-2023. https://t.co/2plDQE0s6K https://t.co/XuiVK8VmO3
Auguste Poiroux @augpoi
165 Followers 158 Following AI4Math & Autoformalization - PhD Student @EPFL - Founding Research Engineer @mathematics_inc
Math, Inc. @mathematics_inc
13K Followers 0 Following Solve math, solve everything. Dedicated to superintelligence via autoformalization
Rajab @RajabRehanDev
408 Followers 990 Following dreamer ⁂ | tmu lead @GoogleDevs | prev. data @Cohere | hackathons W @Stanford @UCBerkeley | HMU, let’s be friends! :)

































