Artem Mats @Reederey

Meet Claude Fable 5.1. It is noticeably ahead of both Fable 5 and Opus 5 in benchmarks. Thanks to a fourfold reduction in the price of cached tokens (caching are now $0.25 per million tokens), it is on average 25% cheaper than Fable 5. Moreover, the volume of reasoning tokens appears to have increased. Rollout of the model is already underway. anthropic.com/claude-fable-a…

There is no way that MOE 284B@13B DeepSeek v4 Flash can be better than MOE 320B@18B GLM 5.3 Flash. They are both very aligned with a tool usage (superior RL) but lacking pure knowledge of bigger 2T - 3T models. Scaling low! GLM 5.3 Flash just has more knowledge.

@BrandonMusicKy Thanks a lot for the excellent EXL3 weights!

Huge improvements!!! GLM-5.3-Flash deployment kit for two DGX Sparks — v1.3.2 is out. ⚡ Cached follow-ups skip the queue: a warm 50k prompt behind a running generation, 45.8 s → 2.6 s (on by default) 🚀 New CUDA kernel for the MoE expert weights that eat most of prefill: long reads +9–16% (178k tokens: 895 → 1,040 tok/s), decode unchanged (opt-in). ⏱️ Cold prompts held by the scheduler get first token in ~2.5 s instead of minutes Release: github.com/Reederey87/glm… #vLLM #DGXSpark #GB10 #GLM

Yes — hard aging cap. The warm bypass isn’t a priority queue, just a size check (uncached remainder ≤ one 3584-token page → no hold). Cold prompts are held only while a peer decodes, and after 1.5 s they proceed anyway at 512 tok/step until their last page. So a chatty session can slow a cold read, never starve it. Both knobs are .env: MAX_WAIT_MS, LATE_CAP. Big room for further improvement on GB10.

Best GLM-5.3 flash nvfp4 weights got updated.

LibertAI 🕶️ @Libertai_DAI

2 days ago

We fixed a bunch of bugs and improved quality on our GLM-5.3-Flash-NVFP4 and GLM-5.3-NVFP4 model quants. If you downloaded them previously, redownload the changed parts. huggingface.co/LibertAIDAI/GL…

0 4 48 7K 27

DeepSeek V4 Flash model with Vision got open weights. huggingface.co/deepseek-ai/De…

Update to the GLM-5.3-Flash deployment kit for two DGX Sparks Before vs after the rebase — same 4-agent coding workflow, same two DGX Sparks, GLM-5.3-Flash: The image behind : vLLM's day-0 GLM-5.3 preview base + EXL3 for GB10 + the DFlash2 drafter + the KV slot-share and cache fixes, cut over to production the same day it was field-tested. Release v1.3.0: github.com/Reederey87/glm…

@xz_keg no, it’s the memory-bandwidth wall of this hardware for now.

GLM-5.3-Flash (320B MoE) on 2× NVIDIA DGX Spark desktops, with a full 1M-token context window. Real measured numbers, not vibes: ⚡ ~70 tok/s on structured output, ~30 tok/s on prose 📥 ~940 tok/s prompt ingestion 🎯 100% speculative-decoding acceptance (every draft token lands) 🔁 Cached re-reads of a 110k-token session: 3.5 seconds instead of 2+ minutes Everything is pinned — exact image digest, exact model revision, byte-verified weights — so what you build is what I measured. The repo ships the same acceptance, serving, and benchmark test batteries I gate production with, so you can verify your own numbers in minutes. I got a 320B-parameter frontier model running at home — and turned the whole setup into a kit anyone can reproduce. 🖥️🖥️ Repo: github.com/Reederey87/glm… #DGXSpark #LocalAI #vLLM #GB10

@Belcebuu1 @MiaAI_lab It is a completely rebased deployment image, as all perf parameters are different from her. She is in the credits, as she first used EXL3. I spent quite a lot of time researching the performance of EXL3 on GB10 before pushing different then NVFP4.

GLM-5.3-Flash kit for 2× DGX Spark 1M-token context window, 70.4 tok/s structured / 29.5 tok/s prose decode, ~940 tok/s cold prefill 🔓 Fixed two bugs that broke caching for AI agents. Before, long coding sessions caused the cache to deplete, requiring re-reading entire conversations. Improving session replay to 95-98% and reducing wait times from 5 minutes to about 6 seconds. ⏱️ No more waiting in line behind large prompts. A quick tool call used to wait 4+ minutes for a 240k-token document load. Now it answers in 8 seconds, reducing latency by 97% for a 5% prefill cost. ✅ The kit is now reproduce-tested - release v1.0.2, which passed the full test battery: tools, vision, 1M-token context, and byte-identical memory pool. github.com/Reederey87/glm… #DGXSpark #LocalAI #vLLM #GB10

Fair catch — the denominator is drafted tokens per verify step: 7 drafted, 7 accepted, on one specific benchmark — a count-1-to-200 structured test at temperature 0. Maximally predictable output, so of course the draft nails it. On regular prose the same drafter accepts only ~30%. And you can't serve the draft alone: the big model still verifies every token and silently fixes the misses — 100% just means that on that easy task there were none.

Same model (GLM-5.3-Flash 320B MoE / 18B active), same nodes, same DFlash2 drafter, both on your 2×Spark cluster: The single most important fact from the research: GB10 (SM121) is missing the `cvt.rn.satfinite.e2m1x2.f32` PTX instruction — the hardware FP4↔FP32 converter that SM120 (RTX 5090, B200) and SM100 have. Consequences, all confirmed across multiple independent sources (Avarok's GB10 bring-up writeup, the "NVFP4 is a trap on GB10" benchmark, NVIDIA forums, vLLM issues) So don't expect NVFP4 will be better than EXL3 on GB10/DGX Sparks.

For GLM-5.3 Fash 2xDGX 's use case — single-stream, long-context, agentic, cache-heavy — EXL3 is measurably the better UX: quicker felt decode speed over NVFP4 baseline, better quality per bit, 1M window, replays in seconds. The prefill point is right and worth respecting (it's why cold 133k prompts cost ~140s), but "3× prefill" only dominates the experience if your workload is cold prompts or batch decode.

Published third-party numbers on the same hardware suggest a large single-stream win — but they are reported with MTP and without an acceptance rate, so treat them as a hypothesis, not a target. The model's native window is 1,048,576, but after ~90.5 GiB of weights per node the KV pool is the binding constraint. 131K at 6.2x concurrency is the useful operating point; raise it and re-read the pool size the engine prints at startup.

Kimi K3 is an awesome model for CUDA kernel work. As far as I know, some frontier labs are running it on research-allocated compute. The main downside is that Kimi's subscriptions are pretty limited. You only start getting a semi useful number of tokens for real work at the +$100 tier.

Awesome!!

Zixuan Li @ZixuanLi_

6 days ago

We’ve seen many comments and requests from our partners regarding framework compatibility. To ensure everything is properly supported and works reliably at launch, we’ve slightly adjusted the open-weight release timeline. Thank you all for your patience and support.

27 29 712 87K 41

@Tech2Wild Getting following... Measured on the pair of DGX Sparks: KV pool 813K–834K tokens, 6.2x concurrency at a full 131K ctx request, TRUE 13.6 tok/s single stream, 26.7 tok/s across 8. github.com/Reederey87/glm…

Big win for Deepseek V4 Flash on DGX Sparks/SM121 in new version of vLLM. #vLLM github.com/vllm-project/v…

vLLM @vllm_project

6 days ago

vLLM v0.28.0 is out! 584 commits from 270 contributors (76 new). 🎉 Highlights: 🌙 A stack-wide optimization push for Kimi-K3 🐳 DeepSeek-V4 sparse MLA now works end to end for plain decode, MTP and DSpark ⚡ Speculative decoding adds DFlash2 and DSpark confidence-scheduled

21 34 334 22K 56