Bờm @quatmo

@MejiasDev If you use Android l, Haven SSH is much better than Collie, Moshi can be use in both iOS and Android.

@JK99928789839 @0xWhiteMage @deepseek_ai @ArtificialAnlys You can just side load a cheap online model for vision.

DeepSeek v4 Flash Vision Exp for 2x DGX Sparks Abliterated (Uncensored) is here 🔥 - off by default, same runtime - same weights, no need to re-download - simple on/off flag in .env Also fixed some bugs along the way. Thanks to @u1tra_instinct for the ablit files! Integrated for this Vision-Exp release. Get it here: github.com/MiaAI-Lab/Deep…

@ali7ep @MiaAI_lab @u1tra_instinct Same issue

@MiaAI_lab Wow, it's much better than 0731

BREAKING: MiniMax H3 Max sets the new Pareto Frontier for video generation, nearly 50x faster than the base model. This model is post-trained by @fal on @MiniMax_AI H3, and it's in a league of its own: no other Image to Video model on the arena delivers higher preference at a lower generation time. Its Image-to-Video generation time is just 6.4 seconds, 18x faster than average, and its Text-to-Video generation time is just 4.7 seconds, 24x faster than the average. Huge congratulations to the @fal team on this release!

@MiaAI_lab @zhijianliu_ You see this yet?

0xBakeer @0xBakeer

7 days ago

Qwen3.8-Flash-Next on a single DGX Spark: up to 97 tok/s. 180B params in 128GB with the full 262k context, because 51B of them are an n-gram table that can sit on NVMe instead of in RAM. The 97 is editing a file you pasted in, where ngram speculation accepts 60-token spans.

34 38 340 47K 315

Big day in LLMs: - Z.ai unmasks Ox Alpha as GLM-5.3-Flash — 320B/18B, trained on Chinese chips, near Opus 4.8 on coding. - Alibaba drops Qwen3.8-Flash-Next. - Apple doubles down on local AI with new Mac mini/Studio. - IBM ships Granite 4.2.

@MiaAI_lab Can't wait for your recipe!

@PeasantSmith @Tech2Wild Ask hermes directly pls.

Ox Alpha啊，你裤子掉了。

Actually gemini-3.7-flash is nice: ⚡ Summary Metrics - Visible Stream Throughput: ~408.7 tokens/sec (Peak: 548.1 tokens/sec) - Internal Thinking / Reasoning Speed: ~176.9 tokens/sec - Effective End-to-End Throughput: ~249.9 tokens/sec (Reasoning + Output tokens combined)

Just pushed DFlash2 (@inco_ai) recipes to the Qwen3.8 27B cookbook⚡️ docs.sglang.io/cookbook/autor… The community has been seeing great results with NVFP4 + DFlash2, and these recipes should be some very good starting points to play with. More Qwen3.8 27B updates on the way 🫡

SGLang @sgl_project

2 weeks ago

Congrats on launching DFlash2! Recipes for DFlash2 on Qwen3.8-27B and Muse Glimmer are on the way. We got some very promising initial results⚡️

17 34 429 58K 192

@seanphan @jun_song 4600 also, crazy

DFlash 2 is here! Qwen3.8-27B at 70 tok/s on an M5 Max MacBook Pro. ⚡ Up to 4.6× the speed of autoregressive decoding, with the same output. This is the next generation of DFlash, seeded at Z Lab and upgraded at Inco AI. Get one more accepted token on every pass, for free! inco.ai/blog/dflash2/

Tired of editing config files every time you swap a local model? The broker is a dependency-free, localhost OpenAI-compatible proxy that exposes one stable model ID (local-auto) while routing requests to your prioritized local model servers. github.com/binhdnguyen/lo…

@MiaAI_lab github.com/binhdnguyen/lo… small tool

Qwen3.8-27B, one DGX Spark: 7.88 -> 75 tok/s solo, 256 aggregate. Spent the weekend trying to make Qwen3.8-27B faster on a single DGX Spark and ended up proving myself wrong about almost everything I assumed going in. Started at 7.88 tok/s. Stock FP8, nothing tuned. My assumption was the same one everybody has: if you want speed, quantize. 4-bit, fewer bytes per token, done. Wrong. Or at least, it was the smallest part of the answer. Speculative decoding got me to 58.5 tok/s on the exact same FP8 weights. That's 7.4x with nothing touched, and it cannot change the output at all, because the full model verifies every drafted token and throws away anything it wouldn't have written itself. Free speed. I had been ignoring it. Then a few things I did not expect. The drafter architecture mattered more than the draft depth. DSpark is a separate 1B drafter that emits a block of 7 tokens at once. MTP re-runs one in-checkpoint layer 7 times and pays a full vocab projection every time. DSpark accepts FEWER tokens per pass and is still 46% faster. Cost per draft token: 0.046 vs 0.153. Cheap passes beat good guesses. Acceptance rate is a trap. Going from k=7 to k=14 dropped acceptance from 98.7% to 68.7% and made generation 27% faster. The late draft positions are basically lottery tickets, the early ones are certain. Mean tokens per forward pass is the number that predicts throughput. Not acceptance. Prefix caching had been silently off the whole time. vLLM disables it by default for hybrid attention models and Qwen3.8-27B reports is_hybrid=True. Nothing in the startup log mentions it. Claude helped me and only found it reading engine source. Turning it on: 14x faster prefill on a 19K shared prefix, 22x on a 53K one. And then the result I actually care about. 4-bit is 27% faster than FP8 at one request. At 4 requests, 20%. At 8, 10%. At 16 concurrent requests the two are within 0.2% of each other. Basically identical. It makes sense once you see it. Single-stream decode is memory bandwidth bound, so halving the bytes nearly halves the work. Batch it, and one weight read serves many sequences at once. The bottleneck moves to compute and byte count stops mattering. So if you're serving agents in parallel, quantization buys you close to nothing. It also costs you 7-9% of prefill at long context, because 4-bit weights get unpacked before every matmul on hardware with no native FP4 path. Final numbers: 7.88 -> 75 tok/s single stream, 256 tok/s aggregate across 16 streams. Run your own task before you trust any 4-bit build. Both serving recipes are public: FP8: github.com/0xBakeer/Qwen3… NVFP4: github.com/0xBakeer/Qwen3…

0xBakeer @0xBakeer

2 weeks ago

Qwen3.8-27B, one DGX Spark: 7.88 -> 75 tok/s solo, 256 aggregate. x.com/i/article/2089…

10 27 246 126K 427

@ivanfioravanti @MiaAI_lab I have the thermal throttling issue with Mia's recipe on the new Deepseek 0731, since the new max thinking always push the temp to 86 degC in TTFT, with 16 concurent. Apply clock cap might increase TTFT abit but not much on tok/s and no thermal throttling. I use 2300 MHz.