λndres Mariscal @SerialDev

Wrote anti-cheat ml, do ML/AI at places you know off and probably use && into graphics||compilers||DBs I like tech, sloths and, chihuahuas. Helsinki Joined July 2015

Tweets

3K
Followers

336
Following

3K
Likes

14K

Daniel Lemire @lemire

4 days ago

Nvidia is proposing a beast of a CPU system for Windows PCs. It has 128 GB of shared memory and comes with up to 6,144 state-of-the-art CUDA cores. CPU wise, the chip has 10 performance cores and 10 efficiency cores. The performance cores are based on the Cortex-X925. These chips appear to support six 128-bit SIMD execution units (SVE2), not as good as recent AMD chips, but better than Apple Silicon (on paper). The game changer is the unified 128 GB memory. That is the path Apple took years ago. Instead of separate memory for the CPU and GPU, everything shares a single pool. It is increasingly popular. The memory is not as fast as dedicated GPU memory, but it is cheap enough while delivering enough bandwidth to run AI models locally. I am not sure how many people will run AI models locally. It still seems like a niche application to me. However, it will make decent machines to play video games. It will be interesting to see how Intel and AMD respond. I think that the AVX-512 instructions supported by all recent AMD processors are far superior to the SVE2 instructions of the Cortex-X925. They can eat more data and they are more versatile. But Intel has been shy, thus far, in making it available on customer systems.

37 22 190 22K 60

View Details

λndres Mariscal @SerialDev

a week ago

@Jonathan_Blow Arm is involved on the tweet sequence too, so I would expect unified memory and built-in tpu or something AI-centric

1 0 0 1K 0

View Details

λndres Mariscal @SerialDev

2 weeks ago

@stake_jevens @luminal_ai Location agnostic?

0 0 0 115 0

View Details

λndres Mariscal @SerialDev

3 weeks ago

@lucasmeijer Token attention capacity is still limited, increased context window is not truly increased imo, just look at how its implemented, Grouped-Query Attention ;; RoPE Scaling ;;YARN / CoPhy, theoretically do compress information so it is in essence lossy. less is more

0 0 0 78 0

View Details

λndres Mariscal @SerialDev

4 weeks ago

@rfleury AI has been the first thing that truly made me understand that there is something so much deeper in our creations besides the outcome. I 100% agree even with a similar result there is something unspoken when you can relate to another person's passion. Even in code

0 0 2 272 0

View Details

λndres Mariscal @SerialDev

a month ago

@lucasmeijer @badlogicgames Godbolt is a regular tab for so many of us lol

0 0 0 26 0

View Details

nathan chen @nathancgy4

2 months ago

one of the very first things i worked on after joining kimi was speeding up KDA's kernels with @yzhang_cs and @uniartisan (i got carried :D). it was super fun optimizing those triton kernels... and now comes FlashKDA, a highly efficient KDA in CUTLASS for the open community! side note: knowing how to write a kernel matters less and less, but knowing how it actually works efficiently matters as much as ever. although I rarely write kernels anymore, and instead mostly use kimi k2.6 / opus 4.5-7 to write them—far from optimized ones, simply for the sake of testing for signs of life—for me, those days of trying to make algorithms as hardware-aligned as possible turned out to be special and shaped many intuition for architectural designs that followed. (arch and infra are really two sides of the same coin). would highly recommend reading basic flash/linear attention's triton kernels in FLA (github.com/fla-org/flash-…) for anyone wanting to better understand how efficient kernels work btw

Kimi.ai @Kimi_Moonshot

2 months ago

We're open-sourcing FlashKDA — our high-performance CUTLASS-based implementation of Kimi Delta Attention kernels. Achieves 1.72×–2.22× prefill speedup over the flash-linear-attention baseline on H20, and works as a drop-in backend for flash-linear-attention. Explore on github:

45 183 2K 214K 618

8 27 376 49K 181

View Details

λndres Mariscal @SerialDev

2 months ago

100% agree a win for Zig, but disagree with the out-of-distribution generalisation, Zig is fine-tuned for performance "writing" and as a result you a over-represented high quality performance focused corpus (Bun, TigerBeetle, etc. and all of the Mike Acton inspired talks!)

Mitchell Hashimoto @mitchellh

2 months ago

Huge W for Zig, used for inference for K2.6. If you want absolute performance with exacting control over what your CPU executes and the way memory is laid out, Zig is the way.

27 59 1K 178K 321

0 0 2 128 0

View Details

λndres Mariscal @SerialDev

2 months ago

@anselmlevskaya SPIR-V and shader langs are underrated too, my time in the gaming industry is wildly underappreciated in ML circles still.

0 0 1 273 0

2 months ago

🙃

0 0 0 35 0

View Details

Alex Zhurkevich @cudagdb

2 months ago

Trtllmgen kernels are now open. Fastest prefill and decode kernels for our target workloads. We wrote these to win InferenceX, MLPerf, other benchmarks. Powering some of today’s top served models. Dive in, learn, use them, or level up your own. Enjoy. github.com/flashinfer-ai/…

13 51 334 148K 260

View Details

Jeremie Pelletier @HostOfMeta

2 months ago

@ThePrimeagen Strudel REPL but for gamedev; that was proof of concept cljs; going for the moon no re-star'. x.com/HostOfMeta/sta…

Jeremie Pelletier @HostOfMeta

2 years ago

Need more tools to bind all these feature together:

3 0 41 9K 7

0 1 3 151 1

View Details

Jeet Desai @Jeet2505

2 months ago

Amazing! 🙌@TigerBeetleDB youtube.com/watch?v=y2_Bqk…

0 3 10 2K 2

View Details

TigerBeetle @TigerBeetleDB

2 months ago

IronBeetle⚡️ Ep 105 Zig's comptime is A W E S O M E for CLI argument parsing youtu.be/BDGkD3jtWpM

1 3 25 2K 10

View Details

John Carmack @ID_AA_Carmack

2 months ago

Without getting all the way down to performance counters, GPU power from nvidia-smi is a better indicator of true utilization than job scheduling or “gpu busy”. I would love to see animated “heat maps” of the big data centers, with each pixel being an individual GPU’s power draw. I am confident that inference and frontier training at the big labs is highly efficient, but I wonder how many GPUs would be dark due to scheduling and inefficient research code. With a little calibration for base load and peak, just the power bill for the datacenter would be a pretty good first order indicator of utilization.

72 62 1K 181K 177

View Details

gengstah @_gengstah

2 months ago

Released WinDbg MCP — attach Claude (or any LLM) to a live Windows process and let it poke around. set breakpoints, read memory, walk the stack, load crash dumps. 55 tools over MCP. github.com/gengstah/windb…