inference is all you need
wrote a detailed worklog of optimizing Softmax kernel in CUDA.
in this piece:
> start with a naive implementation
> go on to identify and quantify the performance issues. > explore the concept of online softmax
> implement the kernel quantify the gains in terms of memory & FLOPs
> apply the coalesced memory pattern
> again quantify the gains in FLOPs, Comms, and Memory
> write the CUDA code.
read it here:
heyyanshuman.com/posts/making_s…
inference is future.
tried going as much in detail as possible for the writing GEMV kernel in CUDA,
hopefully it will be a good read. ciao!
heyyanshuman.com/posts/making_g…
78 Followers 2K Followingbuilding @Harper_Insure | @ucberkeley grad | future farmer 👨🏾🌾 | views are mine | posting into the void until the void starts communicating back 🪐
22K Followers 1K FollowingThe sea is the sea
The old man is an old man
The boy is a boy and the fish is a fish
The sharks are all sharks no better and no worse
52K Followers 1K FollowingCTO @Databricks and prof @UCBerkeley. Working on data + AI, @ApacheSpark, @DeltaLakeOSS, @MLflow, @DSPyOSS, @GEPA_ai, @Omnigent_ai.
13K Followers 348 FollowingNeural Breakdown on YT | Read research with AI: https://t.co/Ef6m4nUpcZ | maker of videos, writer of articles, learner of things
51K Followers 4 FollowingBig Tech and startups, from the inside. The #1 technology newsletter on Substack. Sign up at https://t.co/MPNdQSVnwV. Podcast: https://t.co/nVOulBGYoh
12K Followers 2 FollowingFindMyRoom Bangalore helps you get notified on the latest listings about vacant occupancies in Bangalore.
Use @fmrbangalore to find a flatmate now! 🤝
53 Followers 35 FollowingWe reject 90% of listings so you don’t waste time. Handpicked spaces, total clarity, and complete deal handling till the ink is dry ✍️
13K Followers 4K FollowingCo-CEO @poolsideai // President @ PIC // Board @axon_enterprise
“The best way to predict the future is to invent it.” - Alan Kay
960 Followers 486 FollowingPre-training Nemotron @nvidia | Prev Pre-training Lead @poolsideai, @AssemblyAI, @instadeepai, @ucl_dark, @Uni_Stuttgart
Opinions are my own
1K Followers 26 FollowingPost-training @anyscalecompute. Prev - @UCSanDiego, @C3_AI, @iitmadras. Machine Learning and Systems. Intensity is all you need.
21K Followers 640 FollowingFounder @RicursiveAI, Asst. Prof. of CS at Stanford. Prev: DeepMind, Anthropic, Brain. Co-Creator of MoEs, AlphaChip, Test Time Scaling.