Daniel @GradientLane

@ABadaramoni @karpathy does this sound interesting to you?

I'm Boris and I created Claude Code. Lots of people have asked how I use Claude Code, so I wanted to show off my setup a bit. My setup might be surprisingly vanilla! Claude Code works great out of the box, so I personally don't customize it much. There is no one correct way to use Claude Code: we intentionally build it in a way that you can use it, customize it, and hack it however you like. Each person on the Claude Code team uses it very differently. So, here goes.

I've never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last ~year and a failure to claim the boost feels decidedly like skill issue. There's a new programmable layer of abstraction to master (in addition to the usual layers below) involving agents, subagents, their prompts, contexts, memory, modes, permissions, tools, plugins, skills, hooks, MCP, LSP, slash commands, workflows, IDE integrations, and a need to build an all-encompassing mental model for strengths and pitfalls of fundamentally stochastic, fallible, unintelligible and changing entities suddenly intermingled with what used to be good old fashioned engineering. Clearly some powerful alien tool was handed around except it comes with no manual and everyone has to figure out how to hold it and operate it, while the resulting magnitude 9 earthquake is rocking the profession. Roll up your sleeves to not fall behind.

Jensen Huang told a Cambridge Union "Intelligence is about to be a commodity" For a long time, school and hiring treated “smart” as scarce, because fast recall and clean problem solving were hard to scale beyond the best individuals. AI has flipped that. Jensen explained “When AI takes over all standardized work, the only value humans have left is to handle the poorly defined work.” “The poorly defined work is the most valuable of all work.” - Defined Work (AI Territory) - Poorly Defined Work (Human Territory)

@stats_feed Glasses

Bob Ross vibe coding was the AI slop I never knew I needed in my life

The progressing from the baseline has to be a guardrail of setting goals. Otherwise setting your goal to not change status quo will result in drifting below baseline.

existence is structureless in the absence of selection

Most people's lives are determined by how they choose to cure their boredom.

Griffin architecture, released recently from @GoogleDeepMind, outperforms transformers. 🔥 📌 Hawk (one of the new architecture from this paper) exceeds the reported performance of Mamba on downstream tasks, while Griffin matches the performance of Llama-2 despite being trained on over 6 times fewer tokens. 📌 Griffin can be extrapolated on sequences significantly longer than those seen during training. 📌 Across multiple sizes, Griffin out performs the benchmark scores of transformers baseline in controlled tests in both the MMLU score across different parameter sizes as well as the average score of many benchmarks. The architecture also offers efficiency advantages with faster inference and lower memory usage when inferencing long contexts. 2B version of this on huggingface : 📌 Achieving this with only 300B tokens. The main problem with Transformers is that they are difficult to scale efficiently to long sequences due to the quadratic complexity of global attention. Additionally, the linear growth of the Key-Value (KV) cache with the sequence length makes Transformers slow during inference. Although Multi-Query Attention (MQA) partially mitigates this issue by reducing the cache size by a constant factor, the cache still grows linearly in sequence length. 📌 Recurrent language models present a compelling alternative as they compress the entire sequence into a fixed-sized hidden state which is updated iteratively. However to replace Transformers, new RNN models must demonstrate not only comparable performance at scale but also achieve similar hardware efficiency. 📌 In this paper, they propose the RG-LRU layer, a novel gated linear recurrent layer, around which they design a new recurrent block to replace MQA. They build two new models using this recurrent block: Hawk, a model which interleaves MLPs with recurrent blocks, and Griffin, a hybrid model which interleaves MLPs with a mixture of recurrent blocks and local attention. 📌 The paper shows that: Hawk and Griffin exhibit power law scaling between held-out loss and training FLOPs, up to and beyond 7B parameters (Figure 1(a)), as previously observed for Transformers. Griffin achieves slightly lower held-out loss than strong Transformer baselines at all model scales. 📌 For Griffin, the key advantage of recurrent blocks over global attention is that they use a fixed state size to summarize the sequence, whereas the size of MQA’s KV cache grows proportional to sequence length. 📌 Since local attention has the same property, mixing recurrent blocks with local attention pre-serves this benefit. They have found this combination extremely effective, since local attention accurately models the recent past, while the recurrent layers can transmit information across long sequences. 📌 Griffin uses the same residual pattern and MLP block as our Transformer baseline. However unlike both our MQA Transformer baseline and the Hawk model, Griffin uses a mixture of recurrent blocks and MQA blocks. Specifically, they employ a layered structure by alternating two residual blocks with a recurrent block followed by one residual block which uses the local (MQA) attention block. Unless otherwise stated, the local attention window size is fixed to 1024 tokens.

New simulation hypothesis drop. Maybe the simulation is not physical and exact but neural and approximate. i.e. not about simulating fields or particles with physical equations but a giant Diffusion Transformer++ creating a large "dream".

Quantum entanglement may generate consciousness. 🧵1/13

Predictions for the future of software engineering:

@Sara_Imari So you do believe in infinity?

@karpathy Can’t wait for the video on this 😁

MINTING OPENS IN ~10 HOURS You have just 3 more hours to enter the Twitter giveaway to win Genesis #48 To Enter: ✅ Follow us ✅ Retweet this tweet #nft #mint #generativeart #nftGiveaway #FreeMint #FreeMintNFT

MINT DATE ANNOUNCEMENT Public Mint: SYD 5am Wednesday, 27th July PST 12pm Tuesday, 26th July UTC 7pm Tuesday, 26th July BST 8pm Tuesday, 26th July Price: 0.125Ξ Supply: 800 Nests (700 available) Links: genesismint.start.page

We are giving away Genesis #48 to our awesome Twitter community. To win: ✅ Follow us ✅ Retweet this tweet That's it. The giveaway will be before the public mint on UTC 7pm Tuesday, 26th July #nft #mint #generativeart #nftGiveaway #minting #FreeMint #FreeMintNFT

We have officially deployed our contract to eth mainnet. Mint 1 is beautiful! Do you agree? #generativeart #generative #nft #Cryptoart #genesismint