Mudassir Farooq @MFInterprets

@tomssilver This post - "Making a technical poster": homes.cs.washington.edu/~mernst/advice…

Math people are crazy, this is the list of topics this guy considered “low class” and “beneath him” in grad school becuase they weren’t abstract enough (great thread read the whole thing)

Jonathan Gorard @getjonwithit

5 months ago

I was interested in general relativity, Hamiltonian mechanics, mathematical biology, automated theorem-proving, numerical analysis, complex systems, lots of other things. But these things didn't fit with my then self-image, so I never seriously considered pursuing them. (9/10)

2 15 958 261K 149

I’m not surprised this has 10k likes, but I think it’s pretty bad advice. And often actively destructive to both company culture and performance. In fact, it’s a pretty employee-hostile chart and an abdication of leadership responsibility. Let me explain… The critical assumption underpinning this entire chart is that a single individual contributor, or manager, can resolve the problem That is very often not the case. And what happens when you create a “bring me solutions not problems” culture is that people stop bringing you problems! That is very bad!!! There’s a great Colin Powell quote about this. I hate war analogies in business, but I think it’s relevant here: “The day the soldiers stop bringing you their problems is the day you stopped leading them. They have either lost confidence that you can help them or concluded that you do not care. Either case is a failure of leadership” Do you know what is a good way to get blindsided by big fucking issues? Telling people to solve everything themselves, or that they need to have proposed solutions It is just as likely that they are feeling a *symptom* of a really big problem. Maybe they can solve the symptom. But it doesn’t solve the problem. There is a deep Root Cause Analysis exercise that should be happening. Something structural is broken. Incentives are wrong. Whole departments are misaligned on strategy. The business is missing a critical piece of infrastructure and everyone is doing a dumb workaround Encouraging individual heroics often exacerbates these issues, and kicks the can down the road on the real problem. It is not this person’s problem to solve. It’s your problem. You’re the leader. You need to go investigate. Most importantly though, I think this chart is an abdication of leadership responsibility. Your job is to clear blockers and drive alignment. Yes, your job is to teach people to march up this chart of self-sufficiency for problems within their scope. But they can’t do that without a culture that allows, and even encourages just as much “Level One: There is a problem” identification as Level 5 celebration of autonomy The way that I think about this is that everyone in my department - from the Directors down to the Individual Contributors - is on a constant journey to improve their Problem Identification ability. Is this a problem that can be solved by me, or do I need to escalate this? That’s the right filter. The number of problems that can actually be solved by an IC using this pyramid is quite small. Managers, a bit bigger. Directors, bigger still. Lot of time wasted, and problems unsolved, because cultures are built demonizing problem identification

Alex Lieberman @businessbarista

8 months ago

I stole this idea and now use it with every single employee. It’s the best illustration I’ve seen of teaching someone to be high agency. It says there are 5 levels of work: Level 1: “There is a problem.” Level 2: “There is a problem, and I’ve found some causes.” Level 3:

311 1K 13K 1.9M 20K

I've never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last ~year and a failure to claim the boost feels decidedly like skill issue. There's a new programmable layer of abstraction to master (in addition to the usual layers below) involving agents, subagents, their prompts, contexts, memory, modes, permissions, tools, plugins, skills, hooks, MCP, LSP, slash commands, workflows, IDE integrations, and a need to build an all-encompassing mental model for strengths and pitfalls of fundamentally stochastic, fallible, unintelligible and changing entities suddenly intermingled with what used to be good old fashioned engineering. Clearly some powerful alien tool was handed around except it comes with no manual and everyone has to figure out how to hold it and operate it, while the resulting magnitude 9 earthquake is rocking the profession. Roll up your sleeves to not fall behind.

Do people realize that PCA, or the power iteration algorithm to compute it, is the first and most popular (deep) denoising algorithm? Hence, conceptually, all modern generative AI models are essentially doing the same thing. Now, you decide whether generative AI has its own mind?

A year ago, we verified a preview of an unreleased version of @OpenAI o3 (High) that scored 88% on ARC-AGI-1 at est. $4.5k/task Today, we’ve verified a new GPT-5.2 Pro (X-High) SOTA score of 90.5% at $11.64/task This represents a ~390X efficiency improvement in one year

“Theory of deep learning” went through similar discussions about its goals & purpose some ~5yrs ago. Someone should write about the relations between mech-interp & theory: two communities w/ fundamentally similar motivations (“understand neural nets”), but very different methods.

David Bau @davidbau

6 months ago

At the #Neurips2025 mechanistic interpretability workshop I gave a brief talk about Venetian glassmaking, since I think we face a similar moment in AI research today. Here is a blog post summarizing the talk: davidbau.com/archives/2025/…

23 101 557 109K 377

@cb_doge The actual cover should have credited the people whose research led to this AI boom @SchmidhuberAI

I still maintain that reflective learning is the future of learning algorithms. This is related to but quite a bit richer than thinking about making value functions that work.

Omar Khattab @lateinteraction

8 months ago

ok but does GEPA do it? Reflective prompt optimizers have somewhat solved the problem that “if model can’t guess the right answer, no learning ever happens” The model should be able to LOOK at its failures and be like “ok yea these were stupid guesses, gotta try x in hindsight”

6 9 121 25K 76

when you ask a PhD student when they’ll finish

Has Elon read our new book? To be precise, *current AI* technology is ALL compression and correlation -- only implementation is still far from optimal and natural. But I tend to believe that the intelligence behind mathematics and science is beyond compression and correlation.

Elon Musk @elonmusk

7 months ago

AI is compression and correlation

2K 2K 22K 13.2M 3K

The ladder of intelligence is the ladder of abstraction. L1: Memorizing answers (no generalization) L2: Interpolative retrieval of answers, pattern matching, memorizing answer-generating rules (local generalization) L3: Synthesizing causal rules on the fly (strong generalization) L4: Discovering general principles, metacognition (extreme generalization) To achieve compounding AI you need to reach L4.

average AI / ML engineer at 2 am

Are neural networks Turing complete? This excellent blogpost by colleague @theHessam goes carefully over the limitations and flaws of much of the literature for the case of transformers: lifeiscomputation.com/transformers-a… Note that the first proofs of Turing completeness (to my knowledge) go back to the work of Siegelmann & Sontag in the 90s, showing that a fixed *infinite* precision RNN can simulate an arbitrary Turing Machine: sciencedirect.com/science/articl… Unbounded precision is needed in order to capture the unbounded feature of Turing Machines, namely the unboundedness of their tape contents which into a decimal expansion and manipulated arithmetically to simulate read and write operations. Alternatively, one can resort to an unbounded external memory bank if one wishes to (realistically) maintain finite precision: proceedings.neurips.cc/paper_files/pa… The case of transformers is the same (infinite precision or infinite memory will suffice). Unfortunately, many claims in the literature, which avoid such inclusions but yet make some sort of claim to universal computation, are flawed. Note that to be a universal Turing Machine doesn't mean for each computation you can find an instantion that performs it - it means there is a fixed instantiation that can simulate an arbitrary set of instructions (i.e. simulate the specifications of an arbitrary Turing Machine). This is disanalagous to the notion of universal approximation, which says that given a target function and error threshold, some instantiation of the model size and weights will realize the function within that threshold (thus the instantiation varies with the objective). For transformers, there is a very nice third route. By using a clever decoding scheme, the unbounded nature of decoding can also be used to do universal computation (albeit done in an unconventional way) arxiv.org/abs/2410.03170

By age 40, you should have tens of thousands of unread PDFs

the last thing a model sees before being captured in a cuda graph

Lisan al Gaib @scaling01

7 months ago

of course merged by the 500IQ Tsinghua GOAT himself

83 198 5K 1.5M 2K

You rarely solve hard problems in a flash of insight. It's more typically a slow, careful process of exploring a branching tree of possibilities. You must pause, backtrack, and weigh every alternative. You can't fully do this in your head, because your working memory is too limited. Writing is the external medium that affords the time and precision necessary. Serious thinking must be done in writing. And that's why you can't outsource your writing, because then you're outsourcing your thinking.

🤮

Acyn @Acyn

8 months ago

Pakistan PM: I would like to nominate this great president for the Nobel peace prize. Because I genuinely feel that he is the most genuine and most wonderful candidate… I think that you are the man this world needed most at this point in time. The world will always remember you

2K 773 4K 4.1M 1K

Multiple full days at a time to study and think deeply about a problem is great. Once it stretches out into years it turns into a form of psychological torture akin to solitary confinement.

Andrew Gordon Wilson @andrewgwils

8 months ago

I miss the days of being a PhD student, or postdoc. I would give almost anything to have multiple full days at a time, just to concentrate deeply and single-mindedly on open-ended research.

64 115 3K 297K 458

"mom, how did we get so poor?" "your dad started a PhD"