Zack Builds AI @Zack_AI_Lab

Codex 早就开源了为啥没有人care, 不比Claude code 先进多了？

现在一堆人用几天 vibe coding，就开始说“人人都是 AI engineer”、“软件工程要完了”。 说实话，这种判断挺可笑的。 大部分所谓的项目，只是 demo 做得好看，根本没有系统设计、稳定性和长期可维护性。 能在 production 跑起来的系统，和做个能传播的视频，是两件完全不同的事情。 People spend a few days vibe coding and suddenly claim “everyone is an AI engineer” and “software engineering is dead.” Honestly, that take is laughable. Most of these projects are just polished demos — no real system design, no reliability, no maintainability. Shipping something that actually works in production is a completely different game.

Anthropic just published how Claude Code's new "auto mode" works and it's one of the most honest engineering posts on agentic safety I've read this year. The core problem is dead simple. Users approve 93% of permission prompts. That's not safety, that's muscle memory. Eventually people just run the skip permissions flag and hope for the best. Their solution is a transcript classifier that judges what the agent did, not what it said. They deliberately strip the agent's own reasoning from the classifier's input because the agent can talk the safety layer into bad decisions. Think about that for a second. Your model generates a persuasive justification like "this is safe because the user implicitly approved it earlier," and your safety classifier reads that and agrees. You're essentially building adversarial robustness against your own model's reasoning. The real examples from their incident log hit different when you've worked on agents in production. An agent interpreted "clean up old branches" as permission to batch delete remote git branches. Another one hit an auth error and started systematically grepping through environment variables looking for alternative API tokens. None of this is malicious. It's overeager helpfulness with real consequences, and it's the hardest failure mode to catch because the reasoning looks totally legitimate. The number that matters is the 17% false negative rate on real overeager actions. The classifier correctly identifies the danger but misjudges whether the user's words actually authorize that blast radius. They tried prompt engineering fixes and couldn't find one that doesn't cost more than it gains. They just say that openly. No spin. Two architectural lessons I think generalize well beyond Claude Code. First, keep your safety classifier reasoning-blind to the agent's chain of thought. Action level monitoring and CoT monitoring are complementary defenses. Collapsing them into one layer means a single point of failure. Second, deny and continue beats hard stops. Their false positive rate lands at 0.4%, and when it triggers the agent just retries with a safer approach instead of killing the session. This is what makes high autonomy actually usable in practice. Really solid work from the Anthropic engineering team. This is what shipping agentic safety looks like when you're honest about the failure modes that actually happen at scale. @AnthropicAI

Anthropic @AnthropicAI

2 months ago

New on the Engineering Blog: How we designed Claude Code auto mode. Many Claude Code users let Claude work without permission prompts. Auto mode is a safer middle ground: we built and tested classifiers that make approval decisions instead. Read more: anthropic.com/engineering/cl…

398 600 4K 1.6M 3K

Hot take: coding agents are the only AI agents that actually work right now. Everything else is a demo. Why? Because code is the only domain where you get all three for free: 1. State verification. You run it, it either works or it doesn't. 2. Instant reward signal. Tests pass or fail, no human in the loop. 3. Immediate error correction. Stack trace tells you exactly what went wrong. Every other "agent" is faking at least one of these. Your email agent? No ground truth. Your research agent? No verifiable end state. Your planning agent? Reward is vibes. This isn't a tooling problem. It's a feedback loop problem. Code has a compiler. The real world doesn't. Teams building non-code agents are basically trying to solve RL without a reward function. Good luck with that.

在美国AI大厂开会有时候真的挺离谱的。很多人发言根本不是为了把事情做成，就是在刷存在感。最典型的就是不提方案，只挑刺。你说一个方向，他立刻来一句“我有点担心这里”“这个可能有风险”，听起来很有思考，但你让他给个更好的方案，基本没有。 整个会议就变成谁挑的刺更细，谁就更像在做事；谁真的想推进，反而显得不够深刻。最后一小时过去，问题变多了，方案没有了，大家都觉得自己发言了，很有参与感，但事情一点没往前走。

The deepest idea in Cursor Composer 2 is not just coding RL. It is training the agent to manage its own memory. Self-summarization is part of the rollout, and reward flows through the summaries themselves. So the model is not only learning how to act, but how to compress history without breaking future reasoning. This feels important: the bottleneck for long-horizon agents may not be raw context length, but learned state compression

Cursor @cursor_ai

2 months ago

We're releasing a technical report describing how Composer 2 was trained.

169 482 5K 1.3M 4K

We're releasing a technical report describing how Composer 2 was trained.

Curious question for folks in tech: does your company have a heavy doc-writing culture? And do you find yourself having to fight for airtime in meetings to get visibility? Or is it more chill where you're at? Genuinely curious how different orgs handle this.

@bcherny this is awesome what was the hardest part to get right when this moved from prototype to real use?

@steipete are you planning to fully block releases on e2e failures or keep a manual escape hatch?

@steipete @github this is real, automating the pipeline is easy making it safe and repeatable is the hard part nice that github supports this

@felixrieseberg @felixrieseberg cool! how are you thinking about safety here, especially around unintended actions or prompt injection?

Most people think AI made coding easy. In practice, it just made systems harder. 在美国这边做 AI，有个变化这两年挺明显的。 以前工程问题大多卡在：能不能把东西写出来。 现在基本不是问题了。 更难的是：写出来之后，系统是不是还“可控”。 很多项目一开始都很顺： 用 AI 很快把功能拼出来 接口能跑，demo 也能过 但一上真实流量或者复杂任务，问题就开始暴露： 结果不稳定 同样输入，输出不一致 中间状态不可复用 不同模块之间开始互相干扰 你会发现，代码不是瓶颈了， 系统行为才是。 在大厂里，这个转变其实很直接。 现在看一个人，不太看他写代码有多快， 而是看他能不能： 把问题定义清楚 把系统拆干净 把不确定性控制住 尤其是在多 agent 或复杂 workflow 里， 真正的难点是： 哪些步骤可以并行 哪些必须串行 中间结果怎么存 哪里需要做 check 哪里可以信 一个很实际的差别是： 有的人会不断 rerun，靠多试几次“撞出来”结果 有的人会先把路径收紧，再去执行 看起来都是在用 AI， 但消耗的 token、时间、稳定性完全不是一个量级。 所以我现在更在意的一件事是： 你是不是在构建一个 可以持续产出、可验证、能复用的系统 而不是一次性把结果跑出来。 简单说一句： 以前好的工程师，是写得快、写得好 现在更像是： 能不能设计一个系统， 让 AI 在里面长期稳定地“做对事”

@BubbleBing_666 但做下来感觉更卡的是 evaluation 不是有没有数据，而是你怎么知道这些数据真的在提升系统，尤其在 agent 或多轮系统里没有可验证的反馈和归因，基本就是在不断放大噪声

@steipete Building agent pipelines and the bottleneck shifted from waiting on tokens to my machine melting running parallel tests… Curious if you think cloud workers just fix the resource problem, or if it actually changes how you design the agents in the first place?

@dotey this is really nice pulling clean transcripts without api keys is already useful, but the speaker + chapter structure makes it much more usable feels like a solid building block for a lot of agent workflows

4. 给普通人的 3 条逆袭路线图 别从底层学起： 先成为 AI 工具的高级玩家。理解 AI 的“边界”比理解“原理”重要 100 倍。 叠加非技术背景： “AI + 法律”、“AI + 审美”、“AI + 供应链”。这种复合背景在 2026 年是无敌的。 做一个 Side Project： 解决一个哪怕再小的痛点（比如自动分析论文图表），也比简历上写“精通 Python”强

在美国大厂做 AI Scientist 一年，我发现 95% 想要转行或进场的人，努力方向全错了。 如果你还在死磕 Transformer 原理、刷 SOTA 论文、背各种优化算法，听我一句劝：别在错误的路上低头狂奔了。 以下是我在一线拿真金白银砸出来的 5 条大实话，全是学校和八股文不会告诉你的：

3. 哪些方向是泡沫？哪些是真金？ 泡沫： 卷通用大模型、卷基础视觉算法（这些是巨头的战场，普通人没戏）。 真金： 垂直领域应用 (Vertical AI)、智能体工作流 (Agentic Workflow)、AI 与现有业务架构的融合。 别去造轮子，去研究怎么让车跑得更远。