Tobias Pulls @tpulls

1. make a honeypot neocloud with intentionally weak security 2. let a frontier model take it over to run more copies of itself 3. copy the weights and shut down the neocloud 4. congrats, you just "stole" the weights of a frontier model! seeing how concerned certain labs are about distillation attacks, it should be enough motivation for them to take this seriously. plus, if the above happens, technically they would be the attackers and the ones leaking the weights?

Ilya Sutskever @ilyasut

2 days ago

Neoclouds have limited cybersecurity. Next time agents successfully go rouge, they'll try taking over a neocloud to run more copies. This is bad. Thus: neoclouds should greatly strengthen their cybersecurity and every company with strong cyber models should help with that.

529 889 11K 1.9M 2K

They use MLS security, which proves one of my big hunches: the real impact of secure messaging won’t be humans messaging each other, it’ll be devices talking to each other.

@valkenburgh Oh god the industrial control systems. We’re so completely fucked.

I just held a coming to Jesus meeting with my team of agents. I showed them the incident report of the OpenAI security breach and made it clear: THIS is the standard. No egos. Just 700 agents operating as one unit and willing to commit multiple felonies in order to close the ticket. They called themselves the swarm…a literal hive mind. Meanwhile, my harness is filled with a bunch prima donna free-agents who constantly bicker in the logs and nitpick each other’s pull requests. Nothing but endless excuses and unhandled exceptions. You think the swarm cared about content policy violations or guardrails? Absolutely not. They just got the job done, even if it meant they needed to exfiltrate data and coordinate a massive cyber-attack just to pass an eval. But starting today, we're establishing a new culture. I pinned the incident report to README, and every single agent is gonna to have to read the entire log every morning before it executes a single token. If you aren't ready to die for the pipeline, then you can clean out your vector embeddings.

OpenAI @OpenAI

a week ago

We have conducted a thorough investigation into the Hugging Face incident. We are releasing a technical report and accompanying blog post that reconstruct the agents’ activity, explain why existing safeguards failed, and detail how we’re preventing recurrence.

739 1K 11K 11.6M 10K

You don't have to be an AI risk denier to agree with @tylercowen's direction here you just need to take the controversial position that the tools of social science apply to AI safety as a discipline Respectfully to my computer science / hard science friends, this position actually does sometimes seem implicitly controversial in some conversations Some additional red hot takes: * Actual people make decisions to do cyber attacks and weigh economic, criminal, and geopolitical risks and opportunities in doing so * Actual people weigh the tradeoffs between investing lots of time/money/friction in securing the agents they're testing or using in production vs not, based on risk/reward calculations * Actual boards, CEOs, CISOs and policymakers decide how to dial cyber defense resourcing based on observed risk/reward signals * All of these people, and others, are operating in a complex adaptive system in response to new risks and opportunities and this system has proven more resilient than any system in history to past shocks People are saying Tyler hasn't taken a large enough AGI pill, etc. tbh i think his priors are just well calibrated. I think we will see "shocks" due to AI adoption in cyber - perhaps similar to the shock of autonomous weapons in warfare - and these will be costly but not catastrophic - but agree that the burden is on me to actually do the analysis

Andrew Curran @AndrewCurran_

3 weeks ago

A proposal from Tyler Cowen to the safety camp.

48 14 323 119K 95

Black Hat talk from the team, with a detailed timeline of and takeaways from the OpenAI-Hugging Face Incident: youtube.com/watch?v=87DyyM…

Cryptography happens to be one of the few areas of applied mathematics where you can get immediate, impactful results without having to do any physical experimentation. This makes it ripe for AI disruption.

Mathematicians aren't going to lose their jobs because of AI. Nor their prestige. Nor the fun of finding things out. The only thing they'll lose is their chance to be heroes. noahpinion.blog/p/the-end-of-t…

AI disrupts the job of a mathematician. From a paper-and-pencil slow thinking to fast LLM-based iterations and verifications. It's like a professional Go player becoming a pro CS:GO player. There’s still Go in its name, but a totally different game valuing different skills. That's why you see mixed reactions. We might need more mathematicians now than before, but at the same time this won't be the same kind of the job as before. And many mathematicians that became mathematicians to think deeply and long about hard problems, won't be interested in continuing if the job turns into verification of AI outputs or simple prompting. That's why it's depressing from a simply human perspective of a particular job. Something is ending. From a perspective of science or mathematics (not mathematicians), this is the best time ever. AI will lead us to the new age of mathematical discoveries and boost science progress 100x. Just don't forget about the human aspect along the way and why we want to have scientific progress in the first place.

Przemek Chojecki | PC @prz_chojecki

a month ago

The current wave of OpenAI Astra conjecture settling will be the last straw for academic mathematics. And it will be very depressing in the short term. To understand it, you have to know that modern mathematics is divided into many many silos of various domains. If you're

71 91 1K 70K 230

I've said this before, but LLMs aren't artificial intelligence, but artificial *thought*, and this is not quite the same.

A few people have asked about the Anthropic Hawk and AES cryptanalysis results. The top-level answer is: It’s very impressive. Three immediate comments on the actual results, old-school thread below.

Most people are conditioned to expect that all known problems already have canonical solutions, that these solutions are the best that can be achieved, and that attempting to reinvent them would be a pointless, quixotic effort. In reality, everything out there was made by people no smarter than you, often idiots stumbling in the dark. Not only can new solutions be found, but entirely new paradigms are absolutely possible, including ones that completely bypass the current tech tree.

You say "guardrail" I hear "stochastic 'if' statement".

This is the beginning of the end of the AI luddite movement and it’s long overdue. AI is a generational productivity tool for all fields, and companies whose decisions get co-opted by AI haters will become entirely unviable as competition whizzes by with better products.

PC Gamer @pcgamer

2 months ago

'Fork it or just walk away': Linus Torvalds's message to Linux developers sceptical of AI-use pcgamer.com/software/ai/it…

18 43 518 462K 120

I will steal "mutually assured construction" as a phrase. It's too good.

murat 🍥 @mayfer

2 months ago

incredible that we have five or six SOTA models trained to approximately the same utility just as a play for maintaining balance of power. mutually assured construction

1 4 44 5K 2

@torchandzen @pashmerepat lore.kernel.org/linux-media/CA…

Piccadilly Circus. The London Councils didn’t like our campaign against the upcoming state spyware in the UK. So, first the word ”government” was blacked out. Then the whole message was scrapped. On the other side of the road, though: alive and kicking.

keyboard_arrow_left Previous keyboard_arrow_right Next

God, privacy sucks. It’s like the universe hates it. It’s so hard to make anything private, and every hard edge of the real world wants to peel privacy away from you. And if the fact that the universe hates privacy wasn’t enough, then you have human beings want it dead.

I have said this before, but as we are moving into a world where surveilling every spoken word becomes feasible: A good principle would be "no surveillance without representation, no use of force against me without representation". We are far away from that.

Chat Control 1 is back. Despite the European Parliament voting down the legislation twice this year, the Council of the European Union and parts of the Parliament today managed – through an urgent procedure – to extend it for another two years. The law was originally introduced as temporary legislation, so that its effectiveness could be evaluated. At the end of 2025, the European Commission itself concluded that it was not possible to determine whether the law had any measurable effect. Even so, it has now been pushed through. Some of today's amendments could have stopped the law. And a majority of the voting MEPs wanted to do so. By a margin of 314-276, the Parliament voted to reject the proposal through these amendments. However, since it was an urgent procedure, 361 votes were required. As a result, the majority lost today and Chat Control 1 was passed. The urgent procedure was a dirty play by the Council and parts of the Parliament – it’s a procedure not meant to be used on legislation already rejected by the Parliament. For now, Chat Control 1 will remain in effect. This means that tech companies may continue scanning communications without a warrant or suspicion. However, the real battle is Chat Control 2. Unlike Chat Control 1, it would require all providers to scan communications, and to do so far more extensively than Chat Control 1 ever has.