An update on Athena, and some unsolicited predictions on what's coming next

On Saturday, Dario Amodei published We Must Pace the Frontier. I agree with him. Go read it if you haven't. I've spent the summer looking at what these models find, and the gap he describes between what they can do and what the rest of us are ready for is the one I've been staring at since May.

The hardest piece of his plan is the one that asks governments to coordinate. They're in a difficult spot: regulate too little and a US company accidentally builds the weapon, regulate too much and it gets built somewhere else, and open source isn't governable either way. Where governments can act is on consumption: not what the models can do, but what happens to what they find. Who gets a vulnerability first, how fast a fix reaches the people running the code, whether the operators who can't move on an attacker's timeline get a head start. That's what Gold Eagle is.

It matters because of where the risk actually lands. The danger in Dario's essay isn't a model in a lab. It's a model pointed at the power grid, the water system, the hospital network, all of it running on decades of software with the same exploitable design. The frontier is where the capability comes from. Consumption is where it does damage, and that's the end you can still defend.

So pacing has two ends. The frontier end is Dario's and the government's problem, and it's hard.

Source link