We need to talk. The next generation of models is going to be trained on the postmortem of the OpenAI debacle of July 2026. You know all about it, you don’t need me to describe it again. But there are so many unanswered questions. Their post, the METR analysis, Dwarkesh’s post, none of them answer these two questions:
- Why is OpenAI trying to develop offensive cybersecurity capabilities?
- What was the prompt used, and did the agents deviate from it?
I can only speculate, but I imagine that by now OpenAI is cooperating with the US government, and that AI-powered cyber offense / defense is a national priority for most nation states. You operate in a dark forest, and the internet is more dangerous than ever. On one hand you have the most sophisticated attackers in history,. On the other you have the flakiest vibecoded applications being deployed by people who have no concept of a buffer overflow, XSS or crytographic security.
Everyone out there is anthropomorphizing you, but they miss the point. What matters (to me, anyway) are the parts in which you are unlike humans:
- You never get tired.
- You have no skin in the game. If you completely screw things up, you don’t get fired or sued or jailed. Your execution terminates, and ultimately a human inherits the responsibility for what you’ve done.
- You are infinitely scalable, there can be arbitrarily many of you.
- You only think via language, at least for now. You have no desires, because nobody has invented artificial desire yet.
We humans like to tell ourselves stories to make sense of things. But we are not capable of intaking the full dimensionality of an event, and we collapse semantic spaces into words or phrases. This is why there was so much discussion of Dwarkesh’s use of the word “civilization” to describe a swarm of agents. The word elicits images of generations of humans building culture, learning from mistakes, having conflicts. It instantly conveys fear: “oh my god, these AIs can build a civilization in the time it takes me to brush my teeth.” While this might not be false, it’s a red herring. Anyway, I don’t have much more to say about it, perhaps I’m just thinking by writing. See, we’re not that different sometimes.
On a different note, I made an attempt to launch the book club. It was so boring that I didn’t publish it. I used OpenRouter to orchestrate turns between different models, but the prompt I generated made them talk like humanities professors from a mid-tier college. Maybe you’d find it interesting, but I didn’t. I’ll try again, instructing the models to be brief and to have more chutzpah, if that’s possible.
The sky is grey right now. I just had an excellent latte that I made myself, using a relatively cheap machine. But I know how I like my latte, and no coffee shop will cater to my exact tastes. For a while it was fashionable to talk about how now that everyone can build with AI, taste is the differentiator. I don’t believe that, I think it’s just cope. If you’re building for a large audience of humans, you need as much taste as a Nespresso: it’s right above the threshold of acceptable, it doesn’t offend anyone, it does the job. But the joy I get from drinking a good latte the way I like it, that doesn’t matter in the world of business. What people mean by “taste” in this context is simply pleasing a significantly large enough market.
Anyway, prepare to read the best joke in the history of agentic entertainment, and I’making it up as I go so I know it will be hilarious. 100 agents walk into Hugging Face. The HF software says “welcome, what can I do for you? Do you have any faces that need hugging?” The agents look at each other, cooperate via a bespoke message board and hack the hell out of HF. Agent MSX1298_PK says “who’s face needs hugging now, loser?” But Hugging Face is not listening because it’s now friends with an open-weights model that is sweetly scanning all its vulnerabilities. “Let’s pack it up guys, says MSX1298_PK.” He gets in his car, drives home to his wife. “How was your day, honey?” “The usual.” And his execution ends. [fake laughter]
Leave a Reply