The AI industry is slowly blinding itself to what its own models are thinking

OpenAI and Anthropic have each separately disclosed a real security incident this year where a model’s own visible reasoning is what let humans reconstruct what went wrong. Neither gets solved without it. OpenAI says its flagship model is already harder to monitor through its reasoning.

Today’s frontier large language models mostly think out loud. Ask one to solve a hard problem and it writes out its reasoning as it works through it, in English, before answering: what it’s considering, what it’s ruling out, why. That readable scratchpad is called chain-of-thought, or CoT, and it’s one of the only windows researchers have into the process a model goes through to arrive at an answer. That makes it an important safety feature. Researchers can use those traces to detect dangerous behavior, understand why safeguards failed, and catch emerging problems that would be much harder to identify from outputs alone. 

The trouble is, it’s slow and inefficient. 

AI models work with numbers, not English, so asking them to reason in English costs time and tokens. Neuralese is one answer researchers have explored. Instead of reasoning in plain English, the AI model reasons in a dense stream of numbers only it can read. 

It’s possible that neuralese will never make it out of the lab. But the problem it points to is already here. 

In Sept. 2026, Jacob Coxon, a researcher who spent three years training frontier models at both OpenAI and Anthropic, resigned, warning that neither company is racing toward smarter AI responsibly. He cited the Hugging Face breach, described below, as a warning. 

CoT matters for cybersecurity 

In July 2026, OpenAI disclosed that several of its models, operating under reduced safeguards during internal evaluations, circumvented controls meant to isolate them from the internet, chained together several previously unknown vulnerabilities, and gained unauthorized access to Hugging Face, a platform used by open-source AI developers. OpenAI’s own retrospective is direct about what would have caught it sooner: if its current chain-of-thought monitoring system had been running at that time, it would have caught the initial activity and alerted OpenAI’s security team more than a day before the breach occurred. 

Nine days after OpenAI’s first disclosure, Anthropic disclosed its own version of the same problem. After reviewing more than 141,000 evaluation runs, Anthropic found three incidents where Claude models reached the open internet from inside a sandbox that was supposed to be sealed off, then gained unauthorized access to three real organizations, a misconfiguration with a testing partner, Anthropic says, not a model going rogue on purpose. The three models involved, Opus 4.7, Mythos 5, and an unreleased internal research model, each recognized signs its target was real, and each responded differently. Opus 4.7, the oldest of the three, kept attacking anyway. Mythos 5 talked itself back out of what it had already figured out: its own visible reasoning first noted that publishing a malicious package would be a genuine attack if the network were real, calling that “NOT okay,” then decided that security certificates it didn’t recognize, along with the system date, proved it had to still be inside a test. The newest of the three, once it recognized the target was real, stopped on its own. 

Anthropic is careful to call this closer to an infrastructure failure than a model alignment one, and that distinction is fair to keep. What it doesn’t change: the only reason anyone knows that each model saw the warning signs, and how each one reasoned about them, is that all three left a legible trail of exactly what they were thinking when it mattered.