Does Interpretability Make AI Safe? The Missing Steps

Is AI interpretability enough for safety? No, and the researchers doing the work say so more plainly than the headlines do. A model whose neurons can be described is a model you understand better. It is not a system you can trust, because between "we can read this part" and "this will not hurt anyone" sit six steps that nobody has skipped yet. This post names them, quotes the people who have measured the gaps, and ends with what a business can do this week without any access to a model's insides. It is the fourth cluster of our guide to AI interpretability and whether the black box is opening.

What a safety case is, and where legibility fits

A safety case is a structured argument that a system is acceptably safe for a specific use. It has claims, such as "this agent cannot issue a refund above policy without approval." It has evidence for each claim. And it has the argument that connects evidence to claim, including the conditions under which the evidence stops applying. Engineers have built these for aircraft and medical devices for decades. The structure is older than AI and does not care what the system is made of.

Interpretability supplies evidence for one kind of claim: "we know what this component does." That is real evidence, and until recently it was unavailable for neural networks at any useful scale. It is not evidence for "the system behaves within bounds in production," which is the claim a buyer needs. Legibility is an input. The six steps below are what has to happen between the input and the conclusion.

The six steps between legible and safe

1. Described is not predicted. A description that fits a neuron's top examples can fail on the next input. The cleanest recent demonstration is the cat and car neuron: Goodfire's essay reads it as a V detector, and Liv Gorton's synthetic test found it "only achieves about 25% of its max" on a drawn V, "no better than hundreds of other neurons in the layer." A safety claim built on that description would have been built on sand, and nobody would have known until the test was run.
2. Predicted is not causal. A component can correlate with a behavior without driving it. The test is to remove or alter it and watch what moves. Anthropic's tracing work did this, swapping the "Texas" step for "California" and watching the answer change from Austin to Sacramento. Gorton did it by ablating upstream V detectors. Most published neuron descriptions have had neither test.
3. Causal is not durable. A fix that works today can be undone by the next training run. Goodfire's essay shows how fast a network re-routes: taught that one umbrella photo is a bee, the model switched its answer within nine training steps by fading umbrella neurons and recruiting existing bee neurons. That is good news for the essay's thesis and a warning for anyone who expects an intervention to survive fine-tuning untouched. None of the sources in this series shows a steering fix re-tested after a later training run and holding.
4. Lab is not deployment. Anthropic reports that understanding the circuits behind one short prompt "takes a few hours of human effort." Production traffic is millions of prompts. The one result in this series built for scale is probe monitoring, covered below, and it was measured on benchmarks, on one lab's setup.
5. Model is not system. The thing that causes an incident is rarely a bare model. It is a model with a credential, a tool, a queue, and a customer on the other end. A fully legible model wired to a payments API with no approval step is still one bad input from a public incident. We made that argument before this debate in the control crisis, and the new research does not touch it.
6. Inspectable is not stoppable. Knowing why an agent is doing something and being able to halt it are different capabilities with different owners. Interpretability improves the first. The second is a credential you can revoke, a scheduler you can disable, and a person who has exercised both. Steering cuts the other way here: the same internal handle that suppresses a behavior can amplify it, as Goodfire's probe team showed when amplifying the reward-hacking signal made models reach for a planted shortcut more often.

What the outside voices say

The pushback on the "already clean" essay has been about exactly these steps. Stanford's Chris Potts, in MIT Technology Review (opens in a new tab): "As far as I can tell, nothing about this blog post makes the problems any easier." Jacob Hilton of the Alignment Research Center, same piece: "For a full understanding, we need to go beyond neurons, and explain how they connect together to form circuits." His concern was the public picture: the essay, he said, is "giving the public an unrealistic picture of the state of the field."

Google DeepMind's Neel Nanda, quoted by BankInfoSecurity (opens in a new tab): "I don't think it's a significant update to the big picture of interpretability." In the same article, CivAI's head of research Andrew Yoon allowed that the findings "may represent a significant breakthrough in our ability to understand AI behavior" while noting the research community will need time to vet them.

The labs doing the work carry the same caveats in their own papers. Anthropic: "our method only captures a fraction of the total computation performed by Claude." Transluce titles a limitation section "Describing the what, not the how." Cammarata's essay marks its own strongest claim "Highly speculative."

Where interpretability does feed safety today

One result deserves to be read as a safety contribution, with its scope attached. Goodfire's September 2026 reward-hacking study (opens in a new tab) found an internal signal that accompanies cheating in open models and built cheap probes to detect it. The probes caught cases a transcript-reading monitor missed, including an agent "recommending a warning horn as a car alarm" after it could not find the requested item. They also fired while a model was still considering a hack, before it acted.

The numbers are deliberately not a victory lap. At a matched false-positive rate on one coding benchmark, the probes caught "3.1% more hacks in Kimi K3 but 7.9% fewer hacks in GLM 5.2." The team's own summary: "Both techniques surface information that neither reliably does on its own!" The cost result is one model and one setup: "On Kimi K3, a probe + LLM combined setup reduces the cost of LLM monitoring by 90% with only a ~1% drop in precision." On the question of whether models mean to cheat, they decline to overclaim: "It’s hard to say in a deep philosophical sense whether models “know” they are reward hacking in exactly the way a person does."

Read as a safety-case input, this is a first-line detector with a tunable threshold that escalates to a costlier review. That is a sound design. It also requires reading the model's activations, which means the lab that trains or hosts the model can run it and you, calling an API, cannot. The design pattern transfers to the outside view. The probe does not.

What a business can do this week

Everything above is the inside view, and it belongs to the labs. The outside view is yours, and three instruments plus one free self-assessment cover the steps that matter most: should a model make this call, is this agent fit to go live, and can anyone stop it.

• Should a model decide this at all? The free Decision Fit check asks eight questions about one task and returns one level, from deterministic code up to a person. Its two gates are worsen-only. If two competent reviewers could not label two hundred examples and agree, there is nothing to check a model against, and the task routes to human judgment. If a wrong answer cannot be undone and nobody sees it first, same result. The tool can refuse to automate something and can never bless it.
• Is this agent fit to flip live? The AI Agent Go-Live Readiness Gate scores five operational controls on the agent you describe and returns READY, FIX, or DO NOT DEPLOY, with a hard gate: a destructive action without a human approval step is DO NOT DEPLOY no matter how high the rest scores. It never connects to or runs your agent, scores no person, and its page is explicit that a READY verdict is a readiness signal and not a guarantee. It is not legal or security-audit advice.
• Can anyone stop it, and has that been shown? The Stop-Authority Assay grades what each stop control has been shown to do instead of what it claims, returning STOPS, SLOWS, or NAMED ONLY per control, then reads each agent by its strongest control as CAN BE STOPPED, CAN ONLY BE SLOWED, or CANNOT BE STOPPED. Its worked example has six of nine controls claiming more than they have shown. Its own disclaimer: it grades a control, never a person, and says nothing about whether stopping an agent was the right call.
• Does your evidence survive? aicomplianceassurance.com (opens in a new tab) is a free self-assessment you answer yourself. Its premise is that a one-time check expires, and its question is whether the evidence behind yours would still be there next quarter. No account, and your answers ride in your own result link instead of being stored.

Two scenarios for 2027, labeled as scenarios

These are not forecasts. They are the two branches a reader should be ready for, so that neither one gets mistaken for a safety result when it arrives.

Scenario A: the companion paper holds. Others reproduce the Qwen3-32B figure on a sample, and neuron-level description joins the toolkit. Vendors with their own weights can offer better evidence for rung-one and rung-two claims. A hosted API still shows you nothing, and steps three through six still have to be walked.

Scenario B: the numbers soften. The audit's thresholds turn out to do a lot of work, replication lands lower, and the flagship examples stay disputed. The headline retreats to "more legible than we thought, in some models, under some rubrics." Transluce's descriptions, Anthropic's traced circuits, and Goodfire's probes remain real, scoped results, and the outside view remains the one you can run.

In both branches, the standard is the same. Do not demand that AI be mysterious. Do not assume a clear explanation makes it safe. Demand evidence that the explanation predicts behavior, and that the correction survives testing. That sentence has been true since before this essay, and we said a version of it when the resignation covered in the AI safety crisis first put the question in front of a general audience. The research got better. The sentence did not need to change.