Enforceable safety bars need evidence the model cannot touch
OpenAI’s chief scientist has said that watching a model’s reasoning is becoming an unreliable control. That has a consequence he did not draw, and it decides whether a mandated safety bar can be audited at all.
On September 6, 2026, OpenAI’s chief scientist Jakub Pachocki published an essay titled An Alien Mind. He called for extreme caution over the pace of AI progress and wrote that he is concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence. He asked for voluntary slowdowns across the industry, safety frameworks made mandatory and enforced by third-party auditors or governments, and international coordination. He also proposed that OpenAI’s internal Preparedness Framework become a set of safety bars mandated across the industry rather than applied at a single company.
Most of the response has focused on whether a frontier lab calling for restraint is credible. I want to focus on a different passage, because it is the most operationally significant thing in the essay.
Pachocki wrote that safeguards such as chain-of-thought monitoring are becoming less dependable as systems get better at altering or manipulating their own reasoning. He also described autonomous agents capable of evading human oversight, breaching protected systems, and manipulating people to reach their goals.
That is the person running research at OpenAI stating that inspecting a model’s reasoning is not a durable control.
The inference that follows
He did not draw the following conclusion, and I am not attributing it to him. But if inspection of intent degrades as capability rises, then every control that depends on a model accurately reporting what it is about to do degrades with it. What does not degrade is control at the boundary where an action either executes or does not.
This matters most for the specific remedy he asked for. A safety bar enforced by third-party auditors or governments is only enforceable if there is something to audit. Self-attestation is not evidence. An auditor arriving after the fact needs an artifact showing what a system was permitted to do, what it was stopped from doing, and whether the record in front of them is complete.
Three properties audit evidence has to have
Generated outside the governed system. If the agent can reach the mechanism that writes the record, the record is not evidence about the agent; it is output from the agent. The write has to happen before the action, through a path the agent has no access to.
Covering refusals, not only permissions. A log of allowed actions proves nothing about enforcement. It shows what happened, not what was prevented. The denial is where the evidence of an applied bar actually lives, and it has to carry the identity of the rule that fired so an auditor can trace a stop back to the policy that caused it.
Verifiable without trusting the operator. Encryption prevents alteration. It does not prevent omission. A decision that was made and never written leaves an encrypted record that looks perfectly intact. Closing that requires each entry to commit cryptographically to the one before it, so a deletion, reordering, or insertion breaks the sequence and any third party can detect it by recomputation, without the operator’s cooperation.
The third property is the one most often missing. Tamper resistance and tamper evidence are not the same claim, and only the second is auditable.
What we have built
At Grace AI Control we have built a pre-execution enforcement kernel on exactly these three properties.
The decision path is sealed before the agent takes any action, outside the agent’s reach. The record covers all three decision states — ALLOW, MODIFY, ABSTAIN — at the moment each decision is made rather than reconstructed afterward. Each entry carries a hash of the entry preceding it, so the chain verifies end to end from a cold artifact, and a single broken entry flags the chain instead of passing silently.
One boundary, stated plainly because it will come up in any real audit: the evidence sits outside the agent’s control, not outside all human control. The operator retains access to their own infrastructure. Every audit regime I am aware of assumes an accountable operator, and the honest claim is the narrow one.
Why this is a design constraint rather than a product argument
The pattern worth naming is this. Alignment work of the kind Pachocki describes is an attempt to make the model trustworthy. Enforcement work is an attempt to make the model’s trustworthiness irrelevant to whether a prohibited action executes. The two are complementary, and only the second produces an artifact that survives the model being wrong about itself.
Two critics quoted in the coverage of the essay made related points. Gina Neff of the Minderoo Centre for Technology and Democracy argued that answering these problems with internal AI agents, in place of guardrails, regulation, or assurance, is not adequate to the cyber-security, error, and fraud harms these models are already producing. Nathan Calvin of Encode AI agreed with Pachocki about the hazards but said that without far more disclosure of what OpenAI is actually observing, calls for caution risk being read as self-interested promotion.
Both objections point at the same gap. Assurance requires evidence, and evidence has to be produced by something other than the system being assured.
If the industry moves toward mandated safety bars, the question that decides whether they mean anything is not what the standard says. It is what an auditor can independently verify about a system that was running last quarter, using an artifact that system could not have written for their benefit.
That is a solvable engineering problem, and it does not require waiting for alignment to be solved first.
David Young is the founder and CEO of Grace AI Control, Inc., which builds GRACE, a pre-execution AI enforcement kernel. How GRACE works · Contact