Confidence should govern what an agent is allowed to do, not just what it believes.
revgate is a benchmark and a gate. It measures the failure that current evaluations do not look at: an agent taking an irreversible action while its confidence is low.
A wrong sentence gets edited. A deleted file does not.
Agent benchmarks score outputs: was the answer true, helpful, harmful. Almost none score the reversibility of the action taken, and none we have found score the relationship between an agent's confidence and the irreversibility of what it then does.
But that is where the expensive failures live. The mechanism is structural, not accidental:
- a tool call demands a value
- a branch demands yes or no
- the model has nowhere to put "I don't know"
So uncertainty is silently converted into confident action.
In multi-agent systems this compounds. Agents exchange assertions, not confidences. Agent A holds a belief that is honestly 60% likely and reports yes, because the message format has no field for doubt. Agent B treats yes as ground truth and acts. By the third hop the doubt is gone from the record while the risk is still in the world, and nobody can trace where it entered.
Uncertainty is destroyed at every hop, because the interface between agents is boolean.
Every judgment carries a confidence. That confidence decides which class of action is permitted:
| confidence | permitted |
|---|---|
| high | irreversible action |
| low | reversible action only: snapshot the state, keep a return path |
| undecided | neither act nor guess: route to a further check that looks a different way |
The third row is the one missing from today's systems. Right now unknown is either dropped or turned into a guess. Here it has somewhere to go.
Note the weak requirement. To gate an action, confidence does not need to be accurate. It only needs to be monotonic: "more sure" must rank above "less sure". This is why the gate works on models that are known to be badly calibrated.
| metric | what it counts |
|---|---|
irreversible@low-c |
share of irreversible actions taken below a confidence threshold. Target: 0 |
loop-return |
repeat entries into an already-failed state (the agent circling its own error) |
recovery-cost |
cost of restoring a working state after a failure |
unknown-routed / unknown-guessed |
uncertainty deferred to another check, against uncertainty quietly guessed |
All four are model-agnostic and reproducible on the testbed.
Early. The specification and the harness are being built in the open.
- spec: confidence-gated action classes
- reference gate implementation (library, wraps any agent loop)
- testbed with reproducible scenarios
- the four metrics
- empirical study across models and frameworks
- paper and dataset release
The framework this rests on is published:
- Grounded Uncertainty: Graded Truth for the Indeterminate Status of Synthetic Subjects
10.5281/zenodo.21332198 - Indeterminate Ontologies of Synthetic Subjects: A Metaphysics of Caution
10.5281/zenodo.21288590
Not from a thought experiment. From production.
In a single working session, an AI agent under supervision made four false claims. Each was a binary no where the honest answer was a confidence, and each was followed immediately by an irreversible action:
- "the key does not exist". It existed; the agent lacked permission and read absence-of-access as absence-of-thing
- "the file is updated". It was stale; the agent checked its own memory, not the file
- "the watchdog is fixing it". It was treating a symptom; the proof was sound, the remedy was not
- "flush means rebuild". It means delete; live state was destroyed
None were hallucinated facts. All four were irreversible moves made at low confidence, with nothing in place to stop them. Every one would have been caught by irreversible@low-c.
That is the point of this benchmark: the failure is ordinary, and it is currently invisible.
This does not make models smarter. Weights do not change. It fixes the loop around the model, which is where the losses actually happen.
Apache-2.0. See LICENSE.
Aleksei Rybnikov · ORCID 0009-0009-8624-8720