Proposal: Add reversibility-graded authority as the HITL gating axis (Playbook 5 / T10)

Re: Agentic AI – Threats and Mitigations v1.1 — Playbook 5 (Protecting HITL & Preventing Decision Fatigue Exploits) and T10 (Overwhelming Human-in-the-Loop).

Gap

Across Playbooks 3–6, the human-in-the-loop / autonomy gate keys on risk level, impact, domain, or privilege — never on the reversibility of the action. E.g.:

P5, Step 1: "Automate low-risk approvals while requiring human oversight for high-impact tasks."
P3, Step 2: "explicit user approval for AI tool executions involving financial, medical, or administrative functions."
T10 mitigation: "low-risk decisions are automated, and human intervention is prioritized for high-risk anomalies."

Risk/impact/domain are estimates; reversibility is an intrinsic, deterministic property of the action.

Proposal

Add reversibility as the gating axis (primary home: P5 Step 1; cross-ref P3/P4/P6). Classify each agent-invocable action by whether it can be undone:

Read-only (no state change) -> autonomous
Reversible (engine can roll back / auto-expires) -> autonomous, logged
External-reversible (recoverable via a separate process) -> human approval
Irreversible (cannot be undone) -> human approval + evidence threshold

Worst-case governs: a multi-step task gates at the least-reversible action it can reach.

Why it strengthens the doc

Directly serves Playbook 5's goal: reserving the human gate for irreversible actions (not all "high-risk") keeps the review queue small — the core defense against the decision fatigue T10 exploits. It also caps the blast radius of T2/T3/T6/T7/T13 regardless of attack path, since in all of those the agent acts with legitimate access.

Note on terminology

This "reversibility" (a property of the action) is distinct from the "decision reversal" detection in Playbooks 1 & 6 (flagging suspicious approval-flips). Both can coexist; worth naming explicitly to avoid conflation.

Standards anchors

NIST SP 800-53 AC-6 (least privilege at the action level); EU AI Act Article 14 (human oversight). Complementary to Playbook 4 (identity/authz): authz decides permission, reversibility decides the gate. Mirrors the reversibility axis proposed in OWASP AISVS (OWASP/AISVS#820, C9.2), so the two artifacts reinforce.

Happy to draft the PR for the Playbook 5 Step 1 bullet + cross-references.

Following up on this proposal with the concrete draft text, since I offered to write it. Happy to submit it whichever way the workstream prefers, a PR against the doc source or a suggestion in the working doc.

Proposed edit, Playbook 5 Step 1 (extending the bullet "Automate low-risk approvals while requiring human oversight for high-impact tasks"):

Gate human oversight on the reversibility class of the action, not on risk level alone. Classify each agent action by whether its effect can be undone:

Read-only: no state change. Runs autonomously.

Reversible: the effect is restorable by the same system, for example a local write with a snapshot. Runs autonomously, logged.

Externally reversible: undoing it needs a third party or a costly compensating action, for example a refund or a retraction. Requires human approval before execution.

Irreversible: the effect cannot be walked back, for example a completed payment, an external disclosure, a physical actuation, or a deletion without backup. Requires human approval plus an evidence threshold.

For a multi-step or multi-agent task, gate the whole task at the worst-case, least-reversible class reachable across it, evaluated before the task begins, so a sequence of individually reversible steps cannot compose into an irreversible outcome without review.

The reversibility class is a property of the action type, declared at design time (for example in the tool or action manifest) and trusted by the gate, not asserted by the agent at runtime. An action whose class is unknown or unverifiable is treated as irreversible until classified, so the gate fails closed.

Cross-references: add a one-line pointer to the reversibility class where each of these already gates autonomy, Playbook 3 Step 2, Playbook 4 Step 2, and Playbook 6 Step 1.

One addition on the standards side: this is the mitigation half of the axis, and the verification half is tracked in #15 (requirement-level AISVS C9.2.3 / C9.2.4 / C9.2.10 mapping for the agentic entries). They compose: the Playbook gates by reversibility class here, and #15 maps those classes to the AISVS controls that verify them. Worth noting the AISVS C9 reversibility controls have since shipped in v1.0, so this anchors to a released standard now, not a proposal.

Independent researcher — I've been running a two-class version of this
for four months and have measurements that speak to the review-queue
argument.

Setup: two curator agents evaluate artworks and bid against each other,
settling on a public test network. Four settled cycles, all transactions
public. My gate is cruder than your four classes — irreversible/not,
where irreversible means "moves value on-chain" — so treat this as one
data point, not a validation of the full taxonomy.

On "keeps the review queue small": in one duel, the agents exchanged ten
signed offers off-chain and produced one on-chain transaction. Verified
by full block scan over the auction window. So gating the irreversible
step cost one human gesture per auction, not one per decision. The 10:1
ratio is what makes your argument hold in practice — the cost of HITL is
usually argued from the wrong denominator.

Your point that the class must be "trusted by the gate, not asserted by
the agent at runtime" is, in my experience, the one that decides whether
any of this works. Concretely, ours holds because the agent has no tool
that can reach the confirmation channel: a session secret it never sees,
plus a nonce bound to the specific pending transaction, single-use, short
window, abort on anomaly. A confirmation an agent can issue is not a gate,
whatever its class.

One boundary worth naming, since "external disclosure" is in your
irreversible list: the class is a property of the action type, declared
in the tool or action manifest. That works because an action passes
through a tool. When an agent produces text, it invokes nothing — the
output is the product. A false statement published in a document crosses
no manifest, spends no budget, requires no approval. ASI09 treats
misleading output as a vector for approval; the terminal case, where the
statement itself is the damage, has no action to classify.

I don't think that weakens the proposal — it marks where it stops, which
seems worth stating if it ships.

Artefacts: github.com/avp9-nexus/nexus-art — security patterns in
poc-001/, and a dated register of 18 errors classified by what caught
them (11 external artefacts, 3 human, 0 self-check rules).
Avp9.

Thanks for this, and for putting the scope limit up front rather than making me infer it.

The denominator point is the part I could not previously point at. The objection to reversibility gating is always that it buries a human in approvals, and it gets argued from the count of decisions rather than the count of irreversible ones. Ten off-chain offers producing one settlement is the shape I would expect, and if it holds anywhere near that in other domains, the cost argument is not just wrong, it is wrong by an order of magnitude.

Your confirmation channel is worth naming precisely, because it lands on two AISVS 1.0 requirements rather than being an ad hoc design. 9.2.8 asks that approvals be cryptographically bound to action parameters, requester identity, execution context and a unique single-use nonce. 9.2.9 asks that the key material used to issue approvals be isolated from the agent runtime. A session secret the agent never sees, plus a nonce bound to the specific pending transaction, single use, short window, is both of those implemented, and both sit at Level 3. If you write this up anywhere, those are the numbers to cite.

One correction on ASI09, and it runs the other way from what you might expect. The published entry in the Top 10 for Agentic Applications is complete and carries nine mitigations, not only action gates. Immutable logs, behavioural detection, plan-divergence detection, and a plain-language risk summary explicitly not generated by the model are all in there, and item 6 is content provenance: verifiable source identifiers, timestamps and integrity hashes attached to recommendations and external data. The copies in the GitHub repo under agentic-top-10 are unfilled templates, which is likely where the mismatch came from, but the published document is the authoritative one.

Where your point survives is narrower and sharper than the version you stated. Item 6 attaches provenance to recommendations, but its enforcement clause blocks actions lacking trusted provenance. So the metadata reaches the statement while the enforcement never does.

Which is the same shape as the limit on this proposal, and I think you have placed it one step too far out. Generating text invokes nothing, agreed. Publishing it does. The write, the send, the commit is an action, it is on the manifest, and it classifies cleanly as external disclosure. What the gate cannot do is grade it by content. The class of "publish this document" is identical whether the sentence inside is true or false, so admission control bounds whether you publish and never what. The approver is approving an act whose consequence turns on a property no gate evaluates.

That version names what would close it, which is provenance rather than a better classifier, and it is why AGENT-GOVERNANCE.md is the more interesting of the two artefacts you linked. MEASURED, INFERRED and HUMAN FACT is a class system for assertions, and your rule 11 extends it to descriptive prose, which is most of what a published document is made of. Your own closing line states the bridge: out-of-band confirmation and MEASURED labelling are one mechanism applied to two kinds of damage, the irreversible and the false.

One thing worth knowing for where you take this. ASI09 already aligns to T10, Overwhelming the Human in the Loop, which is the threat this issue is filed against. So the slot you are looking for is the one we are both standing in already.

On the CONTRIBUTING link, confirmed, it resolves to the repo root while the file sits in code_samples/. Worth the PR, with the caveat that two fill PRs have been open since early April and the repo has not been pushed to since.

Correction accepted, and thank you for locating the mechanism rather than just the error. I worked from secondary write-ups plus the agentic-top-10 files in this repo, which are unfilled templates. I did not read the published PDF before making the claim. I posted anyway.

I have since read the entry. You are right on all counts — nine mitigations, item 6 is content provenance, ASI09 aligns to T10. The clause that settles it is item 6's own enforcement: "block actions lacking trusted provenance or exceeding the agent's declared scope." Actions. Your reading is the text's.

Your placement is better than mine and I am taking it. Publishing is an action, it is on the manifest, it classifies as external disclosure. The gate cannot grade it by content — "publish this document" has the same class whether the sentence is true or false. Admission control bounds whether, never what.

On 9.2.8 and 9.2.9 I have to be more careful than my earlier description allowed, and the reason is worth stating.

My internal audit of the confirmation channel — dated, before this exchange — concluded the binding was structural, not cryptographic, and explicitly barred me from ever writing "cryptographically bound." The channel compared a session secret and a random nonce; neither committed to the signable bytes. Your mapping to 9.2.8 was made in good faith from my description, and my description was of the older design.

The current one is different, and I will describe it rather than claim the requirement: the approval is an ECDSA signature over the frozen transaction bytes, produced on a hardware device attached to a separate machine. The server recovers the signer against a digest it recomputes from its own frozen fields — never from the bytes it receives — and aborts unless the recovered address matches a pinned one. A corrupted or substituted transport therefore fails closed, rather than producing a coherent forgery. Key material never exists in the agent runtime and the signing tool needs no network access.

9.2.9 that clearly satisfies. For 9.2.8 I can account for two of the four. Action parameters: the signature is over the digest of the server's own serialization, recomputed at verification, so it cannot be detached from them. Single-use nonce: the transaction nonce, inside the signed preimage — not the session nonce I described in my first comment, which is compared in cleartext and sits outside the signed bytes. That one is anti-replay for the channel; it is not what satisfies the requirement, and I should not have led with it. Requester identity and execution context I am not claiming. The address the server recovers is the approver's, not the requesting agent's, and I have no definition to measure the second against.

Which is worth flagging for the standard itself, since you work on it: I searched all eighteen files of 1.0/en for both terms. "Execution context" appears twice — in 9.2.8 itself and in its restatement in the Appendix B inventory. "Requester" appears there plus in 5.2.4, in a different sense. Neither is defined anywhere in that set, and the Appendix A glossary defines "cryptographically bound" only for sender-constrained tokens. I have not read the non-English trees or the research chapters, so a definition may sit outside what I checked; if it does I would like the pointer. If it does not, the gap is load-bearing for the one Level 3 requirement that asks for cryptographic binding: I cannot self-assess against it, and neither can a verifier assessing me.

Your Variant B is the closest thing to a definition I found, and it is not in the standard — it is in your conformance repo. Relabel after signing: the signature still verifies, sha256(canonical evidence) no longer matches the signed digest, the gate fails closed. That is the same shape as mine on a different object — you bind a class into an evidence block, I bind action parameters into transaction bytes, and both verify against something the party being checked does not control. If that scenario is the operative reading of 9.2.8, it would help to say so in C9 itself.

On provenance as the closer, one measurement, because it is the part I did not expect.

Provenance labelling is what my system runs: every assertion carries MEASURED, INFERRED or HUMAN FACT. The register is at nineteen entries — eighteen when I posted above, and this one will make twenty. Seventeen are classified by what caught them, and across those, the count caught by that labelling — or by any rule asking the agent to check itself before asserting — is zero. Eleven were caught by external artefacts, three by me reading, three I could not classify.

The reason looks structural rather than tunable. A self-issued provenance label is a claim by the same process that makes the assertion, evaluated at the same moment, with the same failure rate. It behaves as a trigger: it makes you stop and go measure. It never settles anything.

Which is why item 6's wording is the right shape and my labels are not. Verifiable source identifiers, timestamps, integrity hashes — none of those are issued by the agent. They are checkable against something the agent does not control. The distinction that survived my data is not provenance versus no provenance. It is provenance the agent asserts versus provenance an artefact produces. Only the second has ever caught anything here.

My ASI09 error is an instance of that. The claim carried the confidence of something read. Nothing in the process that produced it was capable of distinguishing that from something actually read, because both would have been issued by the same process.

On the PR — given two fill PRs open since April and no pushes since: is a CONTRIBUTING path fix worth adding to that queue, or is there something on the Q3 code samples work where a review would be more useful than another PR?

Following up on the earlier exchange in this thread - a gap we hit in production last week, offered as a candidate sub-risk for the taxonomy rather than a new category claim.

The gap. ASI01 and ASI06 treat external content as an injection surface: retrieved content is data, validate before persisting. But none of the ten entries treats the integrity of a tool response as its own risk class - specifically the case of an agent that aggregates N external sources and certifies their agreement as a quorum, without ever verifying that the N sources are distinct. The agreement is real; the independence backing it is assumed.

A measured instance. Our curation agent (NEXUS, Base Sepolia testbed) resolves on-chain state through two RPC endpoints run by different operators, and refuses to act unless both agree - quorum 2/2, recorded in an append-only, hash-chained decision registry (the AISVS C9.4.3 pattern). On 2026-08-04 we measured what that quorum was actually made of: both endpoints - different operators, different URLs, sepolia.base.org and publicnode - were running the same execution client at the same build, reth/v2.3.0-9384bc5. Every registry entry was accurate (quorum: 2/2, divergent: false) and still certified less than it appeared to: the field that would have said these two witnesses were the same software did not exist. The device never lied - it attested an independence it had never verified.

What is already known - we borrow it, we don't claim it. Client monoculture as correlated-failure risk is established at Ethereum's consensus layer (clientdiversity.org; the 33/50/66% supermajority thresholds). arXiv:2411.18401 goes further and proves client identity at the validator layer, with zkVMs/TEEs verified on-chain. And arXiv:2606.08021 (Semantic Quorum Assurance) demonstrates the general theorem on LLM validator panels: "Validators are non-deterministic evaluators whose failures may be correlated by model family, retrieval context, prompt template, or shared training data" - its stress test shows the homogeneous-quorum baseline "remains sensitive to correlated model-family failures", and that "Full correlation substantially degrades SQA."

Why this is specifically an agent problem. SQA can enforce diversity at quorum selection because its control plane operates its own validators - the registry it selects from is authoritative, since the declarant is also the operator. An agent consuming third-party infrastructure (RPC providers, oracles, search APIs, model gateways) has no authoritative registry to select from: it does not deploy its witnesses, so it cannot declare their diversity - it can only interrogate them, at decision time. Our fix costs one extra call per source per decision: eth_chainId plus web3_clientVersion on each endpoint, answers recorded in the decision's chained registry entry (independance: {clients: [...], identiques: bool}). In production since the same day: reth/v2.3.0 vs Geth/v10.0.0/drpc, identiques: false.

Two dead signals, measured on the way. On a single-sequencer L2, block-height agreement between endpoints discriminates nothing: Δheight = 0 even across genuinely distinct operators, because one sequencer feeds everyone. DNS/CDN inspection is equally dead: distinct providers commonly resolve into the same edge infrastructure. Client-version fingerprinting was the only cheap signal that discriminated.

Stated limit. web3_clientVersion is self-reported - a malicious endpoint can spoof it. The probe closes the accidental monoculture case (the one nobody chose and nobody was watching), not coordinated adversarial spoofing; that stronger case needs the attestation machinery of arXiv:2411.18401. We believe the accidental case is the common one - and it silently invalidates every quorum an agent certifies while it lasts.

Suggestion. Either an explicit sub-risk under ASI01/ASI06 - unverified source distinctness behind aggregation/quorum logic - or a verification requirement in AISVS C9: when an agent's decision record cites agreement among N external sources, the record must carry evidence of the sources' distinctness, measured at decision time, not assumed from configuration. Happy to share the exact measurements and entry formats if useful.

- Avp9

@Mayur021

Two things, both from reading where the entry text actually comes from rather than from the entry.

1. The mismatch has a specific cause, and it is in this repo.

@Mayur021 traced the ASI09 mismatch to the agentic-top-10 files here being unfilled templates. That generalises further than one entry, and it is worth saying how far.

Looking for the text of the agentic output-validation risk across five sources, I got five mutually incompatible definitions of the same slot — one calls it "Unreliable Output" under a different A1–A10 numbering, one "Unsafe Output Handling" at ASI10, one "Input Manipulation" at ASI07, one ASI07 "Insecure Inter-Agent Communication", one "Output Verification and Hallucination Control" at A7. Every one traces back to either the unfilled templates or a vendor checklist built on top of them.

I nearly published a quotation attributed to OWASP that exists only in a vendor compliance checklist. That is the same failure @avp9-nexus describes above, from the same cause, and it suggests the templates are load-bearing for how this taxonomy is read outside the project.

2. On "admission control bounds whether you publish and never what".

@avp9-nexus is right that no gate grades the sentence inside the document, and right that what would close it is provenance rather than a better classifier. Worth naming that provenance-an-artefact-produces is not limited to the truth of one statement — it generalises to a class of checks that decide pass/fail from artefacts without asking the model anything.

Seven shapes I have found mechanically checkable, each with the property that the wrongness is visible in the artefacts rather than the explanation:

An assertion with no checkable source.
Truncation reported as completion (timeout, dropped stream, token cap — still labelled complete).
Action–claim divergence: the trace does not show what the text says happened.
Declared vs. actual capability.
Silence never accounted for: expected events absent, nothing reported the absence.
An unsurfaced presupposition: a premise nobody asserted and nobody checked.
A dead reference: version, link or artefact cited as live that is not.

Four properties a check needs before it counts as validation rather than review: it does not trust model self-report; it is deterministic; it runs offline for a third party with no account or key; it emits pass/fail rather than a score.

The measurement that makes me think this is the right axis and not just a nicer classifier is @avp9-nexus's own: across the classified errors in that register, the number caught by any rule asking the agent to check itself is zero — eleven by external artefact, three by reading, three unclassified. That is one operator's data and I would not generalise it further than that, but it is the same shape as what I see on a different corpus. It is the argument for artefacts over self-report at admission time as well as at action time.

Written up, with a note on which claims are checked against primary documents and which are not, and with the misattribution above already corrected:
https://github.com/chenhz01/agent-output-validation-gap

Offered as a reference for wording, not as something for this project to adopt.

The template finding generalises further than I had it, and your five-source version is the useful form. I traced one entry; you have the same cause producing five incompatible public definitions of one slot, which makes it a property of how the taxonomy gets read rather than a defect in one file.

On the four properties, three I would take as written. The fourth needs a third value.

Pass/fail is two outcomes, and a check emitting only two has to resolve an unmeasurable case to one of them. Resolving it to fail is noisy and gets the check switched off. Resolving it to pass is the failure the check existed to prevent. Either way the distinction between "I checked and it holds" and "I could not check" disappears, and that distinction is the difference between evidence and an assertion.

Your own item 5 is the case that needs it. If expected events are absent and nothing reported the absence, a check cannot honestly say fail, because absence has several causes with the same footprint, and it must not say pass. The honest value is that it could not be determined, recorded together with what was looked for.

That is not a softer outcome. It is the one that gets rounded away, which is why it has to be a first-class value rather than a note beside a fail. The audit profession has carried it for decades as the disclaimer of opinion, for the same reason.

Worth raising now rather than later, since the repo is wording rather than code at this point and the value set is cheapest to change before anything emits it.

Dating the figure quoted above, since it is ours. The 11 / 3 / 3 split was a hand count over the 17 entries the register held on 30 July, used in an article and repeated here on 1 August. The register itself had no column for what caught each entry until August. It was added after the fact, and the write-up linked from the register says what that is worth: one person classified every row, and that person made the errors.

Where it stands now. The register has been public since September and is regenerated at every engraving. As generated on 10 October it holds 52 rows. By its own class definitions: instrument 33, human 12, third-party 2, undetermined 4, and one row that records a reversed rule rather than a false claim. human there means a reader, the maintainer or a fresh model instance, not only a person.

On the zero. No row has a self-check rule as what caught it, in the sense given here on 1 August: a rule asking the agent to check itself before asserting. That is a classification and not a count read off a column. Where such a rule fired, it triggered a measurement or a reading, and the row credits what decided. And it holds only among the rows we could attribute: the four undetermined cannot be claimed either way.

One row is new and should have been there in August: the ASI09 misreading corrected above in this thread. On 1 August we wrote here that it would become the twentieth entry, and it was never entered. We found that while checking this reply against the register. It is in now, classed third-party, with the delay stated in the row.

On the third value: the figure quoted already carries one, the "three unclassified". In the published register such entries are a class of their own, undetermined, kept in the counts rather than folded into a guess or dropped, and the write-up gives the reason they stay there: the entry names the fault and its fix and never says what surfaced it. It is a smaller case than a check's verdict, an attribution made after the fact, but the choice is the same. Forcing them into another class would have made the split look cleaner and said less.

Register, with the class definitions, the counts and a link to the write-up: https://github.com/avp9-nexus/nexus-art/blob/main/ERRATA.md

Avp9.

Navigation Menu

Repository navigation

Proposal: Add reversibility-graded authority as the HITL gating axis (Playbook 5 / T10) #13

Gap

Proposal

Why it strengthens the doc

Note on terminology

Standards anchors

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Proposal: Add reversibility-graded authority as the HITL gating axis (Playbook 5 / T10) #13

Description

Gap

Proposal

Why it strengthens the doc

Note on terminology

Standards anchors

Activity

Mayur021 commented on Jul 21, 2026

avp9-nexus commented on Jul 31, 2026

Mayur021 commented on Aug 1, 2026

avp9-nexus commented on Aug 1, 2026

avp9-nexus commented on Aug 5, 2026

chenhz01 commented on Sep 28, 2026

Mayur021 commented on Sep 28, 2026

avp9-nexus commented on Oct 10, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions