Skip to content

feat(skills): gated skills — the approval comes before the agent sees the request (abilityai/trinity-enterprise#751) - #3208

Merged
vybe merged 4 commits into
devfrom
feature/ent751-gated-skills
Oct 4, 2026
Merged

vybe merged 4 commits into
devfrom
feature/ent751-gated-skills

Conversation

@webmixgamer

Copy link
Copy Markdown
Contributor

Summary

A skill marked "requires approval by role X" is no longer dispatched when a request invokes it as a /<name> token. The platform freezes the request, raises an approval for that role, and answers the requester at once. Approve runs it once, exactly as written. Reject, expiry or cancel runs nothing. A proven person who fills the approver role runs their own request directly, and that is audited.

Ships inert: skill_gate_service.list_skill_gates returns {} until the gate map lands (trinity-enterprise#753).

How it works

  • Where it's checked: skill_gate_service.enforce runs at the /chat and /task admission seams, plus a backstop at the top of execute_task for every other producer (scheduler, loops, fan-out, channels, the Workspace, rooms, sessions, A2A, public links, paid calls, operator resumes, validation). The voice tool refuses instead. The gate reads everything a requester put in front of the executor: message, user_message, system_prompt, the quoted reply, the chosen answer, and the rendered loop iteration. It never reads the platform's own framing.
  • The record: skill_gate_requests on both tracks: the SQLite runner, and Alembic 0088_skill_gate_requests, re-parented on 0087_pull_sync (one head). Both agent columns cascade on delete.
    • Each approval dispatches exactly once, via a compare-and-set claim and a UNIQUE run id.
    • Skill fingerprints are re-checked at approval; a changed skill is stale and runs nothing.
    • A sweep reconciles what a dead process leaves behind, and never re-runs anything.
  • Who decides: may_end in the ask sink. Only an addressee can answer; an admin may cancel but not approve. Gate rows are hidden from machine-key reads (test: gate dev → open-PR merges on the unit-test suite #715), and agents learn the outcome, never who decided.
  • Requesters:
    • A Workspace caller is a person, proven by its route.
    • Each schedule is its own requester, read from the pre-created row, and Run now is the person who pressed it.
    • An approved result returns to the thread or channel it came from.
    • A person is not told about a decision they made themselves.
  • Surfaces:
    • HTTP: 202 pending_approval, or a named refusal with X-Trinity-Error-Code.
    • MCP: structured pending_approval / refused results.
    • The Workspace answers a held turn with the notice as a normal reply.
    • The Operations card and /m show a not_addressee refusal next to the controls.

Requirement: docs/memory/requirements/security.md §26.13 (OPS-001-GATE). Flow: docs/memory/feature-flows/skill-gate.md.

Review trail

  • /review: four ways around the gate, each a field the requester controls that the gate did not read. All fixed.
  • /cso --diff: one HIGH (a system_prompt invocation was not scanned) and one MEDIUM (person emails reached agents), both fixed. Report: docs/security-reports/cso-diff-2026-10-02-ent751-gated-skills.{md,json}.
  • Localhost eyeball with two accounts: seven issues, all fixed with tests that failed first:
    • the Workspace was treated as an anonymous channel;
    • a waiting request showed as Failed;
    • approved results did not come back to the thread;
    • a role id leaked into the notice;
    • schedules shared one requester budget;
    • people were notified of their own decisions;
    • a refusal was hidden in a refresh banner.
  • Independent review of the post-eyeball changes: two LOW findings fixed (schedule names could forge lines on the card; /m invited a retry that can never succeed). The rest are stated limits or follow-ups.

Tests

  • New tests: tests/unit/test_ent751_*.py (10 files), plus additions to the ent#329, ent#611, ent#551 and refactor(db): reduce database.py facade signature width — param objects for execution/chat writers #1482 suites; src/frontend/tests/unit/queueCardNotAddressee.spec.js and mobileAdminNotAddressee.spec.js; src/mcp-server/src/chat-gate.test.ts.
  • Backend unit suite on the merged tree: 20,800 passed. The remaining failures are local only:
    • one order-dependent test in an untouched module, which passes alone;
    • the runtime route census, which can't import main.py without the OpenTelemetry exporter package.
  • Frontend unit: 4,546 passed. MCP: chat-gate and chat-depth, 26 passed.

Not in this PR

  • trinity-enterprise#753 (gate storage, UI and API). UI handling of the 202 answer has to land with it.
  • trinity-enterprise#754 (the Skills tab "Requires approval" row, and a marker when an approver's own request skips approval).
  • trinity-enterprise#755 (the approval card, including hiding the decision buttons from non-addressees).
  • trinity-enterprise#752 (the in-container hook for skills named only in prose).

Fixes abilityai/trinity-enterprise#751

🤖 Generated with Claude Code

webmixgamer and others added 3 commits October 3, 2026 13:06
… the request (Abilityai/trinity-enterprise#751)

A skill marked "requires approval by role X" is no longer dispatched when a
request invokes it as a /<name> token. The platform freezes the request,
raises an approval through the ask sink, and answers the requester at once;
Approve runs it once, exactly as written; reject, expiry or cancel runs
nothing. A proven person who fills the approver role runs their own request
directly (audited). Inert until the gate map lands (trinity-enterprise#753):
skill_gate_service.list_skill_gates returns {}.

- The check: one helper (skill_gate_service.enforce) at the /chat and /task
  admission seams and a backstop at the top of execute_task for every other
  producer (scheduler, loops, fan-out, channels, the Workspace, rooms,
  sessions, A2A, public links, paid calls, operator resumes, validation);
  the voice tool refuses. It reads everything a requester put in front of
  the executor (message, user_message, system_prompt, the quoted reply, the
  chosen answer, the rendered loop iteration), never platform framing.
- The record: skill_gate_requests on both migration tracks (SQLite runner +
  Alembic 0087), CASCADE in AGENT_REFS. Exactly once via a compare-and-set
  claim; fingerprints re-checked at approval (stale runs nothing); a sweep
  reconciles what a dead process leaves (unknown, never re-run).
- Who decides: may_end in the ask sink, so only an addressee answers and an
  admin may cancel, never approve. Gate rows are hidden from machine reads
  (#715); agents learn outcomes, never who decided.
- Requesters: the Workspace caller is a person (its route proves it), each
  schedule is its own requester (from the pre-created row), Run now is the
  person who pressed it. Approved results return to the thread or channel
  they came from; a person is not told of a decision they made themselves.
- Surfaces: 202 pending_approval / named refusals with X-Trinity-Error-Code;
  MCP structured results; the Workspace answers a held turn with the notice;
  the Operations card and /m show a not_addressee refusal beside the controls.

Reviewed: /review, /cso --diff (1 HIGH + 1 MEDIUM fixed), a localhost
eyeball with two accounts (7 issues fixed), and an independent review of the
eyeball delta (2 LOW fixed, the rest registered as debt).

Tests: tests/unit/test_ent751_*.py and touched neighbours; full backend unit
suite 20,728 passed; frontend 4,546 passed; MCP chat-gate 15 passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dev added Alembic 0087_pull_sync (trinity-enterprise#703) on the same parent
as this branch's revision, which would have left two heads (and an
`upgrade head` that applies nothing). The skill-gate revision is re-parented
as 0088_skill_gate_requests on top of 0087_pull_sync; the SQLite runner keeps
both migrations, pull_sync first. check_alembic_heads: 1 head.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…low follows the template (Abilityai/trinity-enterprise#751)

- The three gate caps were read from env vars that neither compose file
  wires, so the levers would do nothing on deploy (the #1056 packaging
  class). They are constants with the values the plan gate ruled (10
  pending per requester per agent, 50 per agent, 10 per minute per
  requester); the tests already patch the module attributes.
- docs/memory/feature-flows/skill-gate.md restructured into the required
  sections (Frontend Layer, Backend Layer, Side Effects, Error Handling,
  Security Considerations, Testing with steps/edge cases/status, Related
  Flows) with file:line references. Content unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown

⚠️ Nightly unit-suite check skipped — merge conflict against dev.

Resolve by running git merge dev locally and pushing the result. The next nightly run will re-test once the conflict is gone.

…3210 rooms)

- dev moved the x402 verify → dedup → execute → settle lifecycle into
  services/paid_turn_service.run_paid_turn, shared by the paid chat door and
  the new A2A payment gate. A gated turn raised inside it would have been an
  EXECUTION_ERROR (500 "Task execution failed") with an approval already
  waiting. It is now its own outcome, GATE_HELD (202) / GATE_REFUSED (the
  gate's status): nothing ran, nothing is charged, the claim is released with
  fail() — never complete(), whose unsettled snapshot would make a retry try to
  settle. The x402 door renders it like its other answers; the paid A2A door
  renders input-required / rejected through a2a_payment_gate._OUTCOME_RENDER
  (dev's guard: every outcome kind is a refusal or a rendered task state).
- Rooms (#3210): the agent is shown a budget-fitted delta; the gate reads
  that same `shown` set.
- Guards: the paid `_execute` closure and `_run_a2a_paid_execution` are named
  unwrapped producers (each sends the caller's text as-is). Test harnesses
  follow dev's auth and pricing changes to the A2A door.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@webmixgamer
webmixgamer requested review from dolho and vybe October 4, 2026 13:27
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown

⚠️ Live-instance suite skipped — merge conflict against dev.

Resolve by merging dev locally and pushing the result; the next nightly re-tests.

@vybe

vybe commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

merge-train (2026-10-04): on today's train, merging as-is. Validated at 14a4a50a with /validate-pr, /review and /cso --diff: no critical or high finding. Both findings from the 10-02 /cso report hold as fixed.

What made it READY:

  • Coverage is behavioural. Neutering either admission seam, the execute_task backstop, the claim's state == pending predicate, or unregistering the SQLite migration each turns tests red.
  • The inert path is a no-op. With list_skill_gates returning {}, enforce() and the backstop touch no DB and cannot raise.
  • Both migration tracks agree on all 22 columns and three indexes; one Alembic head (0088 on 0087_pull_sync).
  • Auth is only narrowed. require_admin, assert_admin and ADMIN_GATE_SCOPES are untouched; agent keys cannot approve, cancel or self-approve.

To close before trinity-enterprise#753 supplies a gate map. None of these is reachable while the map is empty.

  1. Schedule name is not scanned. The backstop reads only message (task_execution_service.py:1926), but the schedule name goes into the executor's system prompt (platform_prompt_service.py:740). A schedule named /<gated-skill> … with a neutral message dispatches every tick unapproved. source_mcp_key_name is rendered the same way (:726); we did not verify whether a key name is free text.
  2. Telegram quoted reply is not scanned. message_router.py:870 passes only message.text, while the prompt carries the quote (:159). Replying "@bot do this" to an untagged /<gated-skill> message dispatches with the command in the quote. client_portal/service.py:3211 fixes exactly this for the Workspace. Group history (:587-589) is the same door and needs your call.
  3. Three wiring lines are unpinned. Each can be deleted with the suite green:
    • dependencies.py:704 (is_event_loopback=bool(loopback)); without it the event loopback is a person and can self-approve.
    • main.py:595 (register_ending_observer()) and operator_queue_service.py:1231 (await skill_gate_service.sweep()); without them an approval is recorded and nothing runs.
  4. An approved /task run replays the concatenation. The seam freezes message + user_message + system_prompt (chat_execution_service.py:2130) and _dispatch_approved sends that as message (skill_gate_service.py:925). The command can appear twice and a system prompt arrives as user text.
  5. The inline connector tier has no gate result. inlineConnectorChat (client.ts:2946-2965) does not call readGateResult, so a refusal loses its named code.
  6. Matcher gaps (utils/skill_invocation.py:25,31). Not matched: -/name, _/name_, 1/name, and a soft hyphen (U+00AD), U+180E, U+034F, a variation selector or a tag character after the slash.
  7. Approver email reaches the requester. _notify (skill_gate_service.py:1064-1075) names the decider to a person requester, which includes an external Workspace client.

Smaller, no urgency:

  • Connector test. run_playbook now sends /<name> <input>, and that is live at merge. connector.test.ts:55-62 asserts only /cso/ and /scan repo/, which the old prose form also satisfied; assert.equal(chats[0].message, "/cso scan repo") would pin it.
  • Hot-path read. The seams build requester_from_principal(...) before enforce can short-circuit, which costs one execution-row read on every agent→agent /chat and /task. Building it after read_gates removes that.
  • Write-only columns. requester_mcp_key_id, state_detail and dispatched_at are written and never read.
  • Catalogs. architecture/database.md, backend.md, mcp-server.md and security.md have no skill_gate rows.

One item is the owner's decision rather than a fix, and is already a stated limit in the docs: a user-scoped MCP key counts as its owner and can self-approve.

@vybe vybe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge-train: validated on 2026-10-04 at 14a4a50 (/validate-pr, /review, /cso --diff). No critical or high finding; ships inert. Follow-ups to close before trinity-enterprise#753 are in the merge-train comment.

@vybe
vybe merged commit c508a63 into dev Oct 4, 2026
29 checks passed
@webmixgamer
webmixgamer deleted the feature/ent751-gated-skills branch October 5, 2026 09:17
vybe pushed a commit that referenced this pull request Oct 7, 2026
…lands (#3274) (#3345)

* fix(skill-gate): close the #3208 validation gaps before the gate map lands (#3274)

The gated-skill check (#3208) is inert until the gate map supplies a row.
The #3208 merge-train validation listed seven gaps that a row makes
reachable; this closes them, plus #3233 and the UI's handling of the 202.
It lands before or together with the gate map, never after it.

- Context scan: a schedule's name, an MCP key's name and the requester's
  email reach the executor's system prompt beside the request. The gate
  now scans them (raw and as the prompt renders them), apart from the
  request, shows them on the card when they alone matched, and never
  replays them. A parity test classifies every ExecutionContext string
  field.
- Telegram: a group reply is scanned with the quote line it carries.
  Group history, sender labels and the group title stay unscanned
  (decided; stated limit, naming the runtimes without the hook).
- Wiring: the event-loopback flag, the ending-observer registration and
  the sweep call are each pinned by a test that runs the line.
- Replay: an approved /task or fan-out run is sent the caller's message
  and system prompt apart, never their concatenation.
- MCP: the inline connector tier reads the gate result like the other
  tiers; a pending answer promises delivery only when the backend makes
  one (outcome_delivery), and the delegation contract defers to the
  message (#3233).
- Matcher: -/x, _/x_ and 1/x match; every Default_Ignorable character
  is matched removed (a run before a slash reads as a space) and spaced.
- Notices: the requester never sees the approver's email; a platform
  request reads the display name, everyone else "the agent's approver".
  A requester nobody notifies is told so and where the outcome shows.
- UI: the Chat tab, /m, the public link, the Tasks tab, Playbooks and
  Dashboard show the server's message instead of an error and poll
  nothing; held chat lines stay out of the next turn's history.

Mutation: each call site reverted from a copy turns its test red — 21
backend sites (test_3274_*, test_ent751_skill_invocation), 15 UI
sender branches (skillGateSenders / mobileAdminSkillGate), 2 MCP lines
(chat-gate.test.ts).

Fixes #3274
Fixes #3233

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(skill-gate): give each non-BMP invisible range its own class (CodeQL py/overly-large-range)

CodeQL reads a character range beyond the Basic Multilingual Plane as
`�-�`, so the three such ranges sharing one class in
`_INVISIBLE` (U+1BCA0, U+1D173, U+E0000 blocks) were flagged as
overlapping (alerts 385, 386). Each now sits in a class of its own,
joined by alternation. Same characters either way: 0 of 1,111,998
non-surrogate code points match differently (4,174 invisible before and
after), and the before-slash rule's output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(skill-gate): review fixes — the card shows exactly what runs, a linear matcher, honest refusals (#3274)

From the full /review of PR #3345 (two independent reviewers):

- Card vs run (critical): the card sanitised the JOIN of the message and
  the caller's system prompt, while the approved run sent each sanitised
  alone. A credential pattern spanning the line between them redacted
  the system prompt from the card only, so Approve ran an instruction
  the approver never saw. The card is now built from the exact replay
  parts, with the system prompt labelled ("With these instructions as
  its system prompt:").
- Matcher: the before-slash rule was quadratic on a long run of
  invisible characters (50k took ~10 s on the event loop, reachable
  from a public-link message); a run is now matched only from its
  start. `_AFTER` regressed `/x...y` and `/x._y` against #751; the dot
  rule is #751's again, and a property test holds that every input the
  #751 matcher caught is still caught.
- /m: a refused gated request is kept out of the next message's history
  too; the notice gets its own gray-400 colour (theme token) and
  role="status".
- Playbooks: a gate refusal goes to the agent page's error toast, which
  stays until dismissed. The agent page toast carries role status/alert.
- Public link: the status route returns `gate` (held | refused); a held
  turn shows the grey notice, a refusal the red error box.
- Tests: the Workspace decider-label test now reaches its own arm;
  refusal assertions reject the JSON-rendered detail; the public-link
  spec fails on an assertion, not a timeout.

Mutation: each fix reverted from a copy turns its test red (7/7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: trinity-ability <309458136+trinity-ability@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants