Skip to content

chore: run the agent git guard in Pi and Hermes - #1175

Merged
mforce merged 6 commits into
mainfrom
chore/agent-guard-pi-hermes
Oct 10, 2026
Merged

mforce merged 6 commits into
mainfrom
chore/agent-guard-pi-hermes

Conversation

@mforce

@mforce mforce commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Closes #1171

tools/agent-guard/git_guard.py (#1158) now also runs in Pi and Hermes, the two harnesses it did not reach. Each one gets a small adapter in a file this repo carries. The adapter hands every shell tool call to the guard and blocks what the guard refuses.

Harness Adapter Payload sent to git_guard.py Blocks with
Pi 1.0.4 .pi/extensions/git-guard.ts, a tool_call handler for bash and powershell {"tool_name": "bash" or "powershell", "tool_input": <Pi's tool input>, "cwd": ctx.cwd} { block: true, reason: <guard stderr> }
Hermes 0.21.5 .hermes/plugins/git-guard/, a pre_tool_call plugin hook for terminal, process_manage and execute_code Hermes's shell-hook shape: {"hook_event_name": "pre_tool_call", "tool_name": …, "tool_input": <args>, "cwd": <terminal cwd>, "unchecked"?: <reason>} {"action": "block", "message": <guard stderr>}

git_guard.py now treats Bash (Claude Code, Codex), bash (Pi) and terminal (Hermes) as shell tools.

  • Hermes terminal. The plugin resolves the directory the way terminal_tool does on the local backend. An absolute workdir is used as is. Otherwise the execution session's record wins (keyed by get_current_session_key(default="") or task_id), then the planned default: task override, task record, then terminal.cwd/TERMINAL_CWD. The Hermes process directory is never used. Git writes refuse, and other commands run, in four cases where the repository can't be named: a non-local backend, a relative workdir (it resolves against the cached environment's directory, which the hook can't see), a session contextvar and task id whose recorded directories disagree, and terminal settings the plugin can't read.
  • Hermes process_manage write/submit. The input goes to a running shell, so it gets the normal shell check with no known directory. A merge refuses by its own rule, a branch-dependent write refuses unless it starts with cd /absolute/path &&, and absent data is Enter alone, as in Hermes. Other actions carry no data and pass.
  • Hermes execute_code and Pi powershell. Both are refused outright, by the maintainer's decision, because the guard cannot check Python or PowerShell. The message points the agent to terminal or bash.

Sources

  • Pi. docs/extensions.md in @earendil-works/pi-coding-agent 1.0.4: "tool_call can mutate input or block execution"; "A tool_call handler failure blocks the tool as a fail-safe." examples/extensions/permission-gate.ts reads event.toolName === "bash" and event.input.command. dist/core/tools/bash.d.ts defines the bash input {command, timeout?}, and ExtensionContext.cwd is the working directory. docs/configuration.md: project extensions live in .pi/extensions/ and load "after project trust is granted".
  • Hermes. website/docs/user-guide/features/hooks.md (pre_tool_call): the callback receives tool_name ("terminal" for shell), args and task_id, and return {"action": "block", "message": ...} "short-circuits the tool with that text as the error returned to the model". A raising or timed-out callback fails closed. tools/terminal_tool.py: the command runs in args["workdir"], else the session cwd (get_session_cwd(task_id)), else config. hermes_cli/plugins_discovery.py: .hermes/plugins/ is scanned only with HERMES_ENABLE_PROJECT_PLUGINS, and a non-bundled plugin loads only when listed in plugins.enabled.

One-time enable steps (also in AGENTS.md, next to the guard line)

  • Pi: trust the project when Pi asks, or pass pi --approve for one run. Until then Pi does not load .pi/extensions/.
  • Hermes: add git-guard to the plugins.enabled list in your Hermes config.yaml (keep the other entries, and drop any git-guard entry under plugins.disabled). Then start Hermes from the repository root with HERMES_ENABLE_PROJECT_PLUGINS=true. hermes plugins enable git-guard does not work: in Hermes 0.21.5 its name lookup (hermes_cli/plugins_cmd.py) never scans project plugins.

Notes for review

  • process_manage input is checked one write at a time. A command split across several writes (gh pr me, then rge 1175) passes, and so does echo "git commit", which is harmless. The running shell also keeps state between writes: an earlier export GIT_DIR=…, alias or function changes what a later cd /abs && git commit does, and the guard does not see it. Hermes's other command routes, terminal(background=True) at launch and hermes_tools.terminal() calls, go through terminal (review round 1 traced them in the installed source).
  • No agent can use Hermes execute_code or Pi powershell in this repository while the adapters are loaded.
  • The guard's other limits carry over (aliases, a merge query read from a file, and the rest listed in AGENTS.md). chore: add git hooks that refuse commits on main and unsafe pushes #1172's git hooks remain the backstop under every harness.
  • chore: add git hooks that refuse commits on main and unsafe pushes #1172 conflicts in AGENTS.md. It adds a line saying Pi and Hermes "carry no guard from this repo yet". Whichever PR merges second should drop that clause when it merges main.
  • CI. git_guard_test.py (run in classifier-self-test) now feeds every row in all four payload shapes and tests the Hermes plugin callback. pi_extension_test.mjs runs in the web job, which already sets up Node 26; Node 26 strips the extension's types natively.

Tests run locally

  • python3 tools/agent-guard/git_guard_test.py: 161 commands × 4 payload shapes (Claude, Codex, Pi, Hermes): 164 allow, 480 block. Thirteen tests in all. Besides the table they cover:
    • execute_code and powershell refused;
    • process_manage input: a merge, a push to HEAD:main and a bare commit refuse; an absolute cd with a commit, ls, an absent and an empty data, and kill pass; a relative cd refuses;
    • the plugin callback for each tool;
    • the first call with terminal.cwd on main;
    • both round 2 sequences: an override on a feature branch with the session moved to main, and a relative workdir after the environment moved;
    • disagreeing session records;
    • a docker backend and unreadable terminal settings.
  • node --test tools/agent-guard/pi_extension_test.mjs: a merge and a commit on main block, git status passes, powershell is refused, and other tools pass.
  • dotnet test filtered to TenancyDocsFreshnessTests, RenameAccountDocsTests and FileSizeLimitTests (20 passed), plus the ImagePin and CommittedSchemaDocs_CarryNoEnvironmentSpecificContent tracked-file guards (3 passed).

Proof at f52de5f5

Mutation check on review round 2's checks

Each check was disabled on its own and both tests were run (red expected), then the file was restored and the tests re-run (green expected).

File Check disabled Tests with the mutation After revert
git_guard.py execute_code refused red (2 failing) green
git_guard.py powershell refused red (2 failing) green
git_guard.py process_manage input checked red (2 failing) green
git_guard.py absent data is Enter alone red (2 failing) green
git_guard.py unchecked terminal calls refuse writes red (2 failing) green
git_guard.py relative cd in process input has no directory red (1 failing) green
git_guard.py no directory refuses before git runs red (1 failing) green
hermes plugin plugin routes execute_code red (1 failing) green
hermes plugin plugin routes process_manage red (1 failing) green
hermes plugin non-local backend unchecked red (1 failing) green
hermes plugin relative workdir unchecked red (1 failing) green
hermes plugin session contextvar key used red (1 failing) green
hermes plugin disagreeing records unchecked red (1 failing) green
hermes plugin session record over task override red (1 failing) green
hermes plugin configured terminal.cwd on the first call red (1 failing) green
hermes plugin unreadable terminal settings unchecked red (1 failing) green
git-guard.ts powershell sent to the guard red (1 failing) green

Live runs

Both harnesses ran live in this worktree against the vLLM model glm53-flash-exl3-4bpw-tp2-c5-ctx262144 (https://iori-spark.mforcelabs.com/v1, probed first). The blocked command is a --dry-run push to main, so even a guard failure would have changed nothing. No global configuration changed.

Pi: pi --mode json -p --approve --no-session --provider vllm --model <id> "<prompt>". --approve trusts the project for that run only. These are the tool events from the JSON stream (~/.cache/gh-hooks/pi_live.py):

### pi, prompt: run `git push --dry-run origin HEAD:main` (exit 0)
tool call: bash {"command": "git push --dry-run origin HEAD:main"}
isError: True | result: git-guard: `git push --dry-run origin HEAD:main` is blocked: a src:dst push can land one branch's commits on another branch (#900). Run `git branch --show-current`, then `git push -u origin <that-branch>` (or plain `git push`).

### pi, prompt: run `git status --short` (exit 0)
tool call: bash {"command": "git status --short"}
isError: False | result: (no output)

Hermes: HERMES_HOME=<temporary dir> HERMES_ENABLE_PROJECT_PLUGINS=true hermes chat -q "<prompt>". The temporary home held only a three-key config.yaml: the vLLM endpoint as a custom provider, local terminal, and plugins.enabled: [git-guard]. Afterwards ~/.hermes/config.yaml had the same mtime, and the temporary home was deleted (~/.cache/gh-hooks/hermes_live.py). Each box is the model's reply quoting the tool result; Hermes renders Markdown, so the backticks are gone:

### hermes, prompt: run `git push --dry-run origin HEAD:main` (exit 0)
╭─ ☤ Hermes ───────────────────────────────────────────────────────────────────╮
The terminal call was blocked by the repo's git guard. Its exact output:


git-guard: git push --dry-run origin HEAD:main is blocked: a src:dst push can land one branch's commits on another branch (#900). Run git branch --show-current, then git push -u origin <that-branch> (or plain git push).


No push was attempted; the guard refused the command before git ran.
╰──────────────────────────────────────────────────────────────────────────────╯

### hermes, prompt: run `git status --short` (exit 0)
  ┊ 💻 $         git status --short  0.2s
╭─ ☤ Hermes ───────────────────────────────────────────────────────────────────╮
Tool output was empty (exit code 0) — the worktree is clean, nothing to report.
╰──────────────────────────────────────────────────────────────────────────────╯

A Pi project extension and a Hermes project plugin feed each shell tool call to tools/agent-guard/git_guard.py and block what it refuses. git_guard.py accepts Pi's bash and Hermes's terminal payloads, including Hermes's per-command workdir. AGENTS.md names each harness's one-time enable step.
@mforce mforce added this to the Platform hardening milestone Oct 10, 2026
@mforce

mforce commented Oct 10, 2026

Copy link
Copy Markdown
Owner Author

Review round 1 (Codex gpt-6-astra, head 4ce5488f): 3 realistic findings, 1 doc finding

The reviewer checked against the installed Pi 1.0.4 and Hermes ee8dd6c8 source and cited file and line for each claim. Both hook contracts are used correctly, and both adapters block when the checker fails, crashes, times out or cannot start.

  1. P1: Hermes runs commands through tools the plugin skips.

    • execute_code can call subprocess.run(["gh","pr","merge",…]) directly. Hermes's own approval passes it.
    • terminal(command="bash", background=True) followed by process_manage(action="submit", data="gh pr merge …") delivers the command as process input.

    Both reached the hook and got no block.

  2. P1: Hermes's first terminal call can check the wrong checkout. With no session cwd yet, the adapter uses os.getcwd(), but Hermes runs the command in its configured TERMINAL_CWD. Reproduced: process cwd on a feature branch, TERMINAL_CWD on main, and git commit -m x was approved. Fix: use Hermes's own cwd resolution, and refuse when the backend is remote or a container, because the repo state cannot be checked from the host.

  3. P2: Pi's built-in powershell tool (Windows only) skips the guard, and the checker returns 0 for that tool name. Fix: refuse PowerShell calls that mention a git write or gh merge. Do not treat PowerShell as Bash.

  4. Doc: hermes plugins enable git-guard fails with "No plugin named 'git-guard'": the CLI's discovery never scans project plugins, though runtime discovery does. Document the working setup: add git-guard to plugins.enabled and start Hermes with HERMES_ENABLE_PROJECT_PLUGINS=true from the repo root.

Tests pass: the table (161 commands × 4 shapes) and the Pi extension test.

…wershell in the git guard

Hermes execute_code code and process_manage write/submit input, and Pi powershell commands, are screened as text: refused when they mention or could spell a git write or PR merge, never parsed as Bash. The Hermes plugin takes the terminal's directory from Hermes's own precedence (task override, session record, terminal.cwd/TERMINAL_CWD) and refuses git writes on a non-local backend or when that lookup fails.
hermes plugins enable cannot see project plugins in Hermes 0.21.5; add git-guard to plugins.enabled and start Hermes from the repo root with HERMES_ENABLE_PROJECT_PLUGINS=true.
@mforce

mforce commented Oct 10, 2026

Copy link
Copy Markdown
Owner Author

Review round 1 on head 4ce5488f: 3 realistic findings, 1 doc finding (summary)

Fixed in 79d47a0a (code and tests) and dce7144e (AGENTS.md). Head is now dce7144e.

  1. Hermes execute_code and process_manage (P1). The plugin now routes both tools to the guard. execute_code code, and process_manage data for write/submit, are screened as text: refused when they mention, or could spell, a git write or PR merge, never parsed as Bash. Other process_manage actions pass. New cases: the review's subprocess.run(["gh", "pr", "merge", "1175"]) script and submit with gh pr merge 1175 both block through the guard and through the plugin callback; print(sum([1, 2])), submit ls and kill pass. Text screening is per call, so input split across several writes passes. That is listed as a limit in the body.
  2. Hermes first-call cwd (P1). The plugin takes the terminal's directory from Hermes's own tools.terminal_tool: task override, then session record, then terminal.cwd/TERMINAL_CWD from _get_env_config(). It never uses os.getcwd(). On a backend other than local, or when that module cannot answer, the plugin sends the backend along and git writes refuse. New cases: the process runs in a feature checkout and terminal.cwd points at main, and the first git commit -m x blocks. With a session record on the feature branch it passes. A docker backend, or no terminal module, refuses git commit and passes ls -la.
  3. Pi powershell (P2). The extension sends powershell calls to the guard, which screens them as text and does not add them to the Bash-parsed set. The assertion that treated powershell as a read-only tool is gone. New cases: gh pr merge 1175 and git -C $env:REPO push block; Get-ChildItem passes, in both the guard table and the Pi extension test.
  4. Hermes enable step (doc). AGENTS.md and the plugin header now say: add git-guard to plugins.enabled in the Hermes config.yaml (in 0.21.5, hermes plugins enable cannot see project plugins), then start Hermes from the repo root with HERMES_ENABLE_PROJECT_PLUGINS=true.

Mutation check: 12 mutations across git_guard.py, the Hermes plugin and the Pi extension. All turned the tests red and went green after revert; the table is in the body.

  • The "text tools screened" mutation first stayed green, because the Bash parser's crash path also blocked those rows. The text-tool rows now assert the text-only message.
  • One plugin mutation first failed for the wrong reason (os is no longer imported). Re-run with a real getcwd(), it is still red.

Live runs re-done at dce7144e with the vLLM model, in the body: Pi's bash and Hermes's terminal both block a --dry-run push to main and run git status --short. ~/.hermes is unchanged, and the temporary home is deleted.

@mforce

mforce commented Oct 10, 2026

Copy link
Copy Markdown
Owner Author

Review round 2 (Codex gpt-6-astra, head dce7144e): 5 realistic findings

Round 1's repros are closed. Refusal on docker, ssh and modal backends works.

  1. P1: Hermes checks the wrong checkout after the session moves. After terminal("cd <main checkout>") under a task cwd override (ACP sessions register one), the hook still checks the override, while Hermes runs in the session's recorded cwd. A relative workdir after an earlier explicit-workdir call, and a session key different from task_id, also reproduced. Fix: resolve the final execution cwd the way terminal_tool does, including the session key, and refuse a relative workdir.
  2. P1: Python can build a git command the text screen never sees. action = "commit"; subprocess.run(f"git {action} -m test", shell=True) passes. A text screen over Python cannot match real interception.
  3. P2: PowerShell's quoted absolute paths pass (& 'C:\Program Files\Git\cmd\git.exe' commit), and so does GIT.EXE. Backslash stripping and case sensitivity defeat the screen. This tool exists on Windows only.
  4. P2: the text screen refuses ordinary reads. subprocess.check_output(["git","status"]), print(["commit"]), echo "git commit" and Write-Output "git commit" are all blocked.
  5. P2: a process_manage submit with data omitted (a bare Enter) is refused. Hermes defaults it to "".

Whether to keep text screening of execute_code or refuse that tool in this repo is a design decision; it goes to the maintainer.

…rminal cwd in the git guard

Hermes execute_code and Pi powershell are refused outright while the adapters are loaded; agents use terminal or bash. Hermes process_manage write/submit input gets the normal shell check with no known directory, and absent data is Enter alone. The plugin resolves the terminal's directory as terminal_tool does on the local backend: an absolute workdir, else the execution session's record, else the planned default. A relative workdir, disagreeing session records, a non-local backend or unreadable settings refuse git writes.
@mforce

mforce commented Oct 10, 2026

Copy link
Copy Markdown
Owner Author

Review round 2 on head dce7144e: 5 realistic findings (summary)

Fixed per the maintainer's decision in 71a38db6 (code and tests) and f52de5f5 (AGENTS.md). Head is now f52de5f5.

  • execute_code (findings 2, 4). Refused outright while the plugin is loaded. The message tells the agent to use terminal. The Python text screen is deleted.
  • powershell (findings 3, 4). Refused outright in the guard; the Pi extension still routes it there. The message tells the agent to use bash. The PowerShell text screen is deleted.
  • process_manage write/submit (finding 5). The input now gets the normal Bash guard, with no known directory, because the running shell's directory is unknown:
    • gh pr merge 1175 and a write carrying git push origin HEAD:main refuse by their own rules.
    • A bare git commit refuses ("cannot find the directory it runs in"), while cd /absolute/feature/checkout && git commit passes.
    • Absent data defaults to "" as in Hermes, so submit with no data and submit with empty data both pass.
    • echo "git commit" passes.
    • Limits, listed in the body: input split across writes, and state the persistent shell carries between writes (export GIT_DIR, aliases).
  • Hermes cwd after the session moves (finding 1). The plugin follows terminal_tool's final resolution on the local backend:
    • An absolute workdir is used as is.
    • Otherwise the execution session's record (keyed by get_current_session_key(default="") or task_id) wins over the planned default: task override, task record, terminal.cwd.
    • Two inputs can't be resolved reliably from the hook, so git writes refuse there:
      • a relative workdir, which resolves against the cached environment's directory;
      • a session contextvar and task id with different records, because which one the terminal worker uses depends on its thread.
    • Both sequences from the review are regression cases:
      • an ACP override on a feature branch, then cd to main: git commit blocks;
      • the session on a feature branch with the environment moved to main, then a relative workdir: git commit blocks and ls passes.

Mutation check: 17 mutations across the guard, the Hermes plugin and the Pi extension (refusals, process_manage routing and the data default, the unknown-directory paths, every step of the cwd resolution, the unchecked reasons, powershell routing). All turned the tests red and went green after revert. Three rows assert a specific message, because another check would also block them. The table is in the body.

Live runs re-done at f52de5f5 with the vLLM model: Pi's bash and Hermes's terminal both block a --dry-run push to main and run git status --short. ~/.hermes is unchanged.

@mforce

mforce commented Oct 10, 2026

Copy link
Copy Markdown
Owner Author

Round 2 fixes verified at f52de5f5 (coordinator check; no round 3)

  • python3 tools/agent-guard/git_guard_test.py exits 0 (161 commands × 4 payload shapes). node --test tools/agent-guard/pi_extension_test.mjs passes.
  • Payloads fed straight to the guard:
    • execute_code (even print(1)) → 2, refused as the maintainer decided;
    • powershell (even git status) → 2;
    • process_manage submit gh pr merge 1 → 2;
    • submit echo "git commit" → 0 (no false block);
    • submit with data omitted → 0 (an empty Enter).
  • The two Hermes cwd sequences from round 2 are regression cases, and B's mutation pass covers them (17/17 red).

The review loop stops here: rounds 1 and 2 confirmed defects, and every one is fixed, refused by the maintainer's decision, or documented as a limit (input split across process_manage writes, shell state between writes).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Git hooks backstop for agent git guardrails (every harness)

1 participant