Repository navigation
feat: implement issue #677 — [Phase 1] Thread an optional MCP config into the Claude agentic review tiers (engine.sh) #713
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
24accd1
feat: implement issue #677 — [Phase 1] Thread an optional MCP config …
bef521c
fix(reviews): address review comments [skip ci-relay]
donpetry-bot e39b9fd
Merge branch 'main' into dev-lead/issue-677-20260614-1229
don-petry 0740429
fix(reviews): address review comments [skip ci-relay]
donpetry-bot File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,167 @@ | ||
| #!/usr/bin/env bats | ||
| # Unit tests for engine.sh — opt-in MCP config threading (issue #677). | ||
| # | ||
| # Verifies that REVIEW_MCP_CONFIG / REVIEW_MCP_ALLOWED_TOOLS thread the MCP | ||
| # flags (--mcp-config <file> --strict-mcp-config) and merged --allowed-tools | ||
| # into ONLY the claude branch of the agentic (run_agentic) and rubber-duck | ||
| # (run_duck) tiers — never the triage tier — and that the default (knob unset) | ||
| # behavior is byte-for-byte unchanged. | ||
|
|
||
| SCRIPT_DIR="$(cd "$(dirname "$BATS_TEST_FILENAME")/../../.." && pwd)" | ||
| ENGINE_SCRIPT="$SCRIPT_DIR/scripts/engine.sh" | ||
| STUB_ENGINES_DIR="$SCRIPT_DIR/tests/dev-lead/fixtures/engines" | ||
|
|
||
| setup() { | ||
| export GITHUB_ENV="$(mktemp)" | ||
| export GITHUB_OUTPUT="$(mktemp)" | ||
|
|
||
| unset CLAUDE_TRIAGE_MODEL_CHAIN CLAUDE_DEEP_MODEL_CHAIN CLAUDE_AUDIT_MODEL_CHAIN | ||
| unset CLAUDE_ACTION_MODEL_CHAIN CLAUDE_SINGLE_MODEL_CHAIN | ||
| unset REVIEW_MCP_CONFIG REVIEW_MCP_ALLOWED_TOOLS | ||
|
|
||
| STUB_BIN_DIR="$(mktemp -d)" | ||
| cp "$STUB_ENGINES_DIR/stub-claude" "$STUB_BIN_DIR/claude" | ||
| cp "$STUB_ENGINES_DIR/stub-gemini" "$STUB_BIN_DIR/gemini" | ||
| chmod +x "$STUB_BIN_DIR/claude" "$STUB_BIN_DIR/gemini" | ||
| export PATH="$STUB_BIN_DIR:$PATH" | ||
| export STUB_BIN_DIR | ||
|
|
||
| TEST_PROMPT="$(mktemp)" | ||
| echo "test prompt content" > "$TEST_PROMPT" | ||
| export TEST_PROMPT | ||
|
|
||
| ARGS_RECORD="$(mktemp)" | ||
| export STUB_ENGINE_RECORD_ARGS="$ARGS_RECORD" | ||
|
|
||
| # A readable MCP config file the helper can point at. | ||
| MCP_CONFIG_FILE="$(mktemp)" | ||
| echo '{"mcpServers":{}}' > "$MCP_CONFIG_FILE" | ||
| export MCP_CONFIG_FILE | ||
|
|
||
| export DEV_LEAD_DRY_RUN="false" | ||
| export STUB_ENGINE_EXIT=0 | ||
| } | ||
|
|
||
| teardown() { | ||
| rm -f "$GITHUB_ENV" "$GITHUB_OUTPUT" "$TEST_PROMPT" "$ARGS_RECORD" "$MCP_CONFIG_FILE" | ||
| rm -rf "$STUB_BIN_DIR" | ||
| unset STUB_ENGINE_RECORD_ARGS STUB_ENGINE_EXIT | ||
| unset REVIEW_MCP_CONFIG REVIEW_MCP_ALLOWED_TOOLS | ||
| } | ||
|
|
||
| _source_engine() { | ||
| local engine="${1:-claude}" | ||
| export REVIEW_ENGINE="$engine" | ||
| source "$ENGINE_SCRIPT" | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
|
|
||
| # ── Default (knob unset): behavior unchanged ────────────────────────────────── | ||
|
|
||
| @test "agentic: no MCP knob → no MCP flags, default allowed-tools unchanged" { | ||
| _source_engine "claude" | ||
| run run_agentic "$TEST_PROMPT" "claude-opus-4-8" "deep" | ||
| [ "$status" -eq 0 ] | ||
| ! grep -q -- "--mcp-config" "$ARGS_RECORD" | ||
| ! grep -q -- "--strict-mcp-config" "$ARGS_RECORD" | ||
| grep -q -- "--allowed-tools Bash,Read,Grep,Glob" "$ARGS_RECORD" | ||
| } | ||
|
|
||
| @test "agentic: empty REVIEW_MCP_CONFIG → no MCP flags" { | ||
| _source_engine "claude" | ||
| export REVIEW_MCP_CONFIG="" | ||
| run run_agentic "$TEST_PROMPT" "claude-opus-4-8" "deep" | ||
| [ "$status" -eq 0 ] | ||
| ! grep -q -- "--mcp-config" "$ARGS_RECORD" | ||
| } | ||
|
|
||
| @test "agentic: REVIEW_MCP_CONFIG pointing to unreadable file → no MCP flags" { | ||
| _source_engine "claude" | ||
| local unreadable_file | ||
| unreadable_file="$(mktemp)" | ||
| chmod 000 "$unreadable_file" | ||
| export REVIEW_MCP_CONFIG="$unreadable_file" | ||
| run run_agentic "$TEST_PROMPT" "claude-opus-4-8" "deep" | ||
| chmod 600 "$unreadable_file" | ||
| rm -f "$unreadable_file" | ||
| [ "$status" -eq 0 ] | ||
| ! grep -q -- "--mcp-config" "$ARGS_RECORD" | ||
| grep -q -- "--allowed-tools Bash,Read,Grep,Glob" "$ARGS_RECORD" | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
|
|
||
| @test "agentic: REVIEW_MCP_CONFIG pointing to a directory → no MCP flags" { | ||
| _source_engine "claude" | ||
| export REVIEW_MCP_CONFIG="$(mktemp -d)" | ||
| run run_agentic "$TEST_PROMPT" "claude-opus-4-8" "deep" | ||
| rmdir "$REVIEW_MCP_CONFIG" | ||
| [ "$status" -eq 0 ] | ||
| ! grep -q -- "--mcp-config" "$ARGS_RECORD" | ||
| grep -q -- "--allowed-tools Bash,Read,Grep,Glob" "$ARGS_RECORD" | ||
| } | ||
|
|
||
| # ── Knob set: MCP flags threaded into agentic ──────────────────────────────── | ||
|
|
||
| @test "agentic: REVIEW_MCP_CONFIG set → MCP flags appended to claude call" { | ||
| _source_engine "claude" | ||
| export REVIEW_MCP_CONFIG="$MCP_CONFIG_FILE" | ||
| run run_agentic "$TEST_PROMPT" "claude-opus-4-8" "deep" | ||
| [ "$status" -eq 0 ] | ||
| grep -q -- "--mcp-config $MCP_CONFIG_FILE" "$ARGS_RECORD" | ||
| grep -q -- "--strict-mcp-config" "$ARGS_RECORD" | ||
| } | ||
|
|
||
| @test "agentic: REVIEW_MCP_ALLOWED_TOOLS merged into --allowed-tools" { | ||
| _source_engine "claude" | ||
| export REVIEW_MCP_CONFIG="$MCP_CONFIG_FILE" | ||
| export REVIEW_MCP_ALLOWED_TOOLS="mcp__context7__*" | ||
| run run_agentic "$TEST_PROMPT" "claude-opus-4-8" "deep" | ||
| [ "$status" -eq 0 ] | ||
| grep -q -- "--allowed-tools Bash,Read,Grep,Glob,mcp__context7__\*" "$ARGS_RECORD" | ||
| } | ||
|
|
||
| @test "agentic: MCP config set but no allowed-tools knob → base allowed-tools kept" { | ||
| _source_engine "claude" | ||
| export REVIEW_MCP_CONFIG="$MCP_CONFIG_FILE" | ||
| run run_agentic "$TEST_PROMPT" "claude-opus-4-8" "deep" | ||
| [ "$status" -eq 0 ] | ||
| grep -q -- "--allowed-tools Bash,Read,Grep,Glob" "$ARGS_RECORD" | ||
| ! grep -q -- "--allowed-tools Bash,Read,Grep,Glob," "$ARGS_RECORD" | ||
| } | ||
|
|
||
| # ── Knob set: MCP flags threaded into duck ─────────────────────────────────── | ||
|
|
||
| @test "duck: REVIEW_MCP_CONFIG set → MCP flags appended to claude duck call" { | ||
| _source_engine "claude" | ||
| export DUCK_ENGINE="claude" | ||
| export REVIEW_MCP_CONFIG="$MCP_CONFIG_FILE" | ||
| export REVIEW_MCP_ALLOWED_TOOLS="mcp__context7__*" | ||
| run run_duck "$TEST_PROMPT" "claude-sonnet-4-6" | ||
| [ "$status" -eq 0 ] | ||
| grep -q -- "--mcp-config $MCP_CONFIG_FILE" "$ARGS_RECORD" | ||
| grep -q -- "--strict-mcp-config" "$ARGS_RECORD" | ||
| grep -q -- "--allowed-tools Bash,Read,Grep,Glob,mcp__context7__\*" "$ARGS_RECORD" | ||
| # The duck tier's existing --max-turns flag must still be present. | ||
| grep -q -- "--max-turns 25" "$ARGS_RECORD" | ||
| } | ||
|
|
||
| @test "duck: no MCP knob → no MCP flags on claude duck call" { | ||
| _source_engine "claude" | ||
| export DUCK_ENGINE="claude" | ||
| run run_duck "$TEST_PROMPT" "claude-sonnet-4-6" | ||
| [ "$status" -eq 0 ] | ||
| ! grep -q -- "--mcp-config" "$ARGS_RECORD" | ||
| grep -q -- "--allowed-tools Bash,Read,Grep,Glob" "$ARGS_RECORD" | ||
| } | ||
|
|
||
| # ── Triage tier: never gets MCP flags ──────────────────────────────────────── | ||
|
|
||
| @test "triage: MCP knob set → triage still uses --disallowed-tools only, no MCP" { | ||
| _source_engine "claude" | ||
| export REVIEW_MCP_CONFIG="$MCP_CONFIG_FILE" | ||
| export REVIEW_MCP_ALLOWED_TOOLS="mcp__context7__*" | ||
| run run_triage "$TEST_PROMPT" | ||
| [ "$status" -eq 0 ] | ||
| ! grep -q -- "--mcp-config" "$ARGS_RECORD" | ||
| ! grep -q -- "--strict-mcp-config" "$ARGS_RECORD" | ||
| ! grep -q -- "--allowed-tools" "$ARGS_RECORD" | ||
| grep -q -- "--disallowed-tools" "$ARGS_RECORD" | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.