Repository navigation
feat: implement issue #1973 — [Engine config · Phase 1] Versioned config/ai-engines.json as the single source of truth for providers, model catalog and per-task chains #2135
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
1cce9d9
61a7d2f
738475f
a781787
ad2b9ce
13e11d4
2ea40ea
0f7ee3b
b7234b0
ffe6d52
50a94a4
37297e3
f2884f3
ccf5cad
4e26c16
a4dd136
e50faa7
47b9daf
f57d709
922f6c3
bce75bc
c0d7853
09ac9c1
caa1685
0129b98
70cd676
902ce90
3683639
899eff1
65820a6
18da483
c3ba0c7
9bd98ae
7cd94c1
c00aa97
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -191,7 +191,7 @@ jobs: | |||||||||
| # legacy DEV_LEAD_ENGINES name is still honoured); DEV_LEAD_ENGINE is a | ||||||||||
| # legacy primary override — leave it unset to let AI_ENGINES decide. | ||||||||||
| AI_ENGINES: ${{ vars.AI_ENGINES || vars.DEV_LEAD_ENGINES || '' }} | ||||||||||
| DEV_LEAD_ENGINE: ${{ vars.DEV_LEAD_ENGINE || (startsWith(vars.AI_ENGINES || vars.DEV_LEAD_ENGINES, 'gemini') && 'gemini') || (startsWith(vars.AI_ENGINES || vars.DEV_LEAD_ENGINES, 'copilot') && 'copilot') || 'claude' }} | ||||||||||
| DEV_LEAD_ENGINE: ${{ vars.DEV_LEAD_ENGINE || (startsWith(vars.AI_ENGINES || vars.DEV_LEAD_ENGINES, 'gemini') && 'gemini') || (startsWith(vars.AI_ENGINES || vars.DEV_LEAD_ENGINES, 'copilot') && 'copilot') || '' }} | ||||||||||
| AI_DUCK_ENGINE: ${{ vars.AI_DUCK_ENGINE || '' }} | ||||||||||
| AI_DUCK_MODEL: ${{ vars.AI_DUCK_MODEL || '' }} | ||||||||||
| GEMINI_FLASH_MODEL: ${{ vars.GEMINI_FLASH_MODEL || '' }} | ||||||||||
|
|
@@ -214,32 +214,59 @@ jobs: | |||||||||
| - name: Engine token preflight (#1525) | ||||||||||
| # With the secret declaration relaxed to required:false, presence is | ||||||||||
| # enforced here instead — one actionable line, not a secrets-evaluation | ||||||||||
| # stack error on every event. Skipped when AI_ENGINES leaves claude out. | ||||||||||
| # stack error on every event. Skipped when AI_ENGINES leaves claude out or | ||||||||||
| # config/ai-engines.json disables Claude (provider disablement takes | ||||||||||
| # precedence over AI_ENGINES, so check the file before requiring the token). | ||||||||||
| env: | ||||||||||
| AI_ENGINES: ${{ vars.AI_ENGINES || vars.DEV_LEAD_ENGINES || '' }} | ||||||||||
| DEV_LEAD_ENGINE: ${{ vars.DEV_LEAD_ENGINE || '' }} | ||||||||||
| AGENT_REF: ${{ inputs.agent_ref }} | ||||||||||
| GH_TOKEN: ${{ secrets.GH_PAT_DON_PETRY || secrets.GH_PAT_WORKFLOWS || github.token }} | ||||||||||
| CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} | ||||||||||
| run: | | ||||||||||
| # Is Claude enabled? Only a valid AI_ENGINES list that omits claude turns it | ||||||||||
| # off; empty or invalid values fall back to the default chain, which includes | ||||||||||
| # claude (scripts/lib/engine-chain.sh — mirrored inline because this runs | ||||||||||
| # before the scripts are checked out). | ||||||||||
| claude_enabled=true | ||||||||||
| spec="$(printf '%s' "${AI_ENGINES:-}" | tr '[:upper:],' '[:lower:] ')" | ||||||||||
| if [ -n "${spec//[[:space:]]/}" ]; then | ||||||||||
| # Require the token only when the EFFECTIVE primary engine is claude, as | ||||||||||
| # scripts/lib/engine-chain.sh's ai_engine_primary resolves it (mirrored | ||||||||||
| # inline: this runs before checkout). The file's enabled providers in | ||||||||||
| # fallback_order are the default chain; AI_ENGINES narrows it (a valid list | ||||||||||
| # cannot re-enable a disabled provider; an invalid one means the default | ||||||||||
| # chain and primary); DEV_LEAD_ENGINE wins only when the chain enables it. | ||||||||||
| # Fetched through the API; unreadable config fails closed (claude first). | ||||||||||
| file_chain="$(gh api -H 'Accept: application/vnd.github.raw' \ | ||||||||||
| "repos/petry-projects/.github-private/contents/config/ai-engines.json?ref=${AGENT_REF}" 2>/dev/null \ | ||||||||||
| | jq -r '. as $c | [$c.providers.fallback_order[] | select($c.providers[.].enabled == true)] | join(" ")' 2>/dev/null || true)" | ||||||||||
|
Comment on lines
+234
to
+236
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion: Runtime checks out Assessment: 🟠 Prompt for AI Agent 🤖This is a comment left during a code review.
**Path:** .github/workflows/dev-lead-reusable.yml
**Line:** 234:236
**Comment:**
*Api Mismatch: Runtime checks out `combsmith/.github-private`, but this lookup reads `petry-projects/.github-private`; stale or unavailable data can misidentify the primary and wrongly require or skip Claude's token.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix |
||||||||||
| [ -n "$file_chain" ] || file_chain="claude gemini copilot" | ||||||||||
| effective_primary() { | ||||||||||
| local pref spec e chain="" valid=true first kept="" | ||||||||||
| pref="$(printf '%s' "$1" | tr '[:upper:]' '[:lower:]' | tr -d '[:space:]')" | ||||||||||
| spec="$(printf '%s' "${AI_ENGINES:-}" | tr '[:upper:],' '[:lower:] ')" | ||||||||||
| set -f | ||||||||||
| known=true; has_claude=false | ||||||||||
| for e in $spec; do | ||||||||||
| case "$e" in | ||||||||||
| claude) has_claude=true ;; | ||||||||||
| gemini|copilot) ;; | ||||||||||
| *) known=false ;; | ||||||||||
| claude|gemini|copilot) case " $chain " in *" $e "*) ;; *) chain="$chain $e" ;; esac ;; | ||||||||||
| *) valid=false ;; | ||||||||||
| esac | ||||||||||
| done | ||||||||||
| set +f | ||||||||||
| if [ "$known" = true ] && [ "$has_claude" = false ]; then claude_enabled=false; fi | ||||||||||
| fi | ||||||||||
| if [ "$claude_enabled" = false ]; then | ||||||||||
| echo "claude not in AI_ENGINES (${AI_ENGINES}) — CLAUDE_CODE_OAUTH_TOKEN not required" | ||||||||||
| if [ "$valid" = false ]; then chain=""; pref=""; fi | ||||||||||
| if [ -n "${chain// /}" ]; then | ||||||||||
| for e in $chain; do case " $file_chain " in *" $e "*) kept="$kept $e" ;; esac; done | ||||||||||
| chain="$kept" | ||||||||||
| else | ||||||||||
| chain=" $file_chain" | ||||||||||
| fi | ||||||||||
| [ -n "${chain// /}" ] || { printf '%s' none; return; } | ||||||||||
| if [ -n "$pref" ] && [[ "$chain " == *" $pref "* ]]; then printf '%s' "$pref"; return; fi | ||||||||||
| first="${chain# }" | ||||||||||
| printf '%s' "${first%% *}" | ||||||||||
| } | ||||||||||
| primary="$(effective_primary "$DEV_LEAD_ENGINE")" | ||||||||||
| if [ "$primary" != claude ]; then | ||||||||||
| if [ -z "$CLAUDE_CODE_OAUTH_TOKEN" ]; then | ||||||||||
| # claude may still sit later in the chain; mark it unavailable so | ||||||||||
| # run_writer_with_fallback skips it instead of hitting a terminal auth error. | ||||||||||
| echo "CLAUDE_UNAVAILABLE=1" >> "$GITHUB_ENV" | ||||||||||
| fi | ||||||||||
| echo "effective primary engine is ${primary} (AI_ENGINES=${AI_ENGINES}) — CLAUDE_CODE_OAUTH_TOKEN not required" | ||||||||||
| exit 0 | ||||||||||
|
Comment on lines
+262
to
270
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When Gemini or Copilot is the effective primary and AGENTS.md reference: AGENTS.md:L164-L170 Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The token preflight in .github/workflows/dev-lead-reusable.yml now writes |
||||||||||
| fi | ||||||||||
| if [ -z "$CLAUDE_CODE_OAUTH_TOKEN" ]; then | ||||||||||
|
|
@@ -255,7 +282,7 @@ jobs: | |||||||||
| - name: Checkout caller repo | ||||||||||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||||||||||
| with: | ||||||||||
| token: ${{ secrets.GH_PAT_DON_PETRY_WRITER || secrets.GH_PAT_DON_PETRY || secrets.GH_PAT_WORKFLOWS || github.token }} | ||||||||||
| token: ${{ secrets.GH_PAT_DON_PETRY || secrets.GH_PAT_WORKFLOWS || github.token }} | ||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
If the writer PAT is what lets the first job's pushes and PRs trigger downstream workflows or pass branch protection, those operations now run with a different identity or weaker permissions. Restore the writer token on this checkout, or make the token order consistent across all jobs: token: ${{ secrets.GH_PAT_DON_PETRY_WRITER || secrets.GH_PAT_DON_PETRY || secrets.GH_PAT_WORKFLOWS || github.token }}
Suggested change
Spotted by Graphite |
||||||||||
|
|
||||||||||
| - name: Checkout dev-lead scripts | ||||||||||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||||||||||
|
|
@@ -268,8 +295,10 @@ jobs: | |||||||||
| # Cross-repo callers: github.token cannot read a private org repo — ensure | ||||||||||
| # GH_PAT_WORKFLOWS is set as an org or repo secret in the calling repo. | ||||||||||
| token: ${{ secrets.GH_PAT_DON_PETRY_WRITER || secrets.GH_PAT_DON_PETRY || secrets.GH_PAT_WORKFLOWS || github.token }} | ||||||||||
| # config/ holds ai-engines.json, the engine defaults the scripts read (#1973). | ||||||||||
| sparse-checkout: | | ||||||||||
| scripts | ||||||||||
| config | ||||||||||
|
don-petry marked this conversation as resolved.
Resolved
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
|
||||||||||
| prompts/dev-lead | ||||||||||
| personas/dev-lead | ||||||||||
|
|
||||||||||
|
|
@@ -468,7 +497,22 @@ jobs: | |||||||||
| # Log the CLI version in use, cached or fresh (#1957). | ||||||||||
| echo "claude-code version: $(claude --version 2>/dev/null || echo unavailable)" | ||||||||||
|
|
||||||||||
| case "$DEV_LEAD_ENGINE" in | ||||||||||
| # DEV_LEAD_ENGINE is empty when no legacy override is set and AI_ENGINES | ||||||||||
| # does not start with gemini/copilot. Resolve the primary from the | ||||||||||
| # checked-out config so the fallback CLIs still install; claude is the | ||||||||||
| # last resort when the config cannot be read. | ||||||||||
| engine="${DEV_LEAD_ENGINE:-}" | ||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion: Assessment: 🟠 Prompt for AI Agent 🤖This is a comment left during a code review.
**Path:** .github/workflows/dev-lead-reusable.yml
**Line:** 504:504
**Comment:**
*Incorrect Variable Usage: `ai_engine_primary` accepts overrides case-insensitively, but this preserves the raw value for the lowercase install switch; `GEMINI` skips Gemini CLI installation, so Gemini-primary runs fail.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix |
||||||||||
| if [ -z "$engine" ]; then | ||||||||||
| # shellcheck source=scripts/lib/engine-chain.sh | ||||||||||
| if source .dev-lead/scripts/lib/engine-chain.sh 2>/dev/null \ | ||||||||||
| && primary="$(ai_engine_primary "")" && [ -n "$primary" ]; then | ||||||||||
| engine="$primary" | ||||||||||
| else | ||||||||||
| engine=claude | ||||||||||
| fi | ||||||||||
| fi | ||||||||||
|
|
||||||||||
| case "$engine" in | ||||||||||
| claude) | ||||||||||
| npm install -g @google/gemini-cli || true | ||||||||||
| if ! gh copilot --version >/dev/null 2>&1; then | ||||||||||
|
|
@@ -797,17 +841,19 @@ jobs: | |||||||||
| - name: Checkout caller repo | ||||||||||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||||||||||
| with: | ||||||||||
| token: ${{ secrets.GH_PAT_DON_PETRY || secrets.GH_PAT_WORKFLOWS || github.token }} | ||||||||||
| token: ${{ secrets.GH_PAT_DON_PETRY_WRITER || secrets.GH_PAT_DON_PETRY || secrets.GH_PAT_WORKFLOWS || github.token }} | ||||||||||
|
|
||||||||||
| - name: Checkout dev-lead scripts | ||||||||||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||||||||||
| with: | ||||||||||
| repository: combsmith/.github-private | ||||||||||
| ref: ${{ inputs.agent_ref }} | ||||||||||
| path: .dev-lead | ||||||||||
| token: ${{ secrets.GH_PAT_DON_PETRY || secrets.GH_PAT_WORKFLOWS || github.token }} | ||||||||||
| token: ${{ secrets.GH_PAT_DON_PETRY_WRITER || secrets.GH_PAT_DON_PETRY || secrets.GH_PAT_WORKFLOWS || github.token }} | ||||||||||
| # config/ holds ai-engines.json, the engine defaults the scripts read (#1973). | ||||||||||
| sparse-checkout: | | ||||||||||
| scripts | ||||||||||
| config | ||||||||||
| prompts/dev-lead | ||||||||||
| personas/dev-lead | ||||||||||
|
|
||||||||||
|
|
@@ -983,29 +1029,54 @@ jobs: | |||||||||
| - name: Engine token preflight (#1525) | ||||||||||
| env: | ||||||||||
| AI_ENGINES: ${{ vars.AI_ENGINES || vars.DEV_LEAD_ENGINES || '' }} | ||||||||||
| DEV_LEAD_ENGINE: ${{ vars.DEV_LEAD_ENGINE || '' }} | ||||||||||
| AGENT_REF: ${{ inputs.agent_ref }} | ||||||||||
| GH_TOKEN: ${{ secrets.GH_PAT_DON_PETRY || secrets.GH_PAT_WORKFLOWS || github.token }} | ||||||||||
| CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} | ||||||||||
| run: | | ||||||||||
| # Is Claude enabled? Only a valid AI_ENGINES list that omits claude turns it | ||||||||||
| # off; empty or invalid values fall back to the default chain, which includes | ||||||||||
| # claude (scripts/lib/engine-chain.sh — mirrored inline because this runs | ||||||||||
| # before the scripts are checked out). | ||||||||||
| claude_enabled=true | ||||||||||
| spec="$(printf '%s' "${AI_ENGINES:-}" | tr '[:upper:],' '[:lower:] ')" | ||||||||||
| if [ -n "${spec//[[:space:]]/}" ]; then | ||||||||||
| # Require the token only when the EFFECTIVE primary engine is claude, as | ||||||||||
| # scripts/lib/engine-chain.sh's ai_engine_primary resolves it (mirrored | ||||||||||
| # inline: this runs before checkout). The file's enabled providers in | ||||||||||
| # fallback_order are the default chain; AI_ENGINES narrows it (a valid list | ||||||||||
| # cannot re-enable a disabled provider; an invalid one means the default | ||||||||||
| # chain and primary); DEV_LEAD_ENGINE wins only when the chain enables it. | ||||||||||
| # Fetched through the API; unreadable config fails closed (claude first). | ||||||||||
| file_chain="$(gh api -H 'Accept: application/vnd.github.raw' \ | ||||||||||
| "repos/petry-projects/.github-private/contents/config/ai-engines.json?ref=${AGENT_REF}" 2>/dev/null \ | ||||||||||
| | jq -r '. as $c | [$c.providers.fallback_order[] | select($c.providers[.].enabled == true)] | join(" ")' 2>/dev/null || true)" | ||||||||||
| [ -n "$file_chain" ] || file_chain="claude gemini copilot" | ||||||||||
| effective_primary() { | ||||||||||
| local pref spec e chain="" valid=true first kept="" | ||||||||||
| pref="$(printf '%s' "$1" | tr '[:upper:]' '[:lower:]' | tr -d '[:space:]')" | ||||||||||
| spec="$(printf '%s' "${AI_ENGINES:-}" | tr '[:upper:],' '[:lower:] ')" | ||||||||||
| set -f | ||||||||||
| known=true; has_claude=false | ||||||||||
| for e in $spec; do | ||||||||||
| case "$e" in | ||||||||||
| claude) has_claude=true ;; | ||||||||||
| gemini|copilot) ;; | ||||||||||
| *) known=false ;; | ||||||||||
| claude|gemini|copilot) case " $chain " in *" $e "*) ;; *) chain="$chain $e" ;; esac ;; | ||||||||||
| *) valid=false ;; | ||||||||||
| esac | ||||||||||
| done | ||||||||||
| set +f | ||||||||||
| if [ "$known" = true ] && [ "$has_claude" = false ]; then claude_enabled=false; fi | ||||||||||
| fi | ||||||||||
| if [ "$claude_enabled" = false ]; then | ||||||||||
| echo "claude not in AI_ENGINES (${AI_ENGINES}) — CLAUDE_CODE_OAUTH_TOKEN not required" | ||||||||||
| if [ "$valid" = false ]; then chain=""; pref=""; fi | ||||||||||
| if [ -n "${chain// /}" ]; then | ||||||||||
| for e in $chain; do case " $file_chain " in *" $e "*) kept="$kept $e" ;; esac; done | ||||||||||
| chain="$kept" | ||||||||||
| else | ||||||||||
| chain=" $file_chain" | ||||||||||
| fi | ||||||||||
| [ -n "${chain// /}" ] || { printf '%s' none; return; } | ||||||||||
| if [ -n "$pref" ] && [[ "$chain " == *" $pref "* ]]; then printf '%s' "$pref"; return; fi | ||||||||||
| first="${chain# }" | ||||||||||
| printf '%s' "${first%% *}" | ||||||||||
| } | ||||||||||
| primary="$(effective_primary "$DEV_LEAD_ENGINE")" | ||||||||||
| if [ "$primary" != claude ]; then | ||||||||||
| if [ -z "$CLAUDE_CODE_OAUTH_TOKEN" ]; then | ||||||||||
| # claude may still sit later in the chain; mark it unavailable so | ||||||||||
| # run_writer_with_fallback skips it instead of hitting a terminal auth error. | ||||||||||
| echo "CLAUDE_UNAVAILABLE=1" >> "$GITHUB_ENV" | ||||||||||
| fi | ||||||||||
| echo "effective primary engine is ${primary} (AI_ENGINES=${AI_ENGINES}) — CLAUDE_CODE_OAUTH_TOKEN not required" | ||||||||||
| exit 0 | ||||||||||
| fi | ||||||||||
| if [ -z "$CLAUDE_CODE_OAUTH_TOKEN" ]; then | ||||||||||
|
|
@@ -1026,10 +1097,12 @@ jobs: | |||||||||
| # defaults to "main" → unchanged behaviour (#535/#506). | ||||||||||
| ref: ${{ inputs.agent_ref }} | ||||||||||
| path: .dev-lead | ||||||||||
| token: ${{ secrets.GH_PAT_DON_PETRY || secrets.GH_PAT_WORKFLOWS || github.token }} | ||||||||||
| token: ${{ secrets.GH_PAT_DON_PETRY_WRITER || secrets.GH_PAT_DON_PETRY || secrets.GH_PAT_WORKFLOWS || github.token }} | ||||||||||
| persist-credentials: false | ||||||||||
| # config/ holds ai-engines.json, the engine defaults the scripts read (#1973). | ||||||||||
| sparse-checkout: | | ||||||||||
| scripts | ||||||||||
| config | ||||||||||
|
|
||||||||||
| - name: Resolve PR from check_run | ||||||||||
| id: relay | ||||||||||
|
|
||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| name: Validate Engine Config | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Fresh evidence after the earlier AGENTS.md reference: AGENTS.md:L218-L219 Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added a documented AGENTS.md exception for |
||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'config/ai-engines.json' | ||
| - 'config/ai-engines.schema.json' | ||
| - 'scripts/lib/model-pricing.tsv' | ||
| - 'scripts/validate-ai-engines.py' | ||
| - '.github/workflows/engine-config-validate.yml' | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - 'config/ai-engines.json' | ||
| - 'config/ai-engines.schema.json' | ||
| - 'scripts/lib/model-pricing.tsv' | ||
| - 'scripts/validate-ai-engines.py' | ||
| merge_group: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| validate-ai-engines: | ||
| # The engine config gate (#1973). config/ai-engines.json is the source of | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This new workflow is explicitly an enforcement gate whose expected failure mode is rejecting an invalid engine config, but the inspected AGENTS.md reference: AGENTS.md:L255-L258 Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added |
||
| # truth for enabled providers, the model catalog and each task's chain, so a | ||
| # bad value is caught at PR time instead of in a failing review. Validates | ||
| # the file against config/ai-engines.schema.json (draft 2020-12), then fails | ||
| # on a task model that is missing from `models` or `retired`, a model in | ||
| # another provider's chain, an enabled provider's model with no row in | ||
| # scripts/lib/model-pricing.tsv, and a duck or Copilot chain with more than | ||
| # one model. Fully HERMETIC. This is the required check for config changes. | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: Install Python deps | ||
| run: pip3 install --quiet "jsonschema==4.26.0" --only-binary=":all:" | ||
|
|
||
| - name: Validate config/ai-engines.json | ||
| run: python3 scripts/validate-ai-engines.py | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,6 +15,7 @@ | |
| - 'interaction-contracts/**' | ||
| - 'evals/**' | ||
| - 'release/**' | ||
| - 'config/**' # config/ai-engines.json + schema (validate-ai-engines) | ||
| - '.gitleaks.toml' | ||
| # Hash-pinned pip requirements installed by the jobs below (#1240), so a | ||
| # Dependabot bump of requirements/ci/ is exercised before it merges. | ||
|
|
@@ -33,6 +34,7 @@ | |
| - 'prompts/**' | ||
| - '.github/workflows/**' | ||
| - '.github/**' # check-model-pins scans all of .github | ||
| - 'config/**' | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
|
|
@@ -217,6 +219,9 @@ | |
| tests/test_discover_parity_skills.bats \ | ||
| tests/test_eval_health_notify.bats \ | ||
| tests/test_consumer_manifest.bats \ | ||
| tests/test_validate_ai_engines.bats \ | ||
|
don-petry marked this conversation as resolved.
Resolved
|
||
| tests/dev-lead/unit/test_ai_engines_config.bats \ | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The commit adds AGENTS.md reference: AGENTS.md:L287-L295 Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added |
||
| tests/dev-lead/unit/test_install_step_engine.bats \ | ||
| tests/test_downstream_impact_map.bats \ | ||
| tests/test_downstream_impact_assemble.bats \ | ||
| tests/test_downstream_impact_triage_inline.bats \ | ||
|
|
@@ -371,7 +376,7 @@ | |
| timeout-minutes: 5 | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| # NOTE: persist-credentials is intentionally left at the default (true) — | ||
| # vci_resolve_reusable needs `git fetch origin` to authenticate against this | ||
| # private repo when resolving same-repo channel tags. Setting | ||
| # persist-credentials: false would silently push every such caller into the | ||
|
|
@@ -542,6 +547,28 @@ | |
| - name: Validate per-case eval schema conformance across the tree | ||
| run: python3 evals/validate-cases.py --schema-tree evals | ||
|
|
||
| validate-ai-engines: | ||
|
don-petry marked this conversation as resolved.
Resolved
don-petry marked this conversation as resolved.
Resolved
|
||
| # The engine config gate (#1973). config/ai-engines.json is the source of | ||
| # truth for enabled providers, the model catalog and each task's chain, so a | ||
| # bad value is caught at PR time instead of in a failing review. Validates | ||
| # the file against config/ai-engines.schema.json (draft 2020-12, the same | ||
|
don-petry marked this conversation as resolved.
Resolved
|
||
| # jsonschema setup as validate-eval-cases), then fails on a task model that is | ||
| # missing from `models` or `retired`, a model in another provider's chain, an | ||
| # enabled provider's model with no row in scripts/lib/model-pricing.tsv, and a | ||
| # duck or Copilot chain with more than one model. Fully HERMETIC. | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: Install Python deps | ||
| run: pip3 install --quiet "jsonschema==4.26.0" --only-binary=":all:" | ||
|
|
||
| - name: Validate config/ai-engines.json | ||
| run: python3 scripts/validate-ai-engines.py | ||
|
|
||
| validate-interaction-contracts: | ||
| # Interaction-contract well-formedness gate (Story 2 / #1404, AC #5). Parses | ||
| # every machine-readable interaction contract — the co-located persona lenses | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.