Repository navigation
Description
🏥 CI Failure Investigation - Run #33691
Summary
CI run 33691 failed because the coverage gate counted 137 tests that exist in the codebase but were not executed by the canary jobs.
Failure Details
- Run: 21662949681
- Commit: bb7ce55
- Trigger: push
Root Cause Analysis
The compare-test-coverage.sh step in the failing jobs enumerated 3,977 tests from all-tests.txt but only observed 3,840 entries in executed-tests.txt. The gate therefore failed with ❌ FAILURE: Found 137 tests that are NOT being executed in CI, even though the referenced integration suites completed successfully. The gap exists because the CI job did not run those 137 tests (such as TestAllCommandsExist, TestCompilationStats, TestCompileWorkflowWithValidation_InvalidFile, etc.) that are still tracked in the codebase.
Failed Jobs and Errors
- Integration: CLI Completion & Other — aborted once
compare-test-coverage.shreported missing tests. - canary_go — failed at the same coverage gate step, logging the
137 tests that are NOT being executed in CImessage.
Investigation Findings
./scripts/compare-test-coverage.sh all-tests.txt executed-tests.txtpulled the discrepancy between defined and executed tests.- The missing list grows when new tests are added without ensuring the CI job runs them or when
executed-tests.txtisn’t generated from the suites that include the new names. - This job-level coverage gate is stricter than the individual test suites and therefore fails fast whenever the two files drift apart.
Recommended Actions
- Update the integration jobs so that their
go testinvocations cover the missing test names before runningcompare-test-coverage.sh. - Run
./scripts/compare-test-coverage.sh all-tests.txt executed-tests.txtlocally (after regeneratingall-tests.txt) to catch coverage gate failures before pushing. - If certain tests should remain excluded intentionally, document them and have the coverage script skip those names explicitly.
Prevention Strategies
- Automate regeneration of
all-tests.txtand validation ofexecuted-tests.txtas part of pre-commit hooks or CI prechecks whenever new tests are added. - Introduce a faster preflight step that runs the coverage comparison in a lightweight job so developers discover mismatches before reaching the final canary job.
AI Team Self-Improvement
Before adding or renaming tests, rerun ./scripts/compare-test-coverage.sh and ensure the changes are reflected in the CI jobs that build executed-tests.txt; document any intentional omissions in instructions.md so future agents know how to keep the coverage files aligned.
Historical Context
No prior investigation matching this coverage-gate failure pattern was found in the cached /tmp/gh-aw/cache-memory/investigations directory.
AI generated by CI Failure Doctor
To add this workflow in your repository, run
gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. See usage guide.
- expires on Feb 6, 2026, 8:08 AM UTC
Activity
pelikhan commentedon Feb 4, 2026
/q cli failure doctor should be 1d
github-actions commentedon Feb 4, 2026
🎩 Mission equipment ready! Q has optimized your workflow. Use wisely, 007! 🔫
github-actions commentedon Feb 4, 2026
🎩 Q reporting mission complete!
I've analyzed the CI Failure Doctor workflow based on your request and created an optimization.
What I Found
The CI Failure Doctor workflow currently creates issues that expire in 2 days. Based on your comment "/q cli failure doctor should be 1d", I've optimized this configuration.
Changes Made
✅ Reduced issue expiration from 2d to 1d in
.github/workflows/ci-doctor.mdThis is a surgical change that only modifies the
safe-outputs.create-issue.expiresfield from2dto1d(line 31).Expected Benefits
Next Steps
A pull request has been created with this optimization. Once merged, the Copilot agent will automatically compile the updated workflow file.
Investigation Details: Analyzed recent CI Doctor runs (runs #1562, #1561, #1558, #1554, #1553) and issues (#13716, #13685) to confirm the current 2-day expiration pattern.