Skip to content

chore(scripts): add ci:digest to summarise failing PR checks for agents - #1568

Merged
gabrielseco merged 7 commits into
mainfrom
chore/ci-failure-digest
Oct 15, 2026
Merged

gabrielseco merged 7 commits into
mainfrom
chore/ci-failure-digest

Conversation

@gabrielseco

@gabrielseco gabrielseco commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds a command that turns a failing PR's CI into a short digest: which checks failed, what kind of failure it is, and the error lines. Agents and humans no longer have to open job logs and scroll through thousands of lines.

Why

Diagnosing a red PR means PR → failed job → raw log. A single failed run can be ~10k lines, mostly test stderr and vitest progress output. A digest is cheaper to read and points at the next step, including "this looks like a flake, rerun it" for the sandbox rate-limit case.

What changed

Toggle details
  • npm run ci:digest -- [pr-number] [--max-lines=N] (scripts/ci-digest.ts) lists failing checks with gh pr checks, pulls each failed job's log with gh run view --job <id> --log-failed, and prints a markdown digest.
  • scripts/ci-digest-utils.ts holds the pure logic: strips timestamps and ANSI codes (including the literal ^[[..m form CI logs use), picks error windows (strong markers like FAIL, ##[error], error TS… win over weak Error: lines that tests print on purpose), classifies the failure (format, lint, type, test, build, flake, unknown) and suggests the narrowest next command.
  • CLAUDE.md lists the command so agents and contributors run it before reading raw job logs.
  • Checked against PR chore(deps): update dependency node to v24.22.0 #1566: a ~10k-line failed coverage job becomes about 30 lines containing the failing test, the assertion and the exit code.
  • Not public API. Dev tooling only.
  • Follow-ups, not in this PR: junit/JSON reporter artifacts and a failure-only step summary written from CI itself.

Screenshots

N/A

Related Resources

N/A

Testing

  • Unit tests in scripts/ci-digest-utils.test.ts (parsing, cleaning, windows, classification, rendering).
  • Removed the strong/weak error-tier selection and confirmed 3 tests fail, then restored it. The caret-ANSI test failed before its fix.
  • Ran npm run ci:digest -- 1566 against a real failing PR.
  • Tested against the example/ app in a browser: N/A
  • Feature flag: N/A

🤖 Generated with Claude Code


Note

Low Risk
Dev tooling and docs only; no runtime package or CI workflow changes.

Overview
Adds npm run ci:digest -- <pr> [--max-lines=N], a dev-only CLI that uses the GitHub CLI to list failing PR checks, pull failed job logs, and print a short markdown digest instead of scrolling multi‑thousand-line Actions output.

scripts/ci-digest.ts wires gh pr checks and gh run view --log-failed (current PR if no number is passed). scripts/ci-digest-utils.ts holds the parsing: strip timestamps/ANSI (including ^[[ caret escapes), extract error windows (strong markers like FAIL / error TS over noisy Error: lines), classify failures (format, lint, type, test, e2e, coverage, build, flake, unknown), and suggest a concrete next command. External checks without Actions logs get a fallback message. scripts/ci-digest-utils.test.ts covers the utilities. CLAUDE.md documents running the digest before raw job logs.

Reviewed by Cursor Bugbot for commit 5b3d200. Bugbot is set up for automated code reviews on this repo. Configure here.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

📦 Bundle Size Report

Metric Current Previous Change Status
Total (gzip) 230.76 kB 230.76 kB 0 B (0%) 🟢
Total (raw) 642.57 kB 642.57 kB 0 B (0%) 🟢
CSS (gzip) 21.95 kB 21.95 kB 0 B (0%) 🟢
CSS (raw) 114.51 kB 114.51 kB 0 B (0%) 🟢

Size Limits

  • ✅ Total gzipped: 230.76 kB / 350 kB (65.9%)
  • ✅ Total raw: 642.57 kB / 850 kB (75.6%)
  • ✅ CSS gzipped: 21.95 kB / 25 kB (87.8%)

Largest Files (Top 5)

  1. index.esm-wIkXrqU6.js - 11.4 kB (0 B (0%))
  2. styles.css - 10.97 kB (0 B (0%))
  3. index.css - 10.97 kB (0 B (0%))
  4. internals-BOZceErx.js - 6.13 kB (0 B (0%))
  5. hooks-ThU7CDLI.js - 5.85 kB (0 B (0%))
View All Files (291 total)
File Size (gzip) Change
index.esm-wIkXrqU6.js 11.4 kB 0 B (0%)
styles.css 10.97 kB 0 B (0%)
index.css 10.97 kB 0 B (0%)
internals-BOZceErx.js 6.13 kB 0 B (0%)
hooks-ThU7CDLI.js 5.85 kB 0 B (0%)
sdk.gen-BRPiw4M1.js 5.58 kB 0 B (0%)
index.js 5.51 kB 0 B (0%)
flows/Onboarding/hooks.js 4.67 kB 0 B (0%)
utils-CUgvJHl4.js 4.07 kB 0 B (0%)
FieldSetField-D0HdSLTs.js 4.04 kB 0 B (0%)

✅ Bundle size check passed

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Deploy preview for adp-cost-calculator ready!

Project:adp-cost-calculator
Status: ✅  Deploy successful!
Preview URL:https://adp-cost-calculator-1ae15q25o-remotecom.vercel.app
Latest Commit:5b3d200

Deployed with vercel-action

…ail when truncating

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Deploy preview for remote-flows ready!

Project:remote-flows
Status: ✅  Deploy successful!
Preview URL:https://remote-flows-urhb256ub-remotecom.vercel.app
Latest Commit:5b3d200

Deployed with vercel-action

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

📊 Coverage Report

⚠️ Coverage decreased

Metric Current Previous Change Status
Lines 89.58% 89.59% -0.02% 🔴
Statements 89.45% 89.47% -0.02% 🔴
Functions 88.48% 88.48% 0% ⚪
Branches 81.21% 81.24% -0.02% 🔴

Detailed Breakdown

Lines Coverage
  • Covered: 5396 / 6024
  • Coverage: 89.58%
  • Change: -0.02% (-1 lines)
Statements Coverage
  • Covered: 5461 / 6105
  • Coverage: 89.45%
  • Change: -0.02% (-1 statements)
Functions Coverage
  • Covered: 1428 / 1614
  • Coverage: 88.48%
  • Change: 0% (0 functions)
Branches Coverage
  • Covered: 3350 / 4125
  • Coverage: 81.21%
  • Change: -0.02% (-1 branches)

✅ Coverage check passed

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread scripts/ci-digest-utils.ts Resolved
Comment thread scripts/ci-digest-utils.ts Resolved
Comment thread scripts/ci-digest-utils.ts Resolved
- Treat the Actions '##[error]Process completed with exit code' footer as
  noise so earlier errors stay in the excerpt.
- Stop calling a plain 403 forbidden or a Playwright timeout a flake; only
  rate-limit/blocked 403s and network resets are.
- Add e2e and coverage failure kinds with their own next step instead of
  always suggesting vitest.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 86b3758. Configure here.

Comment thread scripts/ci-digest-utils.ts Resolved
gabrielseco and others added 3 commits October 14, 2026 19:19
…hold miss

Check names like 'Tests with Coverage' always contain 'coverage', so install
errors and suite crashes were labelled as threshold misses.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Only call a failure e2e when the log shows 'playwright test' (or the
  check name says e2e), not on any mention of Playwright.
- Reject a non-integer or non-positive --max-lines instead of producing
  garbage truncation.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@gabrielseco
gabrielseco merged commit 1a0560c into main Oct 15, 2026
16 checks passed
@gabrielseco
gabrielseco deleted the chore/ci-failure-digest branch October 15, 2026 13:37
@gabrielseco gabrielseco mentioned this pull request Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants