Skip to content

feat(cli): check_report names the finding that stopped the check - #7305

Merged
miguel-heygen merged 2 commits into
mainfrom
dobserve-checkcode
Nov 6, 2026
Merged

miguel-heygen merged 2 commits into
mainfrom
dobserve-checkcode

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

What

check_report now carries stop_code: the code of the finding that ended the check early, one of media_copies_unfinished, check_budget_reached or check_runtime_failure. It is left out when the check ran to the end.

Why

The event had only counts per section, so a check that stopped on unfinished media copies or on its budget looked the same as any other failed check. Nobody could count how often a check stops, or why.

Related work

Follows the lint event's code_counts, which already sends closed codes the same way.

How

checkPipeline.ts picks the first runtime finding whose code is in STOP_CODES (the codes runtimeFailure gives a stopping failure; StopCode is the union of them, so the sent code is a compile-time closed set) and passes it as stopCode; trackCheckReport sends it as stop_code only when set. Only the code, never the finding's message or file. There is one caller of trackCheckReport and no field allowlist or doc elsewhere in the repo (grep for the event's field names).

A stop in the finding-crops or finding-frames phase (--snapshots / --finding-frames runs) is added after buildReport has already sent check_report, so it sends no stop_code, the same way ok shows the state from before the crops: the count is complete only for stops before them.

Test plan

  • telemetry/events.test.ts "names the finding that stopped the check, and omits it when none did" and commands/check.test.ts (the budget scenario asserts stopCode: "check_budget_reached"; an ordinary failing check has none): the first two fail without the change; both files pass 3 runs in a row (245 tests).
  • Unit tests added/updated
  • Manual testing performed: not run against a real project; only check_budget_reached is covered through the pipeline, media_copies_unfinished only as an accepted code.
  • Documentation updated: no docs list this event's fields.
  • Comments follow CONTRIBUTING.md "Comments"

Size

About 57 changed lines (under the 100-line floor) on purpose: one new field on one event, set at its single call site, with its type and two tests. It does not bundle with other OSS work because this lane has none open that touches the check pipeline or its telemetry; folding it into an unrelated PR would make a one-field change harder to revert.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Fallow audit report

Quality gate failed

Found 4 findings: Dead code 1, Duplication 3.

Dead code (1)
Severity Rule Location Description
major fallow/unused-export packages/cli/src/utils/checkPipeline.ts:2270 Export 'STOP_CODES' is never imported by other modules
Duplication (3)
Severity Rule Location Description
minor fallow/code-duplication packages/cli/src/telemetry/events.ts:388 Code clone dup:728fb1b2 (37 lines, 3 instances)
minor fallow/code-duplication packages/producer/src/services/render/observability.ts:104 Code clone dup:728fb1b2 (37 lines, 3 instances)
minor fallow/code-duplication packages/producer/src/services/render/perfSummary.ts:127 Code clone dup:728fb1b2 (37 lines, 3 instances)

Generated by fallow.

Gate outcomes: failed audit-verdict.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at be6bacf. Requesting changes. There's one blocker.

Blocker: the Comments workflow fails on this head. Its "A package file's comment share may not rise" step flags both new doc comments:

  • packages/cli/src/telemetry/events.ts: 17.4% → 17.5% (the stopCode JSDoc line)
  • packages/cli/src/utils/checkPipeline.ts: 183 → 184 comment lines (the STOP_CODES JSDoc line)

Remove both lines, or move what they say into a name or type. For example, type stopCode as a StopCode union exported next to STOP_CODES instead of string. That also makes the closed set a compile-time guarantee, not just a comment.

Not blocking (scope note): budget or deadline stops that happen in the crops or finding-frames phases (withFindingCrops / withOptionalFindingFrames → runtimeFailure) are added after buildReport has already fired check_report. Those stops send no stop_code, the same way ok already shows the state from before crops. That only affects --snapshots / --finding-frames runs. Consider mentioning it in the PR body so nobody reads the count as complete.

Everything else checks out. The selection logic matches every code runtimeFailure produces when no override is passed. Those three codes are created nowhere else in packages/cli/src. Only the code is sent, never the message or path. The tests cover both a set and an omitted stop_code. This is a feat PR, so the "fails without the fix:" line isn't required.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 267 (base branch 267), smooth 131 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 021c04e. Approving.

  • The comments blocker is resolved. stopCode is now a StopCode union derived from STOP_CODES (as const), the two comment lines are gone, and the Comments workflow passes on this head.
  • The change from be6bacf7 is limited to the typing. The selection logic is unchanged: it still picks the first runtime finding whose code is in the set. events.ts imports from checkPipeline with import type, so there's no runtime cycle.
  • My earlier non-blocking note still applies: stops in the crops or finding-frames phases happen after check_report is sent, so they carry no stop_code.

The main CI, CodeQL and Windows workflows were still running when I reviewed. This approval doesn't stand in for them.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Nov 6, 2026
Merged via the queue into main with commit 9fcd06c Nov 6, 2026
206 of 214 checks passed
@miguel-heygen
miguel-heygen deleted the dobserve-checkcode branch November 6, 2026 10:02
SanHsien added a commit to SanHsien/hyperframes that referenced this pull request Nov 6, 2026
…view

Adopts heygen-com#7299 (accented Latin letters draw in a bundled family at its
bundled weight), heygen-com#7301 (installing Chrome on a busy Windows PC), heygen-com#7262
(cloud render logs a line a minute while a job holds its status), heygen-com#7265
(video starts a second early and catches up hidden) and heygen-com#7305
(check_report names the finding that stopped the check). Player packed
budget takes upstream's +1655 from heygen-com#7265 since the player bundles core's
media-steer. Defers heygen-com#7302; heygen-com#7261 and heygen-com#7263 do not apply. Watermark
advances to 9e2510b.

Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
Co-authored-by: WaterrrForever <miao.yang@heygen.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant