Take the workflow list from GitHub, not from run history - #2
Merged
Merged
Conversation
Run history outlives the workflow file, so deriving "which workflows exist" from it reports jobs that are long gone - and if the last run of a vanished workflow failed, the repo is red forever for something nobody can fix. Jebel-Quant/platform had sat red for 12 weeks on `.github/workflows/latex.yml`, a workflow deleted in June. GitHub displays orphaned runs under their file path rather than a name, which is why it surfaced with a .yml "name" at all. Across the fleet this was 1 of 8 red repos; the other 7 were real. Filtering those out exposed the same fault in a second guise: a *renamed* workflow appears under both names, so the old name's last run lingers just as the deleted one did. Workflow id 286602591 on that repo produces runs called both "Build PDF" and "Build vision.pdf". Both come from treating run history as the source of truth. So: - /actions/workflows is consulted for the authoritative set, and runs whose workflow is no longer active are skipped. Disabled workflows go too - a switched-off job is not a failing one. - Runs are keyed by workflow_id, giving one series per real workflow, and labelled with the name the workflow has *now*. Where two active workflows share a display name, both fall back to their path so the metric labels stay unique - the earlier name-keyed version exported duplicate label sets and Prometheus silently dropped 16 samples per scrape. - If the listing cannot be read the filter is skipped entirely, so a transient API error over-reports rather than blanking a repo's CI. Costs one extra call per repo per refresh: measured 4718/5000 remaining. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Run history outlives the workflow file. Deriving which workflows exist from it reports jobs that are long gone — and if the last run of a vanished workflow failed, the repo is red forever for something nobody can fix.
Jebel-Quant/platformhad sat red for 12 weeks on.github/workflows/latex.yml, a workflow deleted in June. GitHub displays orphaned runs under their file path rather than a name, which is why it surfaced with a.yml"name" at all — the tell that it wasn't a real workflow.Across the fleet this was 1 of 8 red repos. The other 7 are real and unaffected.
A second instance, found while fixing the first
Filtering out deleted workflows exposed the same fault in another guise: a renamed workflow appears under both names, so the old name's last run lingers exactly as the deleted one did. Workflow id
286602591on that repo produces runs called bothBuild PDFandBuild vision.pdf.Both stem from treating run history as the source of truth for what exists.
The change
/actions/workflowsis consulted for the authoritative set. Runs whose workflow is no longer active are skipped. Disabled workflows go too — a switched-off job is not a failing one.workflow_id, giving one series per real workflow, labelled with the name the workflow has now. Where two active workflows share a display name, both fall back to their path so labels stay unique — the earlier name-keyed version exported duplicate label sets and Prometheus silently dropped 16 samples per scrape.Verification
Run from a clean
git archiveof the branch, against live GitHub:platformBuild PDFonly, 0 redEvery remaining failure was cross-checked against
/actions/workflowsand confirmed to be a workflow that still exists.Costs one extra API call per repo per refresh — measured 4718/5000 remaining.
🤖 Generated with Claude Code