feat: GitHub and GitLab repos on one board - #30
Merged
Merged
Conversation
repos.yml entries gain an optional `forge: gitlab`, inferred from the origin remote's host where the entry has a checkout. Both forges are read into the same `RemoteRepo` shape and appear on the same tiles and tables. Nothing about that record was GitHub-specific to begin with - default branch, protection, CI verdict, what is open - so this is a second collector rather than a second record. `localgit.py` needed nothing at all: it shells out to git. Three decisions worth knowing. *The history survives.* Adding a label to a metric family starts a new series and orphans the old one, and thirty of the forty families are keyed on `repo` alone - the history behind every trend panel and alert rule. So the forge goes on `jq_repo_info` and nowhere else: an `Always 1` join target whose label set churns anyway. Rendering one identical GitHub-only snapshot through both versions leaves 54 of 57 series byte-identical, and the three that move are all `*_info`. *Links come from the API.* The dashboard built 20 URLs by pasting the `repo` label onto `https://github.com/`. That cannot be right for two forges, and never had to be, because every API hands the URL over - `state.py` was already storing them and `metrics.py` simply never exposed them. Two new `*_info` families carry the rest. The alert runbooks now point at the dashboard drill-downs, since those rules fire off metrics labelled with `repo` alone and have nothing to join against for a per-forge address. *One remote pass, not one loop per forge.* `Store.update(remote=...)` replaces the whole map, so a loop each would have both threads publish only their own share. Health stays per forge, so a GitLab outage reads as `jq_collector_errors{source="gitlab"}` and leaves the GitHub repos standing. GitLab namespaces nest, so `repo` is the whole path - `acme/platform/infra/web`, not `infra/web`, which is what the old last-two-segments origin parser produced. That parser existed twice, in repos.py and localgit.py, both discarding the host; it is one function in origin.py now, and the host it keeps is what infers the forge for free. The same path on both forges is refused at startup rather than silently merged into one set of series. One panel group cannot be filled: GitLab's Dependabot counterpart is Ultimate-tier and its REST API is being retired, so those tiles read as unknown for GitLab repos - which the collector already distinguishes from "zero open alerts", so an unscanned repo never renders green. Verified end to end against live gitlab.com, including a four-segment subgroup: every rewritten panel query returns its URL labels, MR check states populate, and CI's own assertions still pass unmodified. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| self, | ||
| ref_cache: dict[str, tuple[str, str]], | ||
| coverage_cache: dict[str, tuple[int, tuple[float, int] | None]], | ||
| ) -> tuple[dict[str, RemoteRepo], frozenset[str]]: ... |
| coverage_cache: dict[str, tuple[int, tuple[float, int] | None]], | ||
| ) -> tuple[dict[str, RemoteRepo], frozenset[str]]: ... | ||
|
|
||
| def close(self) -> None: ... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
repos.ymlentries gain an optionalforge: gitlab, inferred from the origin remote's host where the entry has a checkout. Both forges are read into the sameRemoteReposhape and land on the same tiles and tables.Nothing about
RemoteRepowas GitHub-specific to begin with — default branch, protection, CI verdict, what's open — so this is a second collector, not a second record.localgit.pyneeded nothing at all: it shells out togit.Three decisions worth reviewing
The history survives. Adding a label to a metric family starts a new series and orphans the old one, and 30 of the 40 families are keyed on
repoalone — the history behind every trend panel and alert rule. So the forge goes onjq_repo_infoand nowhere else: anAlways 1join target whose label set churns anyway.Rendering one identical GitHub-only snapshot through both versions:
jq_merged_pull_request_timestamp_secondsinitially gained aurltoo — reverted, because that timestamp is the merged-PR timeline the panel reads. Its URL is on a sibling family instead. A test pins the census.Links come from the API. The dashboard built 20 URLs by pasting the
repolabel ontohttps://github.com/. That can't be right for two forges (GitLab spells it/-/merge_requests) and never had to be —state.pywas already storing the real URLs andmetrics.pysimply never exposed them. The alert runbooks now point at the dashboard drill-downs, since those rules fire off metrics labelled withrepoalone and have nothing to join against for a per-forge address.One remote pass, not one loop per forge.
Store.update(remote=...)replaces the whole map, so a loop each would have both threads publish only their own share and the board would flicker between halves. Health stays per forge, so a GitLab outage reads asjq_collector_errors{source="gitlab"}and leaves the GitHub repos standing.Nested namespaces
GitLab namespaces nest, so
repois the whole path —acme/platform/infra/web, not theinfra/webthe old last-two-segments origin parser produced (wrong project, 404ing API path). That parser existed twice, inrepos.pyandlocalgit.py, both discarding the host; it's one function inorigin.pynow, and the host it keeps is what infers the forge for free. The same path on both forges is refused at startup rather than silently merged into one set of series.Two things I got wrong and fixed
head_pipelineon the merge-request response, but only the single MR endpoint returns it — the listing omits the key. The first cut read that absence as "no pipeline" and reported every GitLab MR as unchecked. Caught by querying the live API rather than trusting the docs; now one follow-up per MR, same as GitHub, and the module docstring's "it is cheaper" claim is corrected to an honest cost breakdown.*binds tighter thanand,>and==, so appending thegroup_leftjoin to a bare comparison attached it to the wrong operand — silently dropping the labels on four panels and 400ing on five more. Caught by running every rewritten panel query against a live Prometheus.Known parity gap
Dependabot. GitLab's counterpart is dependency scanning and the vulnerability report — Ultimate tier, and the REST Vulnerability Findings API is being retired for GraphQL. Those three tiles read as unknown for GitLab repos. The collector already separates "the feature is off" from "zero open alerts", so an unscanned repo never renders green. Two smaller ones, documented in
docs/metrics.md: required-reviews is always 0 (Premium feature) and coverage has no line count (it's a pipeline field, which is genuinely cheaper — no artifact zip).Verification
pytest: 470 tests, 100% coverage on every module;ruff checkandformat --checkclean.gitlab-org/security-products/analyzers/semgrep: forge and both URL shapes correct per forge, one CI row per GitLab job, MR checks resolving tosuccess/failure/cancelled/none, coverage 75.6%, Dependabot honestly unknown.custom.hiddenconfirmed as the real "Hide in table" option in the pinned Grafana 11.3.1.Scope was agreed up front: gitlab.com only (self-hosted would need the API base per entry), one row per GitLab job, one PR.
🤖 Generated with Claude Code