Skip to content

feat: GitHub and GitLab repos on one board - #30

Merged
tschm merged 1 commit into
mainfrom
feat/gitlab-support
Aug 31, 2026
Merged

feat: GitHub and GitLab repos on one board#30
tschm merged 1 commit into
mainfrom
feat/gitlab-support

Conversation

@tschm

@tschm tschm commented Aug 31, 2026

Copy link
Copy Markdown
Member

repos.yml entries gain an optional forge: gitlab, inferred from the origin remote's host where the entry has a checkout. Both forges are read into the same RemoteRepo shape and land on the same tiles and tables.

repos:
  - path: ~/repos/jebel-quant/rhiza      # github.com, from the origin remote
  - path: ~/repos/acme/web               # gitlab.com, from the origin remote
  - repo: acme/platform/infra/web        # no checkout, so it has to say
    forge: gitlab

Nothing about RemoteRepo was GitHub-specific to begin with — default branch, protection, CI verdict, what's open — so this is a second collector, not a second record. localgit.py needed nothing at all: it shells out to git.

Three decisions worth reviewing

The history survives. Adding a label to a metric family starts a new series and orphans the old one, and 30 of the 40 families are keyed on repo alone — the history behind every trend panel and alert rule. So the forge goes on jq_repo_info and nowhere else: an Always 1 join target whose label set churns anyway.

Rendering one identical GitHub-only snapshot through both versions:

series byte-identical across the upgrade: 54 of 57
identity changed:  jq_ci_last_run_info, jq_pull_request_info, jq_repo_info   (all *_info)
newly added:       jq_ci_workflow_info, jq_merged_pull_request_info

jq_merged_pull_request_timestamp_seconds initially gained a url too — reverted, because that timestamp is the merged-PR timeline the panel reads. Its URL is on a sibling family instead. A test pins the census.

Links come from the API. The dashboard built 20 URLs by pasting the repo label onto https://github.com/. That can't be right for two forges (GitLab spells it /-/merge_requests) and never had to be — state.py was already storing the real URLs and metrics.py simply never exposed them. The alert runbooks now point at the dashboard drill-downs, since those rules fire off metrics labelled with repo alone and have nothing to join against for a per-forge address.

One remote pass, not one loop per forge. Store.update(remote=...) replaces the whole map, so a loop each would have both threads publish only their own share and the board would flicker between halves. Health stays per forge, so a GitLab outage reads as jq_collector_errors{source="gitlab"} and leaves the GitHub repos standing.

Nested namespaces

GitLab namespaces nest, so repo is the whole path — acme/platform/infra/web, not the infra/web the old last-two-segments origin parser produced (wrong project, 404ing API path). That parser existed twice, in repos.py and localgit.py, both discarding the host; it's one function in origin.py now, and the host it keeps is what infers the forge for free. The same path on both forges is refused at startup rather than silently merged into one set of series.

Two things I got wrong and fixed

  • MR checks are not free. GitLab documents head_pipeline on the merge-request response, but only the single MR endpoint returns it — the listing omits the key. The first cut read that absence as "no pipeline" and reported every GitLab MR as unchecked. Caught by querying the live API rather than trusting the docs; now one follow-up per MR, same as GitHub, and the module docstring's "it is cheaper" claim is corrected to an honest cost breakdown.
  • PromQL precedence. * binds tighter than and, > and ==, so appending the group_left join to a bare comparison attached it to the wrong operand — silently dropping the labels on four panels and 400ing on five more. Caught by running every rewritten panel query against a live Prometheus.

Known parity gap

Dependabot. GitLab's counterpart is dependency scanning and the vulnerability report — Ultimate tier, and the REST Vulnerability Findings API is being retired for GraphQL. Those three tiles read as unknown for GitLab repos. The collector already separates "the feature is off" from "zero open alerts", so an unscanned repo never renders green. Two smaller ones, documented in docs/metrics.md: required-reviews is always 0 (Premium feature) and coverage has no line count (it's a pipeline field, which is genuinely cheaper — no artifact zip).

Verification

  • pytest: 470 tests, 100% coverage on every module; ruff check and format --check clean.
  • Live end-to-end against gitlab.com with a mixed fleet including the 4-segment gitlab-org/security-products/analyzers/semgrep: forge and both URL shapes correct per forge, one CI row per GitLab job, MR checks resolving to success/failure/cancelled/none, coverage 75.6%, Dependabot honestly unknown.
  • Every rewritten panel query run through a live Prometheus: 15/15 succeed, every panel with rows carries its URL labels. custom.hidden confirmed as the real "Hide in table" option in the pinned Grafana 11.3.1.
  • CI's own assertions reproduced locally and pass unmodified — the startup log line CI greps is deliberately unchanged, with the forge split on a second line so a GitHub-only fleet logs exactly what it always did.

Scope was agreed up front: gitlab.com only (self-hosted would need the API base per entry), one row per GitLab job, one PR.

🤖 Generated with Claude Code

repos.yml entries gain an optional `forge: gitlab`, inferred from the origin
remote's host where the entry has a checkout. Both forges are read into the
same `RemoteRepo` shape and appear on the same tiles and tables.

Nothing about that record was GitHub-specific to begin with - default branch,
protection, CI verdict, what is open - so this is a second collector rather
than a second record. `localgit.py` needed nothing at all: it shells out to
git.

Three decisions worth knowing.

*The history survives.* Adding a label to a metric family starts a new series
and orphans the old one, and thirty of the forty families are keyed on `repo`
alone - the history behind every trend panel and alert rule. So the forge goes
on `jq_repo_info` and nowhere else: an `Always 1` join target whose label set
churns anyway. Rendering one identical GitHub-only snapshot through both
versions leaves 54 of 57 series byte-identical, and the three that move are
all `*_info`.

*Links come from the API.* The dashboard built 20 URLs by pasting the `repo`
label onto `https://github.com/`. That cannot be right for two forges, and
never had to be, because every API hands the URL over - `state.py` was already
storing them and `metrics.py` simply never exposed them. Two new `*_info`
families carry the rest. The alert runbooks now point at the dashboard
drill-downs, since those rules fire off metrics labelled with `repo` alone and
have nothing to join against for a per-forge address.

*One remote pass, not one loop per forge.* `Store.update(remote=...)` replaces
the whole map, so a loop each would have both threads publish only their own
share. Health stays per forge, so a GitLab outage reads as
`jq_collector_errors{source="gitlab"}` and leaves the GitHub repos standing.

GitLab namespaces nest, so `repo` is the whole path - `acme/platform/infra/web`,
not `infra/web`, which is what the old last-two-segments origin parser produced.
That parser existed twice, in repos.py and localgit.py, both discarding the
host; it is one function in origin.py now, and the host it keeps is what infers
the forge for free. The same path on both forges is refused at startup rather
than silently merged into one set of series.

One panel group cannot be filled: GitLab's Dependabot counterpart is
Ultimate-tier and its REST API is being retired, so those tiles read as unknown
for GitLab repos - which the collector already distinguishes from "zero open
alerts", so an unscanned repo never renders green.

Verified end to end against live gitlab.com, including a four-segment subgroup:
every rewritten panel query returns its URL labels, MR check states populate,
and CI's own assertions still pass unmodified.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
self,
ref_cache: dict[str, tuple[str, str]],
coverage_cache: dict[str, tuple[int, tuple[float, int] | None]],
) -> tuple[dict[str, RemoteRepo], frozenset[str]]: ...
coverage_cache: dict[str, tuple[int, tuple[float, int] | None]],
) -> tuple[dict[str, RemoteRepo], frozenset[str]]: ...

def close(self) -> None: ...
@tschm
tschm merged commit e405d14 into main Aug 31, 2026
6 checks passed
@tschm
tschm deleted the feat/gitlab-support branch August 31, 2026 10:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant