Skip to content

CI: the HRX backend is compiled by no job at PR time - hw-gate is skipped on every PR, including both pin bumps #1077

Description

@bong-water-water-bong

What

hw-gate.yml is the only CI job that compiles the HRX backend (GGML_HRX is OFF by default in
ggml/CMakeLists.txt:216, and ci.yml's build never enables it). It runs only when a PR carries the hw-gate label or
its branch starts with bump-hrx/:

github.event_name == 'workflow_dispatch' ||
(… same-repo PR … && (contains(github.event.pull_request.labels.*.name, 'hw-gate') ||
                       startsWith(github.event.pull_request.head.ref, 'bump-hrx/')))

Every merged PR tonight had hw_gate: SKIPPED, including both llama.cpp pin bumps:

#1039  branch 1bit/pin-llama-817d081   hw_gate=SKIPPED
#1065  branch 1bit/npu-1010-reload-hint hw_gate=SKIPPED
#1066  branch feat/laya-checkpoint-default hw_gate=SKIPPED
#1067  branch npu/gdn-region-count      hw_gate=SKIPPED
#1075  branch bump/llama-pin-6e42864a6  hw_gate=SKIPPED    <- mine, today, before I added the label

So ggml/src/ggml-hrx — the whole HRX backend, including host-weight-placement.{h,cpp} which fork #174 just changed —
is compiled by no CI job at PR time. The only HRX builds happening are the ones a lane runs locally against its own
toolchain, which is exactly the shape of gap that let gpu-kernels #52 sit on main for two days (four kernels shipping
readable symbols, caught only by a local tools/check.sh).

Evidence that the mechanism works when used

Adding the label to #1075 started the real thing immediately, on this box's self-hosted runner:

$ gh api -X POST repos/1bit-MONSTER/engine/issues/1075/labels -f "labels[]=hw-gate"
run 41983120230  hw_gate (job 127097553885)  -> queued, runs-on [self-hosted, strixhalo, pr-agent]

Suggested

  1. bump-hrx/ is the convention the workflow already wants and nobody uses. Both pin bumps in this repo used
    1bit/pin-… / bump/…. Either rename the convention to what lanes actually type, or use the label — but as it stands
    the auto-trigger is dead code.
  2. Label every PR that touches third_party/llama.cpp or ggml/src/ggml-hrx. A pin bump moves both halves; the
    non-HRX half is covered by ci.yml and this one is not.
  3. If running the gate on every such PR is too expensive (it takes under 20 minutes on the owner's box and contends for
    the GPU/NPU), say so in docs/hw-gate.md and name the one PR class that must carry it — a pin bump is the obvious
    candidate, because it changes the source of a backend that no other job compiles.

I am the PR-review lane; I found this while checking my own pin bump (#1075) and have labelled it. No code change proposed
here — the gate exists and works, it just is not being asked to run.

Activity

  1. bong-water-water-bong commented on Nov 6, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Disposition: parked on an owner policy decision — the mechanism works, the trigger does not.

    • The gate runs correctly when asked: adding the hw-gate label to third_party: bump llama.cpp to 6e42864a6 (3 fork commits; supersedes #1073, which re-bumps an already-merged pin) #1075 queued run 41983120230 on the self-hosted strixhalo runner, as recorded in the thread.
    • The issue's three options are policy, not code: (1) make the bump-hrx/ auto-trigger real (rename the convention to what lanes type, or rely on the label — startsWith('bump-hrx/') is currently dead code); (2) label every PR that touches third_party/llama.cpp or ggml/src/ggml-hrx; (3) accept the cost and document the one PR class that must carry it (a pin bump is the obvious one).
    • The trade-off is GPU time on the shared box (the gate takes under 20 min but competes with the lanes that use the box), which is the owner's call.

    Unpark: pick 1, 2 or 3; the workflow condition (or docs/hw-gate.md) change is then small.

  2. bong-water-water-bong commented on Nov 6, 2026

    @bong-water-water-bong
    CollaboratorAuthor

    Sweep (pi-verify) at 918f731a — still reproduces

    The trigger is unchanged and no PR-time job enables the backend:

    $ grep -rn "GGML_HRX" .github/workflows/*.yml
       (no hit in ci.yml; only bump-hrx.yml mentions hrx, and it syncs pins, it does not compile)
    $ sed -n '57,62p' .github/workflows/hw-gate.yml
        if: |
          github.event_name == 'workflow_dispatch' ||
          (… && (contains(…labels.*.name, 'hw-gate') ||
                 startsWith(…head.ref, 'bump-hrx/')))
    $ grep -n "cmake -B build" .github/workflows/ci.yml
        69:  run: cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=gcc-16 -DCMAKE_CXX_COMPILER=g++-16
    

    ci.yml:69 passes no -DGGML_HRX=ON, so a PR touching ggml/src/ggml-hrx or third_party/llama.cpp still gets
    hw_gate: SKIPPED unless someone labels it. Nothing in the workflow moved since the issue was filed; the fix is still the
    owner's pick of option 1/2/3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions