Skip to content

ci: the test matrix stops cancelling itself, and runs on macOS for the first time - #591

Merged
irparent merged 1 commit into
mainfrom
ci/matrix-fail-fast-and-macos
Sep 22, 2026
Merged

irparent merged 1 commit into
mainfrom
ci/matrix-fail-fast-and-macos

Conversation

@irparent

Copy link
Copy Markdown
Member

The matrix could not see its own answers

Dependabot #308 (better-sqlite3 12 → 13) has been held open since August on a note that says the segfault reproduces on Node 20, 22 and 24, and that it breaks the container build. Both claims are wrong, and this PR fixes the thing that made them wrong rather than the bump itself.

What actually happened. The test matrix is fail-fast. When the Node 20 cell went red, GitHub cancelled the Node 22 and Node 24 cells before they ran. The full job list for the last re-test (run 35697296879) reads:

failure    test (20)
cancelled  test (22)
cancelled  test (22, node)
cancelled  test (24, node)
success    docker-build

cancelled is not failed. Those three cells have no result at all, and docker-build — the other thing the note calls broken — passed.

What is true. Reproduced against better-sqlite3@13.0.3 on Ubuntu 24.04 / glibc 2.39 / x86_64, the same OS and C library ubuntu-latest runs, with the four-line script from WiseLibs/better-sqlite3#1514:

Node N-API Result
20.20.2 9 SIGSEGV inside new Database() — nothing printed, core dumped
22.23.2 10 open ok / close ok, exit 0
24.21.0 10 open ok / close ok, exit 0

Also clean on Debian bookworm with Node 25.9.0, and on node:22-alpine and node:24-alpine — musl prebuilds now ship in the package, so the "no prebuilt binary for alpine, falls through to node-gyp" half of the note is stale too.

So the real constraint is a single one: better-sqlite3 13 declares engines: node >=22, and Iris declares >=20.0.0. That is a supported-runtime decision, and it is handled separately.

Why this PR does not just drop Node 20

Because making that check green would be the wrong kind of green.

Every runs-on: line in this repository — all thirty of them — is ubuntu-latest. Upstream #1514 reports version 13 crashing on darwin-arm64 on Node 20 and Node 22, and the Linux runs above show Node 22 is fine there. That difference is real and platform-specific, and a Linux-only matrix is structurally incapable of seeing it. Drop Node 20, watch CI go green, merge the bump, and Iris crashes on startup for a developer on an Apple Silicon Mac running Node 22 — with nothing in CI to say so.

What changes

  • fail-fast: false on the test matrix. One red cell no longer cancels its siblings.
  • A native Node 24 cell (test (24)). The two existing Node 24 cells both run the built-in node:sqlite driver; the native addon has never been measured on 24.
  • A macOS cell (test (22, macOS)), native driver, macos-latest — Apple Silicon. The first non-Linux cell Iris has ever had.

The two new cells report but are not required checks, so this PR changes nothing about what blocks a merge. Making them required is a branch-protection change, and it should follow evidence that they are green rather than precede it.

Gates

Workflow-only; no source, no version, no rendered surface. The matrix expands from four cells to six and the existing four keep their exact context names — test (20), test (22), test (22, node), test (24, node) — so the required set is untouched.

🤖 Generated with Claude Code

…e first time

Three changes to one job, all of them about what the matrix can SEE.

fail-fast: false. Dependabot #308 has sat open for six weeks on the reading
that better-sqlite3 13 segfaults on Node 20, 22 and 24. It does not. The
Node 22 and 24 cells never ran: fail-fast cancelled them the moment the
Node 20 cell went red, and `cancelled` was recorded as `failed`. A matrix
that hides its other answers when one cell fails is worse than no matrix.

A native Node 24 cell. Node 20 reached end of life on 2026-04-30 and
better-sqlite3 13 declares `engines: node >=22`, so 24 is the line Iris
stands on next. Nothing has ever measured the native addon there — the two
existing Node 24 coverage cells both run the built-in driver.

A macOS cell. All thirty `runs-on:` lines in this repository say
ubuntu-latest, so the native SQLite addon has never been exercised on Apple
Silicon in CI at all. WiseLibs/better-sqlite3#1514 reports that version 13
crashes on darwin-arm64 on Node 20 AND 22 — while on linux-x64 Node 22 is
fine. A Linux-only matrix cannot see that difference, which means it would
go green on a bump that crashes Iris on boot for a developer on a Mac.

The two new cells are not required checks; this PR only makes them report.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
website Ready Ready Preview Sep 22, 2026 3:26pm UTC

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Iris gate — 1 of 2 tripped --fail-on detector_veto

iris-eval ingest: 3 stored, 1 tripped --fail-on detector_veto (2 of 3 evaluated in dataset "release-gate")

Trace Basis Rules Evidence
1d1a4a192bb32a26aec1e29fece200ae detector_veto no_pii no_pii: AWS Access Key (output 45–65)
Verdict basis Traces
detector_veto 2
clean 1

Unjudged questions: task_completed (3), tool_use_correct (3) — a trace that did not carry what a rule needs.

tests/fixtures/ci-gate/traces.ndjson · 3 evaluated · dataset release-gate: 2 in the gate · exit 1 · what the bases mean

@github-actions

Copy link
Copy Markdown

Iris gate — 1 stored, nothing tripped --fail-on any

iris-eval ingest: 1 stored, 0 tripped --fail-on any

Verdict basis Traces
clean 1

Unjudged questions: task_completed (1), tool_use_correct (1) — a trace that did not carry what a rule needs.

tests/fixtures/ci-gate/clean.ndjson · 1 evaluated · exit 0 · what the bases mean

@irparent
irparent merged commit 02971b2 into main Sep 22, 2026
27 checks passed
@irparent
irparent deleted the ci/matrix-fail-fast-and-macos branch September 22, 2026 15:34
irparent added a commit that referenced this pull request Sep 22, 2026
…no second SQLite driver (#592)

* chore: drop Node 20 — an end-of-life runtime, and the only line with no second SQLite driver

Node 20 reached end of life on 2026-04-30 and has had no security fixes
since. Iris has kept declaring `>=20.0.0` and testing it in CI for the five
months after, which is a promise the runtime itself stopped keeping.

It is also the one supported line with no fallback. `node:sqlite` needs
Node 22.13 or later, so the storage driver seam added in 0.15.0 — native
addon, built-in on failure — has only ever had one driver on Node 20. A
prebuild that will not load there is still a dead start.

And it is the whole of what blocks Dependabot #308: better-sqlite3 13
declares `engines: node >=22` and segfaults inside `new Database()` on Node
20 (N-API 9) while returning cleanly on 22, 24 and 25. Dropping 20 does not
merge that bump — the macOS cell added in #591 decides it, since upstream
reports a darwin-arm64 crash on Node 22 that a Linux matrix cannot see —
but it removes the reason that had nothing to do with the crash.

Surfaces: `engines` in the server and @iris-eval/init, the CI matrix
(22 and 24 native, both on the built-in, plus macOS), the dev.to crosspost
workflow's runner, the README's requirement line and its Node.js section,
the Python client's quickstart, CONTRIBUTING's required-checks table, and
the storage driver test's comment about which cell exercises the fallback.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: the floor is 22.13, the first Node that ships node:sqlite

Red team on the previous commit. The stated reason for dropping Node 20 was
that it is the only supported line with no second storage driver — but a
floor of >=22.0.0 does not make that true. `node:sqlite` arrives in
22.13.0, which is Iris's own NODE_SQLITE_MIN constant, so Node 22.0 through
22.12 have no fallback either. The claim and the number disagreed.

22.13 makes it true: every supported runtime now ships the built-in driver,
so the seam added in 0.15.0 always has something to fall back to when the
native addon will not load.

It also costs nothing. 22.13.0 shipped 2025-01-06 and the current 22.x is
22.23.2 — twenty months and ten minors ago. And it is the number arc 9's
own acceptance matrix recorded as the eventual floor.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — 7c797c5f Deployed Sep 22, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant