Repository navigation
A rivet project for the gates, and a release spine for the plugin - #216
Merged
Merged
Conversation
Five plugin files changed after v0.32.0 was set in 6a5ab57 — the README, three memory files and the repo-hygiene skill — so two different trees carried that version. The plugin cache keys on it, which makes the second tree silently never arrive for anyone who already installed the first.
This repo publishes `pulseengine-claude` and had 0 tags and 0 releases while rivet, varve and witness each had 30. The version a user installs was bound to nothing, and five plugin files had changed since v0.32.0 was set with no bump. ## The V, scoped to the gates Nine requirements, one per CI gate, each with a `verification` carrying a `verifies` link and — for eight of them — a `# rivet: verifies REQ-...` marker in the gate script, so `rivet verify` cites both the artifact link and the line of code. `rivet validate` passes with 0 warnings and all nine reach `verified`. The scope is deliberately the gates and nothing else. Each one is already a mechanical oracle with a red-first history, so the mapping is tight rather than invented; nine requirements that genuinely reach `verified` are worth more than sixty sitting at `implemented`. The website content and the eighteen skills are not modelled yet. `REQ-CLAIMS-HOLD` is covered by its artifact link but carries no source marker: `claim-check.py` is the plugin's shared reference implementation, used by other repos, so stamping it with a pulseengine.eu requirement id would be wrong. Coverage is 8/9 and says so rather than being forced to 9/9. No `externals:`. Cross-repo externals are why sigil's release has been dead since 2026-09-08 (pulseengine/sigil#284) — the sibling checkouts do not resolve in CI, every link into them reports zero targets, and the release fails on its own compliance step. Not importing that here. rivet is pinned to v0.39.0, the version the signed varve layer ships, rather than latest. A project that validates on one version and runs another in CI is the same shape as a gate that passes against the wrong instrument, which this session hit twice. The install sequence was run end to end locally with only the platform pattern changed. ## The release spine `check-plugin-version-current.py` makes the bump mandatory: it finds the commit that set the current version and fails if any plugin file changed after it. Red first against the real defect — it reported the five files that had drifted — then green after the bump to 0.33.0. An uncommitted bump is treated as in flight rather than as an error. `release.yml` fires when that version changes on main. The bump is the trigger, so there is no separate "cut a release" step to forget, and the two ends meet: content cannot change without a bump, and a bump cannot land without a release. It closes the V before it ships, not after, and `--strict-empty` means a config typo that loads zero artifacts fails instead of reporting 100%. It does not use an `on: release` trigger anywhere. A release created with GITHUB_TOKEN does not start one — the recursion guard that cost synth 42 releases with no compliance report (pulseengine/synth#1453). The workflow says so where someone would otherwise add it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
Two corrections to the spine, both to match what the other repos already do. The trigger was a version bump on main, which invented a convention. rivet, varve, witness and synth all use `on: push: tags: ["v*"]` with a `workflow_dispatch` that takes an existing tag, and all four say the same thing: the tag must already exist, the workflow does not create tags. This now does the same, so a release is cut by bumping the manifest, merging, and pushing the tag. The tag and the manifest are checked against each other. A tag that disagrees with plugin.json would publish bytes under a version they do not carry, so the job refuses it and says which commit to tag instead. The release body is a generated note, not a hand-written one. `rivet release notes` implements Automotive SPICE 11-03 from the artifact store and the commit trailers, which is what rivet uses for its own releases. It is generated in CI at the tagged commit so what ships is what the trace says, with the same guards rivet uses: rivet itself refuses a note over an empty scope, and the step refuses one that is empty or does not begin with a release-note heading. An empty note would ship as the release body and read as though nothing had been delivered. Artifacts are scoped to the release, without which the note has nothing to describe. `rivet release status 0.33.0` now reports 19 of 19 verified and "Cuttable — every artifact is release-ready". The verification artifacts are `verified` rather than `implemented`. Every gate here was red-first checked, run against the defect it exists to catch and seen to fail, before being wired into CI. That is verification of the test itself and the reason `gate-potency` is in this plugin. Left at `implemented` the note reported them as withheld, which misdescribes a gate that is running and has been proven potent. Also: no `gh` anywhere. The self-hosted runners do not have it — the first CI run of this branch died on `gh: command not found`, the same thing that made the report freshness check silently examine nothing in #214. Both the rivet install and the release publish use curl against public URLs and the REST API, and both JSON generators were executed locally against real inputs rather than read. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
avrabe
added a commit
that referenced
this pull request
Oct 14, 2026
* feat(skills): name the epoch a claim expires in, and gate output over access
Two additions from external research, each tied to something this repo hit.
**claim-verification gains a fourth move: name the epoch.** mark -> bind ->
gate catches a document drifting from its evidence. It does not catch the
evidence drifting out from under a claim that still greens, because a claim's
validity is bounded in time and context and nothing recorded that bound.
The case was in this repo on 2026-10-14. The operating contract's routing rule
carried its epoch in prose ("no model versions — this file named two that were
superseded") while the gate bound only the sentence. Editing the rule tripped
the gate on wording, and the claim had meanwhile narrowed: the string was
checked, the scope was not. Gating the narrower claim then immediately found a
stale version pinned in a second document. An epoch with no proof surface is a
comment.
This is DO-330's requalification trigger applied to prose, and it now has
published vocabulary: "epoch limits" and "proof surfaces", from Fifty Years of
Specification Completeness (arXiv:2606.25120), which maps DO-178C traceability
and DO-330 triggers onto AI governance documents and finds 37% of them below a
structural-quality threshold, with no standards body requiring these properties
of a governance document. Worth adopting because it converts "we have a claim
gate" into a term a certification audience already holds.
**oracle-gate-a-change gains the access-versus-output rule.** Restricting what
an agent may touch is the intuitive control and the weaker one. A 2026
systematisation of 39 agent execution-security papers (arXiv:2607.05743)
reports 69-98% of real denylists failing, and benign out-of-scope agent actions
at rates up to 17.1% under realistic prompting — a class no access-control
paper in its corpus addresses, because the action is not malicious, only
outside the task. A permission boundary cannot catch a well-intentioned edit to
the wrong thing; an oracle over the diff can. The sandbox stays; it does not
substitute for the gate.
Plugin 0.35.0 -> 0.36.0: 0.35.0 is already installed locally, so changed skill
content needs its own version or the two trees are indistinguishable to anyone
who has it (check-plugin-version-current, #216).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
* docs(blog): the standards cover AI in the product, not AI in the toolchain
Measured across 40 standards pages fetched and read. Separates the two
questions that get conflated: certifying a system that CONTAINS machine
learning, versus qualifying an agent that WRITES code for a certified
product. The first has a pipeline. The second has almost nothing.
For AI in the toolchain, exactly one document's scope explicitly covers
using AI to build safety-related functions: ISO/IEC TR 5469:2024, which
is informative, so nothing can be assessed against it. The second item
is a draft VDA Guidelines 3rd-edition chapter reported second-hand,
which requires review of AI-generated output before acceptance and does
not say how. DO-330/ED-215 is from January 2012 with no AI material, and
SC-240's accepted plan for future work has one open revision item, which
is COTS at Levels A and B. ISO 26262-8 has sat at "to be revised" since
8 July 2024. ISO/PAS 8800 states on its own ISO page that it "does not
give specific guidelines for software tools that use AI".
The post then argues the draft ASPICE sentence names the right obligation
and leaves the mechanism open, with the 2026 literature on why review by
reading does not close it: LLM-as-judge misses 26% of failures an
executable spec test catches (arXiv:2605.26457); ~25% of test-passing
patches admit counterexamples and only 56% of agent-written specs pass
audit (arXiv:2609.21190); Dafny RL learns specification hacking, 58.1%
dropping to 9.7-31.1% once underspecified tasks are filtered
(arXiv:2605.30914); 6 of 144 Isabelle-accepted repairs modified protected
text (arXiv:2608.13459); and proofs compile and pass a `sorry`-token scan
while depending on `sorryAx` (arXiv:2608.28639).
Our dates are stated as two, not rounded into one: substrate from the
first week of March 2026 (rivet 7 March, spar 8 March, rules_rocq_rust
11 January, rules_verus 20 February), agent methodology from 24 May 2026
with the V-closure release gate on 2 June. It says explicitly that we do
not predate the Five Eyes agentic guidance of 30 April and 1 May 2026,
and that securing agents is a different axis from assuring their output.
Includes both of our own gate failures, because a post about mechanical
review that omits them is not credible: spar's Lean gate passing 12 of 12
sorries through a self-exemption its own authors wrote, and the still-open
residue that a token scan cannot see a transitive `sorryAx` dependency
where `#print axioms` can.
Prose conventions: 0 em-dashes, no single-line beat paragraphs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
* docs(blog): correct ten factual errors found by review, add bibliography and figure
Three reviewer passes on the post before publishing. The fact-check pass
fetched 27 primary sources and checked 47 assertions: 20 verified, 4 wrong,
23 unverifiable. Every error it found is corrected here, and the
unverifiable ones are either softened to what a source actually supports or
named in a new "what we could not establish" section.
The four errors:
1. The arXiv:2605.30914 figures were presented as a before and after on one
metric. They are two different experiments. 58.1% is the verified reward
after RL (up from 2.2%), and 9.7% to 31.1% is a RISE on a separately
filtered benchmark. Rewritten to say so.
2. The ISO/PAS 8800 quotation was not verbatim. It reads "This document does
not provide specific guidelines for software tools that use AI methods";
the post had "give" for "provide" and dropped "methods" inside quotation
marks. Now exact.
3. The VDA Guidelines 3rd edition was called a draft. VDA's publications
page dates it July 2026 after a public review that closed 12 May 2026, so
it is published. The chapter content remains second-hand and is now
flagged harder, and its claim softened from "requires review before
acceptance" to "needs verification before it is accepted".
4. The S-CORE standards list was attributed to the wrong page. ASPICE
appears on the project's own site; its Eclipse project page lists only
the two ISO standards.
Claims withdrawn rather than softened: that DO-330 "contains no AI material",
which we never read the standard to establish. What survives is what we did
read, which is SC-240's accepted forward-work plan having one open revision
item, and it being COTS. The DO-330 date is now given as 2011 or 2012 because
catalogues disagree. ED-324's consensus date, Q4 target and IDAL C scope are
attributed to the EASA AI Days deck. TS 22440's stage is marked second-hand
because iso.org returns 403. The Five Eyes date drops to 1 May 2026, the only
one CISA's own page carries. The EU AI Act FAQ now carries the Commission's
own "preliminary" caveat.
Also from review:
- Prose pass (6 rules, 7 tells, counted mechanically): 8 violations fixed.
One single-line beat paragraph, five adjectives where numbers belong, one
not-X-but-Y, one stock closer. Body prose is at 0 em-dashes; the 3 in the
file are inside verbatim ISO titles, where the em-dash is part of the
official title and removing it would falsify the citation.
- An 18-entry bibliography, split into research, standards and projects, which
states per entry whether we read the thing or are relying on a second-hand
report, plus a section naming the four things that resisted checking:
iso.org and faa.gov 403s, an autosar.org TLS failure, and the fact that our
negative claims about working groups rest on published pages rather than
minutes.
- A themed inline SVG comparing the two spaces, built on the site's CSS custom
properties so light mode needs no patching. It had a real bug: blank lines
inside the block ended it under CommonMark and injected 10 <p> tags into the
<svg>, which browsers do not render as SVG children. Collapsed to one
contiguous block, verified at 35 text nodes and 0 injected paragraphs. My
first oracle for that missed it by matching the first <svg> on the page,
which is site chrome.
The figure carried two claims the prose had just withdrawn ("NO AI MATERIAL",
"stage 30.60"). Both corrected, and the figure is now checked for stale claims
alongside the prose.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
* docs(blog): my self-criticism was false — ordeal runs the axiom gate
Adversarial review found the worst possible error: the post's second
admitted failure claimed we run no axiom-level audit and that four
repositories have no equivalent gate. Both false.
ordeal/lean/AxiomCheck.lean exists for exactly this purpose. Each
`#guard_msgs` pins the exact axiom list of a capstone theorem, so a new
axiom changes the message and fails elaboration, and the file is a default
lake target so the required Lean CI job fails with it. Its header says the
sorry budget is guarded separately and this file "guards everything
`sorry`-shaped guards miss", naming sorryAx and native_decide.
Cause, which is now in the post: I grepped the repositories checked out on
this machine, and ordeal and gale are not among the 22 that are. The
instrument could not see two of the relevant repositories, returned
silence, and I read silence as a measurement. That is check 5 of our own
verify-your-own-claims procedure, committed by its author.
A false self-criticism is worse than an overclaim. It converts a real
engineering asset into apparent performance, and it is falsifiable from our
own public repositories in one search.
Also corrected:
- "strictly stronger" was false as a formal relation. An axiom audit over
selected capstones misses a sorry in a file nothing imports; a token scan
misses an inherited sorryAx. They are incomparable, and the post now says
so.
- "counts every form" contradicted itself two paragraphs later. spar counts
sorry, sorryAx, admit and a bare axiom over comment-stripped text, and
does not match native_decide.
- The 2 June 2026 date was presented as a traceability gate. That commit
touches six files, all Markdown and JSON: it is a procedure an agent is
told to follow, not a check that fails. By this post's own argument it does
not count. The enforced gate is older — rivet audit, 18 May 2026, 799
lines of Rust with CLI tests (rivet #297).
- The two-date section claimed we were building on this question before
knowing it was a gap. Typed traceability and an AADL parser are not
agent-specific, and TR 5469 predates all of it by two years. Deflated to
dates rather than priority.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
* docs(blog): address the remaining adversarial findings
Second and third reviewer passes closed. All five blockers and most of the
should-fix list are now in.
- The central negative is no longer defended. "Exactly one document's scope
covers it" becomes "one published document whose scope we could actually
read", and the post names its own likely counterexample: ISO/IEC TS 22440
is reported to carry the same scope bullet about using AI to design,
develop and verify safety-related functions, and it is a Technical
Specification with a Part 1 titled Requirements. We could not confirm it,
because iso.org 403s us and the committee mirror shows 22440-1 with the
scope preview blank. An unprovable negative is not worth keeping.
- The DER's first objection is answered rather than left open. DO-178C
qualifies a tool only when its output is not otherwise verified, and
ISO 26262-8's tool confidence level falls when malfunctions are likely to
be detected. Both are technology-neutral and both already cover an agent.
The post now says so, and relocates its own gap to the right place: every
framework makes verifying the output the hinge, and none says what the
verification has to produce.
- Eclipse S-CORE is no longer characterised as publishing nothing. It
inherits the Eclipse Foundation's Generative AI Usage Guidelines, v1.0 of
April 2024, which require a committer to put generated output through the
project's normal vetting. The stronger "validate all generated code before
submission" wording is a proposed template, not an adopted rule, and the
post says which is which. This makes the gap sharper: three frameworks all
say verify the output, none says what it produces.
- The working-group sweep is dated 14 October 2026, names ELISA's two groups,
narrows to "no published workstream on qualifying agent-written code
against a safety standard", and concedes the CNCF counterexample along with
the fact that published pages are weaker evidence than minutes.
- Smaller corrections: the description carries the second-hand hedge again,
since it is the text that gets quoted; "four 2026 results" no longer
implies anyone measured human review, because none of them did; a scope
exclusion is described as a boundary rather than as a body admitting a gap;
COVESA's "exactly this" and "will decide" are softened to "closest to it"
and "will shape"; the 56% is scoped to its own study's audit; "safe"
becomes "sufficient", since safety is a system property; and the spar
failure now carries its issue number.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
* docs(blog): unify the correction section to the post's 'we' voice
The new section describing the false self-criticism was written in first
person singular. The post is bylined, so 'I got it wrong' would read as the
author admitting an error made during agent-assisted drafting. Changed to
the 'we' the rest of the post uses, which is accurate at the level that
matters: the org would have published it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This repo publishes
pulseengine-claudeand had 0 tags and 0 releases, whilerivet,varveandwitnesseach had 30. The version a user installs was bound to nothing.The V, scoped to the gates
Nine requirements, one per CI gate, each with a
verificationcarrying averifieslink and — for eight — a# rivet: verifies REQ-…marker in the script itself, sorivet verifycites both the artifact link and the line of code.Scope is deliberately the gates and nothing else. Each is already a mechanical oracle with a red-first history, so the mapping is tight rather than invented. Nine requirements that genuinely reach
verifiedbeat sixty sitting atimplemented. Website content and the eighteen skills are not modelled yet.8/9, not 9/9, on purpose.
REQ-CLAIMS-HOLDhas its artifact link but no source marker:claim-check.pyis the plugin's shared reference implementation used by other repos, so stamping it with a pulseengine.eu requirement id would be wrong. The number says so rather than being forced.No
externals:. Cross-repo externals are exactly why sigil's release has been dead since 2026-09-08 (pulseengine/sigil#284). Not importing that here.rivet pinned to v0.39.0 — the version the signed varve layer ships, not
latest. A project that validates on one version and runs another in CI is the same shape as a gate passing against the wrong instrument, which this session hit twice (CARGO_BIN_NAMEunder cargo vs Bazel; the freshness check withghlocally vs absent on the runner). The install sequence was run end to end locally with only the platform pattern swapped.The release spine
check-plugin-version-current.pymakes the bump mandatory — it finds the commit that set the current version and fails if any plugin file changed after it. Red first against the real defect:Then green after the bump to 0.33.0. An uncommitted bump is treated as in flight, not as an error.
release.ymlfires when that version changes on main. The bump is the trigger, so the two ends meet: content cannot change without a bump, and a bump cannot land without a release. It closes the V before shipping, with--strict-emptyso a config typo that loads zero artifacts fails instead of reporting 100%.It deliberately uses no
on: releasetrigger. A release created withGITHUB_TOKENdoes not start one — the recursion guard that cost synth 42 releases with no compliance report (pulseengine/synth#1453). The workflow says so at the point where someone would otherwise add it.Note on review
The first release this produces will be
v0.33.0. Worth a look at the workflow before it runs unattended, since nothing in this repo has ever cut a release.All gates pass locally, plus
rivet validateand the two new checks.🤖 Generated with Claude Code
https://claude.ai/code/session_01G5qpB7zvxLvdrvn5YRyj4R