Skip to content

schema(claims): carry the reproducibility claim and its re-execution result (#366) - #370

Merged
imran-siddique merged 3 commits into
agentrust-io:mainfrom
chernistry:feat/schema-reproducibility-366
Sep 20, 2026
Merged

imran-siddique merged 3 commits into
agentrust-io:mainfrom
chernistry:feat/schema-reproducibility-366

Conversation

@chernistry

Copy link
Copy Markdown
Collaborator

What this changes

Section 3.1.4 merged in #364 (0ec0453) with the schema, the reference model and docs/schema.md left narrower than the text; #366 records the gap. This closes it in all three and adds the vectors that pin what the schema now holds. Schema change tracking a merged spec change, so no sponsor per CONTRIBUTING.

schema/trace-claim.json and src/agentrust_trace/schema/trace-v0.2.json (identical bytes, held by test_packaged_schema_matches_the_normative_schema):

  • top-level reproducibility: function, code_identity, optional code_resolver, input_closure as {id, digest, resolver} entries with every member required, transcript_digest; additionalProperties: false at both levels; the digest pattern is the one every other digest carries.
  • appraisal.method: the closed set re-execution.
  • appraisal.re_execution: outcome of reproduced, diverged, not-attempted; observed_digest; reason; optional verifier_code_identity.
  • the three presence rules the section states, as conditionals: re_execution present exactly when method is re-execution (if/then/else on appraisal), observed_digest required on diverged, reason required on not-attempted.

models.py: Reproducibility, ClosureEntry, ReExecution, Appraisal.method and Appraisal.re_execution, with the same three rules as validators, so the two artifacts agree. Exported from the package. test_the_schema_and_the_models_agree.py's FULL fixture now reaches the five new pattern constraints (15 in the schema, all mirrored verbatim); test_safe_integer_range.py's maximal record carries the block.

docs/schema.md: a top-level row, a reproducibility section with a reproducibility.input_closure entries table, the two appraisal rows, and an appraisal.re_execution members table. Enum values are inline code in their field's row.

examples/reproducibility-claim/: 21 signed vectors from a seeded key, generated by gen_reproducibility_vectors.py and held byte-for-byte by test_generators_reproduce_fixtures.py. Five accepted: a claim with no result, and one result per outcome (reproduced, diverged, two not-attempted: an unresolved closure entry, and a read beyond the closure). Sixteen rejected, two per rule (#124): result without method, method without result, diverged without observed_digest, not-attempted without reason, an incomplete closure entry, a method outside the closed set, an outcome outside it, an incomplete claim. transcript_digest and the closure digests are SHA-256 over the RFC 8785 bytes of JSON carried inline under context, so they recompute from the vector alone; vector 03's observed_digest is the same computation over a transcript that differs in one decision. code_identity and verifier_code_identity are seed-derived placeholders and the README says so: the set pins the shape of the claim and its result, not the re-execution, which needs a function and a closure from a producer that has both.

tests/test_reproducibility_claim_vectors.py holds every vector to its verdict on both validators, to the locus its code names and nowhere else (a vector rejected for an unrelated reason would otherwise read as coverage), to its signature, and to its digests. test_adequacy_all_sets.py grades the set like every other: both directions pinned, no code carried by one vector.

Two things worth a maintainer's eye

  1. Sequencing with test(docs): narrow field-and-enum coverage guard for schema and docs (#247) #365. Once the coverage guard lands, SECTION_OF_PATH in tests/test_schema_doc_coverage.py needs three entries, and then reports nothing undocumented. Verified on a local merge of the two branches:
    "reproducibility": "## `reproducibility`",
    "reproducibility.input_closure[]": "### `reproducibility.input_closure` entries",
    "appraisal.re_execution": "### `appraisal.re_execution` members",
    Whichever merges second, I will add them in the rebase.
  2. A rule the text does not state. Section 3.1.4 binds appraisal.re_execution to appraisal.method, and observed_digest and reason to outcome. It does not bind appraisal.method: "re-execution" to the presence of reproducibility on the record. A result with nothing on the record for it to be a result of is odd, but the schema follows the text and does not add the rule. If the section should say it, that is a spec change and I would open it separately.

Type of change

  • Editorial (typo, link fix, clarification: no normative effect)
  • Non-breaking spec change (new optional field, new platform profile, informative addition)
  • Breaking spec change (requires 14-day comment period and Project Lead sign-off)
  • Schema change
  • Example addition

Spec section

3.1.4 reproducibility, as merged in #364. No spec text changes here.

Checklist

  • DCO sign-off on all commits (git commit -s)
  • CHANGELOG.md updated
  • Breaking changes marked with <!-- CHANGED: #NNN: description --> in spec text (none: additive, optional members only)
  • Backward compatibility statement included (not a breaking change; a record without the new members validates exactly as before)

1742 passed, ruff, mypy and tools/check_dashes.py clean.

Closes #366.

…result (agentrust-io#366)

Section 3.1.4 merged in agentrust-io#364 with the schema, the reference model and
docs/schema.md left narrower than the text. This closes the gap in all
three, and adds the vectors that pin what the schema now holds.

schema/trace-claim.json and the packaged copy, identical bytes: a
top-level `reproducibility` object (function, code_identity, optional
code_resolver, input_closure entries of {id, digest, resolver} with every
member required, transcript_digest); `appraisal.method` as the closed set
`re-execution`; `appraisal.re_execution` (outcome of reproduced, diverged
or not-attempted, observed_digest, reason, optional
verifier_code_identity). The presence rules the section states are
conditionals: re_execution present exactly when method is re-execution,
observed_digest required on diverged, reason required on not-attempted.

models.py: Reproducibility, ClosureEntry, ReExecution, and the same three
rules as validators, so the two artifacts agree; the agreement test's
FULL fixture reaches the five new pattern constraints and the
safe-integer sweep's maximal record carries the new block.

docs/schema.md: a top-level row, a `reproducibility` section with its
closure-entry member table, the two appraisal rows and a
`appraisal.re_execution` member table.

examples/reproducibility-claim/: 21 signed vectors from a seeded key,
five accepted (a claim with no result, one result per outcome) and
sixteen rejected, two per rule. transcript_digest and the closure digests
are SHA-256 over the RFC 8785 bytes of JSON carried under `context`, so
they recompute from the vector alone; code_identity and
verifier_code_identity are seed-derived placeholders, stated as such.
tests/test_reproducibility_claim_vectors.py holds each vector to its
verdict on both validators, to the locus its code names and nowhere
else, to its signature, and to its digests; the set is graded by
tests/test_adequacy_all_sets.py like every other.

Signed-off-by: chernistry <sanderchernitsky@gmail.com>
imran-siddique
imran-siddique previously approved these changes Sep 20, 2026

@imran-siddique imran-siddique left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at the head I just pushed. The schema matches section 3.1.4 where it counts: diverged requires observed_digest, not-attempted requires reason, and re_execution is required exactly when method is re-execution. Both schema copies are byte-identical and the vector set carries accept and refuse cases for each rule.

Merging #365 half an hour ago left the new coverage guard with no section mapped for reproducibility, reproducibility.input_closure[] and appraisal.re_execution; your page already had the sections, so I pushed the three map entries. The suite is green with them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Schema narrower than spec after #364: reproducibility and appraisal.method/re_execution rejected

2 participants