Repository navigation
schema(claims): carry the reproducibility claim and its re-execution result (#366) - #370
Merged
imran-siddique merged 3 commits intoSep 20, 2026
Conversation
…result (agentrust-io#366) Section 3.1.4 merged in agentrust-io#364 with the schema, the reference model and docs/schema.md left narrower than the text. This closes the gap in all three, and adds the vectors that pin what the schema now holds. schema/trace-claim.json and the packaged copy, identical bytes: a top-level `reproducibility` object (function, code_identity, optional code_resolver, input_closure entries of {id, digest, resolver} with every member required, transcript_digest); `appraisal.method` as the closed set `re-execution`; `appraisal.re_execution` (outcome of reproduced, diverged or not-attempted, observed_digest, reason, optional verifier_code_identity). The presence rules the section states are conditionals: re_execution present exactly when method is re-execution, observed_digest required on diverged, reason required on not-attempted. models.py: Reproducibility, ClosureEntry, ReExecution, and the same three rules as validators, so the two artifacts agree; the agreement test's FULL fixture reaches the five new pattern constraints and the safe-integer sweep's maximal record carries the new block. docs/schema.md: a top-level row, a `reproducibility` section with its closure-entry member table, the two appraisal rows and a `appraisal.re_execution` member table. examples/reproducibility-claim/: 21 signed vectors from a seeded key, five accepted (a claim with no result, one result per outcome) and sixteen rejected, two per rule. transcript_digest and the closure digests are SHA-256 over the RFC 8785 bytes of JSON carried under `context`, so they recompute from the vector alone; code_identity and verifier_code_identity are seed-derived placeholders, stated as such. tests/test_reproducibility_claim_vectors.py holds each vector to its verdict on both validators, to the locus its code names and nowhere else, to its signature, and to its digests; the set is graded by tests/test_adequacy_all_sets.py like every other. Signed-off-by: chernistry <sanderchernitsky@gmail.com>
This was referenced Sep 18, 2026
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
imran-siddique
previously approved these changes
Sep 20, 2026
imran-siddique
left a comment
Member
There was a problem hiding this comment.
Approving at the head I just pushed. The schema matches section 3.1.4 where it counts: diverged requires observed_digest, not-attempted requires reason, and re_execution is required exactly when method is re-execution. Both schema copies are byte-identical and the vector set carries accept and refuse cases for each rule.
Merging #365 half an hour ago left the new coverage guard with no section mapped for reproducibility, reproducibility.input_closure[] and appraisal.re_execution; your page already had the sections, so I pushed the three map entries. The suite is green with them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Section 3.1.4 merged in #364 (
0ec0453) with the schema, the reference model anddocs/schema.mdleft narrower than the text; #366 records the gap. This closes it in all three and adds the vectors that pin what the schema now holds. Schema change tracking a merged spec change, so no sponsor per CONTRIBUTING.schema/trace-claim.jsonandsrc/agentrust_trace/schema/trace-v0.2.json(identical bytes, held bytest_packaged_schema_matches_the_normative_schema):reproducibility:function,code_identity, optionalcode_resolver,input_closureas{id, digest, resolver}entries with every member required,transcript_digest;additionalProperties: falseat both levels; the digest pattern is the one every other digest carries.appraisal.method: the closed setre-execution.appraisal.re_execution:outcomeofreproduced,diverged,not-attempted;observed_digest;reason; optionalverifier_code_identity.re_executionpresent exactly whenmethodisre-execution(if/then/elseonappraisal),observed_digestrequired ondiverged,reasonrequired onnot-attempted.models.py:Reproducibility,ClosureEntry,ReExecution,Appraisal.methodandAppraisal.re_execution, with the same three rules as validators, so the two artifacts agree. Exported from the package.test_the_schema_and_the_models_agree.py'sFULLfixture now reaches the five new pattern constraints (15 in the schema, all mirrored verbatim);test_safe_integer_range.py's maximal record carries the block.docs/schema.md: a top-level row, areproducibilitysection with areproducibility.input_closureentries table, the twoappraisalrows, and anappraisal.re_executionmembers table. Enum values are inline code in their field's row.examples/reproducibility-claim/: 21 signed vectors from a seeded key, generated bygen_reproducibility_vectors.pyand held byte-for-byte bytest_generators_reproduce_fixtures.py. Five accepted: a claim with no result, and one result per outcome (reproduced,diverged, twonot-attempted: an unresolved closure entry, and a read beyond the closure). Sixteen rejected, two per rule (#124): result without method, method without result,divergedwithoutobserved_digest,not-attemptedwithoutreason, an incomplete closure entry, amethodoutside the closed set, anoutcomeoutside it, an incomplete claim.transcript_digestand the closure digests are SHA-256 over the RFC 8785 bytes of JSON carried inline undercontext, so they recompute from the vector alone; vector03'sobserved_digestis the same computation over a transcript that differs in one decision.code_identityandverifier_code_identityare seed-derived placeholders and the README says so: the set pins the shape of the claim and its result, not the re-execution, which needs a function and a closure from a producer that has both.tests/test_reproducibility_claim_vectors.pyholds every vector to its verdict on both validators, to the locus its code names and nowhere else (a vector rejected for an unrelated reason would otherwise read as coverage), to its signature, and to its digests.test_adequacy_all_sets.pygrades the set like every other: both directions pinned, no code carried by one vector.Two things worth a maintainer's eye
SECTION_OF_PATHintests/test_schema_doc_coverage.pyneeds three entries, and then reports nothing undocumented. Verified on a local merge of the two branches:appraisal.re_executiontoappraisal.method, andobserved_digestandreasontooutcome. It does not bindappraisal.method: "re-execution"to the presence ofreproducibilityon the record. A result with nothing on the record for it to be a result of is odd, but the schema follows the text and does not add the rule. If the section should say it, that is a spec change and I would open it separately.Type of change
Spec section
3.1.4
reproducibility, as merged in #364. No spec text changes here.Checklist
git commit -s)CHANGELOG.mdupdated<!-- CHANGED: #NNN: description -->in spec text (none: additive, optional members only)1742 passed,
ruff,mypyandtools/check_dashes.pyclean.Closes #366.