Repository navigation
feat(schema): add top-level manifest reference to v0.3 draft schema (#483) - #538
Conversation
…gentrust-io#483) Add the optional top-level manifest {id, digest, media_type} object to schema/trace-claim-v0.3-draft.json per the reference-only ruling on agentrust-io#483, preserve properties.manifest in scripts/gen_v03_draft_schema.py, and validate the agentrust-io#487 fixtures and malformed manifest shapes in tests/test_v03_draft_schema.py. Signed-off-by: Raghuvar Durga <177662415+drk-ram@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
imran-siddique
left a comment
There was a problem hiding this comment.
Approving 75b5fbb. gen_v03_draft_schema.py --check passes and test_v03_draft_schema.py runs 27 of 27. I broke seven constraints in the manifest block one at a time (type on id, minLength on media_type, the digest pattern twice, additionalProperties, required, the whole member) and a targeted test failed for each. The shape matches the reference-only ruling on #483 and agent-manifest §6.3.2 as merged in 3c02529.
Two follow-ups, not blocking. §6.3.2 names the digest as sha256 only, and ManifestRef in trace-token-experimental-v1.json still requires version; #483 asked for one shape. Please change "Fixes: #483" to "Refs": no conformance module reads the reference yet.
dinakarjs
left a comment
There was a problem hiding this comment.
Independent schema/interoperability audit of 75b5fbb. No blocking defect found within this schema-and-generator PR's stated scope. Current-head CI is green. Python 3.12.14 / hashed development and conformance locks: reference 4,055 passed / 48 skipped; conformance 778 passed / 5 xpassed. Generator --check, root Ruff and mypy pass.
Executed compatibility matrix against the actual v0.3 manifest subschema and experimental token's $defs.ManifestRef:
| Reference | v0.3 draft | Experimental token ManifestRef |
|---|---|---|
| sha256, application/json, no version | accepts | refuses |
| sha256, application/agent-manifest+cose, no version | accepts | refuses |
| sha384, application/agent-manifest+cose, no version | accepts | refuses |
| sha256, application/agent-manifest+cose, version 0.2 | refuses | accepts |
These are deliberately different schemas today, not interchangeable reference objects. Agent Manifest section 6.3.2 at merged 3c02529 requires sha256 and defines the digest preimage: distributed COSE bytes for v0.2, manifest_hash_in_report preimage for v0.1. Accepting a digest-shaped value here does not establish that profile or those bytes were checked.
Additional concrete coverage gap for #483: the fixture-only checker in conformance/tests/test_manifest_ref_draft_vectors.py hardcodes sha256. I rebuilt and correctly re-signed two otherwise identical v0.3 records over the fixture resolver bytes. Both validate against this draft schema and their Ed25519 signatures verify. With the matching sha256 digest, check() reports manifest='established'; with the matching sha384 digest, it reports 'manifest-digest-mismatch'. This is a limitation of the existing test reference checker, not evidence that a production manifest evaluator mishandles sha384: no conformance module evaluates the reference yet.
Keep #483 open. Its remaining implementation/profile decisions include digest-algorithm support, convergence or explicit separation from the experimental shape, and evaluator integration. A schema pass must not be reported as manifest verification. The existing Refs wording is therefore correct; the record signature, digest binding, manifest validity and producer assertion remain separate results.
|
#544 is stacked on this PR. It re-signs fixtures 01-05 as v0.3 records, so your schema tests can read them without patching |
…gentrust-io#538 follow-up) agentrust-io#538's schema tests read these fixtures but patch eat_profile to v0.3 first, which breaks each record's signature, so no checker could run the schema, the signature and the digest binding on the same bytes (dinakarjs's item 3 on agentrust-io#483). 01-05 are now signed with the v0.3 profile tag and validate against trace-claim-v0.3-draft.json as signed; 03 is rejected for its signed manifest_verification_result, on purpose; 06 stays the v0.2 control. Two tests pin this without any patch. Generator writes UTF-8 with newline="". Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: babyblueviper1 <babyblueviperbusiness@gmail.com>
…gentrust-io#538 follow-up) agentrust-io#538's schema tests read these fixtures but patch eat_profile to v0.3 first, which breaks each record's signature, so no checker could run the schema, the signature and the digest binding on the same bytes (dinakarjs's item 3 on agentrust-io#483). 01-05 are now signed with the v0.3 profile tag and validate against trace-claim-v0.3-draft.json as signed; 03 is rejected for its signed manifest_verification_result, on purpose; 06 stays the v0.2 control. Two tests pin this without any patch. Generator writes UTF-8 with newline="". Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: babyblueviper1 <babyblueviperbusiness@gmail.com>
…digest algorithm (sha256 or sha384) (#551) * conformance: sign manifest-reference fixtures 01-05 as v0.3 records (#538 follow-up) #538's schema tests read these fixtures but patch eat_profile to v0.3 first, which breaks each record's signature, so no checker could run the schema, the signature and the digest binding on the same bytes (dinakarjs's item 3 on #483). 01-05 are now signed with the v0.3 profile tag and validate against trace-claim-v0.3-draft.json as signed; 03 is rejected for its signed manifest_verification_result, on purpose; 06 stays the v0.2 control. Two tests pin this without any patch. Generator writes UTF-8 with newline="". Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: babyblueviper1 <babyblueviperbusiness@gmail.com> * docs: consolidate v0.3 fixture README's byte-identity statement The README stated the records were byte-identical to 2cd4ae7 in one paragraph, then said they were no longer byte-identical in the next (re-signed for the profile change) -- merge into one accurate statement. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs: house style, no em dash in the v0.3 fixture README Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * conformance: manifest-reference checker recomputes with the declared digest algorithm (sha256 or sha384) The fixture checker always hashed sha256, so a correctly signed v0.3 record whose digest is sha384 (the draft schema accepts both) reported manifest-digest-mismatch (dinakarjs, #483 audit point 1). It now uses the algorithm the signed digest names; any other algorithm is manifest-digest-unsupported, not a mismatch. New fixture 07-digest-matches-sha384 (v0.3, schema-valid as signed) and a fifth wrong checker, always_sha256, caught by 07 only. 01-06 are byte-identical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * conformance: wrap the manifest-ref checker docstring (E501) and say five wrong checkers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * conformance: README nits (seven fixtures, explicit repo-root regen path) Addresses @rajnisht7's review on PR #551: the fixture count stayed at "Six" after 07 was added, and the regen command's path only works from conformance/ while the script itself lives one level up. * conformance: lock the manifest-digest-unsupported outcome with a unit test A signed v0.3 record whose digest is the correct sha512 of the manifest it ran under must report manifest-digest-unsupported: not established, not a mismatch. A unit test, not a fixture, because the draft schema rejects a sha512 digest, so the record is not schema-valid as signed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Signed-off-by: babyblueviper1 <babyblueviperbusiness@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: babyblueviper1 <federico@invinoveritas.dev>
What this changes
Refs #483
Contributor: Raghuvar Durga (@drk-ram)
Adds the optional top-level
manifestobject ({id, digest, media_type}) toschema/trace-claim-v0.3-draft.jsonper the reference-only ruling on #483 and updatesscripts/gen_v03_draft_schema.pyto preserveproperties.manifestalongsideproperties.eat_profileandproperties.runtime.properties.evidence:schema/trace-claim-v0.3-draft.json:manifestobject with"required": ["id", "digest", "media_type"]and"additionalProperties": false:id:{"type": "string", "minLength": 1}digest:{"type": "string", "pattern": "^sha(256:[0-9a-f]{64}|384:[0-9a-f]{96})$"}, matching the digest pattern acrossschema/trace-claim.jsonand serving as the target ofruntime.evidence.binds: "manifest-digest"media_type:{"type": "string", "minLength": 1}manifest_verification_resultout of the schema per v0.3: let a record name the agent manifest it ran under #483 andagentrust-io/agent-manifest#498(reference-only: a producer-asserted verification result is not carried in the record).scripts/gen_v03_draft_schema.py:_copy_member()sobuild(canonical, draft)preserves bothproperties.runtime.properties.evidenceandproperties.manifestat their declared positions inschema/trace-claim-v0.3-draft.json.tests/test_v03_draft_schema.py:test_draft_differs_from_canonical_only_in_its_declared_membersto stripproperties.manifest.01,02,04,05, and06fromconformance/tests/vectors/manifest-ref-v0.3-draft/(conformance: v0.3 draft fixtures for the manifest reference (#483) #487) validate againstschema/trace-claim-v0.3-draft.jsonunder the v0.3 profile whileschema/trace-claim.jsoncontinues to rejectmanifeston v0.2 records.03-mismatch-with-asserted-result.jsonis rejected byschema/trace-claim-v0.3-draft.jsononmanifest_verification_resultand validates oncemanifest_verification_resultis removed.manifest(type: "object", requiredid/digest/media_type,type: "string",minLength: 1,sha256/sha384lowercase hex digest patterns, andadditionalProperties: false).Cold-reader review:
A separate
code-reviewagent inspected the initial diff against #483,conformance/tests/vectors/manifest-ref-v0.3-draft/, andCONTRIBUTING.mdand found four items that we updated before opening this PR:test_draft_rejects_malformed_manifestinitially omitted non-string test cases forid,digest, andmedia_type; because JSON SchemaminLengthandpatternignore non-strings, removing"type": "string"from any of those three properties would not have failed the test. Addednon-string-id,non-string-digest, andnon-string-media-typecases.test_draft_rejects_malformed_manifesttestedsha384:positively (test_draft_accepts_sha384_manifest_digest) but only tested negative digest patterns onsha256:andsha512:. Addedsha384-uppercaseandsha384-wrong-lengthcases so weakening the384:[0-9a-f]{96}alternation branch fails the test.test_draft_accepts_manifest_reference_fixturesandtest_draft_rejects_malformed_manifest, tightened assertions to verify that06-control-no-manifest.jsonpassescanonical(schema/trace-claim.json), thatcanonicalerrors on01/02/04/05specifically namemanifest, and thatdrafterrors on malformedmanifestinputs occur atabsolute_path[:1] == ["manifest"], with explicitpytest.param(..., id=...)names.manifest.properties.digest.descriptionwithmanifest.descriptionandmanifest.properties.id.description("the manifest bytes this execution ran under").Type of change
Spec section
None in
spec/trace-v0.2.md. Targetsschema/trace-claim-v0.3-draft.jsonfor #483 alongside the #487 draft fixtures inconformance/tests/vectors/manifest-ref-v0.3-draft/.Tests for a fix
Each constraint in
schema/trace-claim-v0.3-draft.jsonunderproperties.manifest(and the top-level refusal ofmanifest_verification_result) has a corresponding test intests/test_v03_draft_schema.pythat fails when switched off:manifest {id, digest, media_type}on v0.3 records (fails ifproperties.manifestis removed):tests/test_v03_draft_schema.py::test_draft_accepts_manifest_reference_fixtures[01-digest-matches.json]tests/test_v03_draft_schema.py::test_draft_accepts_manifest_reference_fixtures[02-digest-mismatch.json]tests/test_v03_draft_schema.py::test_draft_accepts_manifest_reference_fixtures[04-unresolvable.json]tests/test_v03_draft_schema.py::test_draft_accepts_manifest_reference_fixtures[05-added-after-signing.json]tests/test_v03_draft_schema.py::test_draft_accepts_sha384_manifest_digestmanifest_verification_resulton v0.3 records:tests/test_v03_draft_schema.py::test_draft_rejects_asserted_manifest_verification_resultmanifestvalues (each fails if the corresponding constraint inproperties.manifestis switched off):manifest.type: "object":tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[non-object]manifest.required(id):tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[missing-id]manifest.required(digest):tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[missing-digest]manifest.required(media_type):tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[missing-media-type]manifest.properties.id.type: "string":tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[non-string-id]manifest.properties.digest.type: "string":tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[non-string-digest]manifest.properties.media_type.type: "string":tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[non-string-media-type]manifest.properties.id.minLength: 1:tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[empty-id]manifest.properties.media_type.minLength: 1:tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[empty-media-type]manifest.properties.digest.pattern(sha256lowercase hex and 64-char length):tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[sha256-uppercase],tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[sha256-wrong-length]manifest.properties.digest.pattern(sha384lowercase hex and 96-char length):tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[sha384-uppercase],tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[sha384-wrong-length]manifest.properties.digest.pattern(supported algorithm prefix):tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[unsupported-digest-algorithm]manifest.additionalProperties: false:tests/test_v03_draft_schema.py::test_draft_rejects_malformed_manifest[unexpected-property]Validation commands run:
python scripts/gen_v03_draft_schema.py --check-> passed (exit 0)pytest tests/test_v03_draft_schema.py tests/test_runtime_evidence_vectors.py tests/test_safe_integer_range.py -v-> passed (182 passed)pytest conformance/tests -q --no-cov-> passed (778 passed, 5 xpassed)ruff check src tests scripts-> passed (All checks passed!)python tools/check_dashes.py-> passed (no dashes outside examples/, spec/trace-v0.1.md)mypy src/agentrust_trace-> passed (Success: no issues found in 15 source files)Checklist
git commit -s)CHANGELOG.mdupdated (for any normative change)<!-- CHANGED: #NNN: description -->in spec textRefs: #483