Repository navigation
feat(scripts): snapshot pinned contract details schemas and flag breaking changes - #1580
Merged
Merged
Conversation
…king changes Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Contributor
📦 Bundle Size Report
Size Limits
Largest Files (Top 5)
View All Files (291 total)
✅ Bundle size check passed |
Contributor
|
Deploy preview for remote-flows ready!
Deployed with vercel-action |
Contributor
|
Deploy preview for adp-cost-calculator ready!
Deployed with vercel-action |
Contributor
📊 Coverage Report✅ Coverage increased! 🎉
Detailed BreakdownLines Coverage
Statements Coverage
Functions Coverage
Branches Coverage
✅ Coverage check passed |
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… feat/schema-snapshot-guard
The baselines are behaviour fingerprints, not scripts, and a top-level snapshots folder leaves room for other flows to add their own. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Changes under scripts/schema-canary/ trigger the Schema Canary workflow on PRs. The fingerprint and baseline helpers belong to the snapshot check, so they now live in scripts/schema-snapshot/. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…tests A new optional field with a limit is no longer reported as breaking, and a choice that stops showing the fields it used to show is breaking instead of informational. The nightly issue text no longer blames the backend alone. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
gabrielseco
marked this pull request as ready for review
October 16, 2026 06:58
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit d215739. Configure here.
…reaking Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…jsf 1.2.19 The fieldset fixture declared the conditional field as `false` in the base schema and tried to reveal it in `then`. jsf 1.2.19 (pulled in by kit 1.1.4) keeps a `false` property subschema when a branch adds a constraint to it, so the field never became visible. Declare the real field in the base and hide it in `else`, like the other fixtures. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
cammellos
approved these changes
Oct 16, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Adds a nightly check that the contract details schema served for each version this library pins still behaves the same as when we last committed it, and reports a breaking difference if not.
Why
A pinned schema version is meant to be frozen, but the backend can change what it serves for a version without publishing a new one. That happened when a backend flag was removed: three fields became required for Spain on an already-published version, and we only found out through a failing e2e test on unrelated PRs.
The existing nightly canary didn't catch it, because it auto-fills whatever the schema asks for and then submits. It proves a form can still be saved, not that the schema is the same.
Comparing the schema text isn't enough either: conditions and component references expand on the server, so the served JSON has to be built into a form and compared by what a user would see.
What changed
Toggle details
scripts/schema-snapshot/behaviour.ts: builds the served schema the way the Onboarding flow does and records a fingerprint: which fields are visible and required with an empty form, what each single radio, select or checkbox choice adds or removes, and each field's input type, allowed values and length/number limits. Titles, descriptions and ordering are not recorded.scripts/schema-snapshot.ts(npm run schema-snapshot): fetches the pinned version for every EOR onboarding country from the sandbox gateway and compares againstsnapshots/contract-details/<COUNTRY>.v<N>.json.--updaterewrites the baselines,--country=XXXlimits the run,--create-missingonly adds baselines that don't exist yet. Breaking differences, a missing baseline or a fetch error exit with code 1. Files it writes are formatted withoxfmt, so they passcheck-format.snapshots/contract-details/: baselines for the 92 countries at their current pins (about 930 KB). The version is in the file name, so a bumped pin is never compared against the old version's baseline.--create-missing, which writes the baseline for the new version, keeps the older ones so a reverted pin still has its baseline, commits it to the PR branch as a bot, and comments on the PR with how the new version differs from the previous one. It never overwrites an existing baseline, so a pinned version that changed still fails the nightly check. It does not run for PRs from forks, which have no access to the secrets.scripts/schema-snapshot/baselines.ts: small helpers to list baselines and find the previous one..github/workflows/schema-snapshots.yml: the nightly job compares every country and opens or updates aschema-snapshotissue, closing it when the check passes again. The PR job above reuses the existing canary GitHub App and credentials. Cancellation of in-progress runs is off so the bot's own push can't cancel the run that is still posting the comment.scripts/schema-snapshot/behaviour.test.tsandbaselines.test.ts: cover a newly required field, an optional field that becomes required, a conditional that starts requiring another field, a tightened limit, and description-only changes producing no diff, for bothuseHeadlessFormstrategies.Known limits: it only sees what the sandbox serves, uses fixed scenario values (choices that depend on employer data, dates or salary bands can go unseen), and tries one choice at a time, not combinations. Pins that point at a version the backend still edits in place are held to the same rules.
No public API change. Everything lives under
scripts/,snapshots/and.github/.Screenshots
N/A
Related Resources
scripts/schema-canary.ts(the existing nightly canary)Testing
example/app in a browser: N/A, no app code changed.Ran
npm run schema-snapshot -- --updateagainst the sandbox for all 92 countries, then a second comparison run: 89 matched and 3 differed because of aNaNlimit that doesn't survive JSON. That is fixed, and those 3 now match on re-run.Replaying the Spain change (the three fields removed from
requiredin the baseline, then compared with what the sandbox serves now) reports all three as "now required".Simulated a bump locally by renaming ESP's baseline to an older version and running
--create-missing: it wrote the new file and reported that the new version behaves the same as the old one; a second run found nothing to do. The bump PR job itself has not run in CI yet, so its first real run is on the first PR that bumps a pin.Mutation checks: with the choice-by-choice scenarios disabled, the two conditional tests fail; with required-field detection disabled, the two required-field tests fail.
🤖 Generated with Claude Code
Note
Medium Risk
New CI and bot commits guard onboarding contract-details pins and depend on sandbox secrets; no runtime app API change, but false negatives/positives could affect release confidence for form behavior.
Overview
Adds contract details schema snapshotting so pinned JSON schema versions are compared to committed baselines by form behavior (visibility, required fields, conditionals, constraints)—not raw JSON—catching silent backend changes the submit canary would miss.
Introduces
npm run schema-snapshotandscripts/schema-snapshot/(fingerprint/diff via the same headless form path as Onboarding, baseline helpers, tests), stores per-country/version fingerprints undersnapshots/contract-details/, and markssnapshots/**as linguist-generated.CI: a scheduled workflow runs the compare against sandbox, opens/updates a
schema-snapshotissue on breaking drift, and closes it when green; on PRs that touch pins, a bot job runs--create-missing, commits new baselines, and comments with diffs vs the previous version (reuses existing canary sandbox credentials and GitHub App).Minor: oxlint copy now refers to schema scripts under
scripts/generally.Reviewed by Cursor Bugbot for commit 5694ec0. Bugbot is set up for automated code reviews on this repo. Configure here.