Skip to content

feat(scripts): submit each country's contract details in the schema canary - #1491

Merged
gabrielseco merged 29 commits into
mainfrom
feat/schema-canary-submit
Oct 13, 2026
Merged

gabrielseco merged 29 commits into
mainfrom
feat/schema-canary-submit

Conversation

@gabrielseco

@gabrielseco gabrielseco commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Extends the contract details schema canary to submit filled values for every country and check the employment saved them unchanged.

Why

  • A schema that builds isn't enough: it also has to submit, and the backend has to store what we sent.
  • The read-back already found a real bug (GBR's overtime_eligible was dropped after save; Tiger has since fixed it).
  • Split out of feat(scripts): add a contract details schema canary #1481 so the build/validate canary and the submit check can be reviewed separately.

What changed

Toggle details

Adds a submit check after pinned and latest: fills the pinned schema (scripts/fill-schema.ts plus per-country seeds), runs it through the SDK's build, validation, money conversion and parsing, sends the flow's PATCH /v1/employments/{id}, then reads the employment back. Fails the run on any value saved differently or not at all. Skipped when the pinned schema fails to build.

Read-back exceptions are narrow:

  • Forced values (const equal to default, via checkFieldHasForcedValue) are skipped only when missing.
  • KNOWN_UNSAVED_FIELDS lists per-country fields the backend drops. It's empty now that Tiger saves GBR overtime_eligible again. An entry fails once the value is saved again, so it gets removed.

CONTRACT_DETAILS_SEEDS moves to scripts/contract-details-seeds.ts, shared with seed-onboarding (adds a FRA seed). vitest.config.ts also picks up scripts/**/*.test.tsx.

Also adds the submit check to the PR canary job from #1493.

Screenshots

N/A

Related Resources

Testing

  • Tested against the example/ app in a browser: N/A, this is a script.

  • Feature flag: N/A

  • Unit tests (MSW): scripts/schema-canary/submit-contract-details.test.ts, plus the submit cases in lib.test.ts.

  • Locally: npm run schema-canary -- --country=FRA (needs .env.sandbox). All 92 countries pass pinned and submit against sandbox.

🤖 Generated with Claude Code


Note

Medium Risk
New sandbox PATCH/GET integration in CI can fail on backend persistence mismatches; logic is test-heavy and gated on pinned build success, but it exercises live employment APIs with secrets.

Overview
Adds a third submit check to the contract-details schema canary: after pinned/latest build validation, it auto-fills the pinned schema (shared CONTRACT_DETAILS_SEEDS, deterministic seedFor per country), runs the same headless-form build/validation/parsing as Onboarding, PATCHes sandbox employments, and GETs them back to assert saved contract_details match what was sent. Submit is skipped when pinned build fails; forced-only-missing fields and empty KNOWN_UNSAVED_FIELDS allow narrow read-back exceptions.

PR CI now runs --checks=pinned,submit (workflow path filter includes the new seeds file). Seeds move to scripts/contract-details-seeds.ts (adds FRA) for reuse by seed-onboarding. Unit coverage lives in submit-contract-details.test.ts and updated lib.test.ts; Vitest picks up scripts/**/*.test.tsx.

Reviewed by Cursor Bugbot for commit cf57420. Bugbot is set up for automated code reviews on this repo. Configure here.

gabrielseco and others added 6 commits October 8, 2026 16:37
…React

buildHeadlessForm, parseValuesForValidation and withFieldsetObjects move
from useHeadlessForm into src/common/headlessForm.ts, so a script can run
the exact build and validate path the hook uses. useHeadlessForm calls
them with no change in behaviour.

An oxlint rule keeps the two builders importable only by useHeadlessForm,
scripts/ and tests, so they don't become a second way to build forms.

Onboarding gets CONTRACT_DETAILS_STRATEGY and
getContractDetailsStrategy(country), so the flow and anything that needs
to mirror it pick the useHeadlessForm strategy from the same function.

No public API changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
npm run schema-canary checks every EOR country from /v1/countries
against the sandbox gateway:

- seed: creates a sandbox employment, so the schema is fetched with a
  real employment_id like the flow does; archived afterwards, including
  when a later setup step fails.
- pinned: builds the contract_details version the example app pins with
  the same useHeadlessForm strategy Onboarding uses, and validates.
- latest: the same at json_schema_version=latest, warning only.
- submit: fills the schema, runs the values through the SDK's build,
  validation, money conversion and parsing, sends the flow's PATCH, then
  reads the employment back and fails on any value saved differently or
  not at all.

Forced values (const equal to default) are skipped in the read-back only
when missing. KNOWN_UNSAVED_FIELDS lists per-country fields the backend
drops (GBR overtime_eligible) and fails once they are saved again, so
the entry gets removed when the fix ships.

CONTRACT_DETAILS_SEEDS moves to scripts/contract-details-seeds.ts,
shared with seed-onboarding. Runs locally with .env.sandbox; CI comes in
the next PR.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Nightly (04:00 UTC) and manual: every check with --write. Opens or
  updates an auto-PR (chore/schema-canary-report-auto) when
  scripts/reports/schema-canary.json changes; the report has no
  timestamp, so a quiet night opens nothing. The first nightly creates
  the file. On failure it opens or updates a schema-canary issue with
  the failing rows, and closes it once the canary is green again.
- PRs: pinned and submit checks when anything the canary reads changes.

Uses the SCHEMA_CANARY_CLIENT_ID / _CLIENT_SECRET / _REFRESH_TOKEN repo
secrets (company-scoped refresh token).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The canary in this PR now only builds and validates the pinned and latest
schemas. The submit check (fill, PATCH, read back) moves to a stacked PR so
each one is smaller to review.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…anary

Adds the "submit" check: fill the pinned schema with fake values, run them
through the same SDK form build, validation and parsing the Onboarding flow
uses, PATCH the employment, then read it back and compare. Seeds move to
scripts/contract-details-seeds.ts so seed-onboarding shares them, and
KNOWN_UNSAVED_FIELDS covers fields Tiger accepts but doesn't save.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Deploy preview for adp-cost-calculator ready!

Project:adp-cost-calculator
Status: ✅  Deploy successful!
Preview URL:https://adp-cost-calculator-inqxbdfn4-remotecom.vercel.app
Latest Commit:cf57420

Deployed with vercel-action

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Deploy preview for remote-flows ready!

Project:remote-flows
Status: ✅  Deploy successful!
Preview URL:https://remote-flows-inq7w212s-remotecom.vercel.app
Latest Commit:cf57420

Deployed with vercel-action

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

📊 Coverage Report

⚪ Coverage unchanged

Metric Current Previous Change Status
Lines 89.58% 89.58% 0% ⚪
Statements 89.46% 89.46% 0% ⚪
Functions 88.47% 88.47% 0% ⚪
Branches 81.21% 81.21% 0% ⚪

Detailed Breakdown

Lines Coverage
  • Covered: 5393 / 6020
  • Coverage: 89.58%
  • Change: 0% (0 lines)
Statements Coverage
  • Covered: 5458 / 6101
  • Coverage: 89.46%
  • Change: 0% (0 statements)
Functions Coverage
  • Covered: 1427 / 1613
  • Coverage: 88.47%
  • Change: 0% (0 functions)
Branches Coverage
  • Covered: 3345 / 4119
  • Coverage: 81.21%
  • Change: 0% (0 branches)

✅ Coverage check passed

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

gabrielseco and others added 4 commits October 9, 2026 07:51
The submit check moved to its own PR, so the PR job runs the pinned check
on its own. Runs on the same ref now queue instead of overlapping, so two
canaries never seed sandbox employments or push the report branch at once.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gabrielseco
gabrielseco changed the base branch from feat/schema-canary-node to ci/schema-canary October 9, 2026 05:53
gabrielseco and others added 6 commits October 9, 2026 07:59
All runs share the same sandbox credentials and rate limit, so canaries
from different PRs in a stack hit 429s when they run at the same time.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Runs every check and refreshes the report and failure issue right after a
relevant merge, instead of waiting for the next nightly run. Nightly stays,
since it is what catches backend schema changes that land without a commit
here.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With the default single pending slot, a new run replaced the one waiting
in the shared group, so stacked PR, main and nightly runs could be dropped.
queue: max keeps them all waiting in order.

A job-level timeout marks the run cancelled, which skipped the failure
issue. A step-level timeout under the job's marks the canary step failed
instead, so a hang opens the issue while a manual cancel still doesn't.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Main runs don't gate anything, so they run on every merge rather than only
when a listed path changes. That costs about two extra runs a day and still
catches a change the hand-kept path list misses. PR runs keep the filter.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…h a different value

A field in KNOWN_UNSAVED_FIELDS was always left out of the read-back
comparison, so a wrong saved value passed silently. It's now ignored only
when the employment returns nothing for it, the same rule forced fields
already follow.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
gabrielseco and others added 3 commits October 9, 2026 17:01
Tiger now returns overtime_eligible on the employment GET, so the canary
no longer needs to skip it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d3e8076. Configure here.

gabrielseco and others added 5 commits October 9, 2026 18:22
- Reject unknown --checks values and an empty country list (unknown
  --country or an empty /v1/countries payload), and exit 1 from
  decideExitCode when no row passed or failed, so a misconfigured or
  empty run can no longer report green.
- Wrap each live contract_details schema with
  withJobTitleEligibilityRiskAcknowledgement, matching what the
  Onboarding flow actually builds.
- Trim the seedEmploymentForCountry JSDoc to a caller-facing line.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Pushes and PRs made with the default github.token don't trigger other
workflows, so lint, tests and the title check never ran on the nightly
report PR. Minting a short-lived token from a GitHub App makes the PR
look like any other contributor's, so pr.yml and pr-title.yml run on it.

The job's own token now only needs contents: read and issues: write.
Needs SCHEMA_CANARY_APP_CLIENT_ID and SCHEMA_CANARY_APP_PRIVATE_KEY
secrets from an App installed on this repo with contents and
pull-requests write.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Take main's braced parseArgs guard now that #1481 is merged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Keep both the submit-failure and nothing-checked decideExitCode tests,
and apply main's curly lint rule to the submit stage.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Base automatically changed from ci/schema-canary to main October 12, 2026 08:43
gabrielseco and others added 2 commits October 12, 2026 10:46
Keep the submit check and the contract-details-seeds path filter in the
schema canary workflow now that #1493 is merged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@gabrielseco
gabrielseco merged commit d6ccf1f into main Oct 13, 2026
24 of 25 checks passed
@gabrielseco
gabrielseco deleted the feat/schema-canary-submit branch October 13, 2026 15:13
@gabrielseco gabrielseco mentioned this pull request Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants