Skip to content

test: align calibration fixtures with synthetic gate - #16

Merged
streamentry merged 1 commit into
mainfrom
agent/synthetic-result-recalibration-fixture-drift
Jul 27, 2026
Merged

streamentry merged 1 commit into
mainfrom
agent/synthetic-result-recalibration-fixture-drift

Conversation

@streamentry

Copy link
Copy Markdown
Owner

Change

  • Make the calibration happy-path E2E fixture test-only but provenance-unclassified, so it can exercise quantitative gate behavior without being mistaken for eligible synthetic evidence.
  • Keep an explicit synthetic-label option and preserve the shipped synthetic example as a fail-closed recalibration input.
  • Make audit tests generate ignored CLI artifacts in temporary directories instead of depending on a prior Make invocation.
  • Update the lab-result example README to document the actual generated output filenames.

Why

The recent synthetic-result boundary correctly rejected every happy-path fixture because the fixture itself labeled all records SYNTHETIC. Two audit tests also assumed generated files existed under ignored outputs/ paths. This PR repairs the test contract without weakening the gate.

Evidence

  • Focused calibration suite: 99 passed.
  • Collection graph: 12,386 tests collected.
  • make agent-check: passed; 108 claim findings remain advisory and pre-existing.
  • make doctor: 8/8 checks passed.
  • Full configured suite: 12,358 passed, 20 failed, 8 skipped. The remaining failures are unrelated stale-doc, environment/package, and API-drift failures documented in the handoff; the synthetic recalibration cluster is resolved.
  • The normal pre-push hook could not collect because its separate interpreter lacks yaml and jsonschema; push used the documented one-time ECC_SKIP_PREPUSH=1 bypass.

Safety impact

No production calibration logic, safety policy, thresholds, candidate data, or release status changed. The tests now explicitly require the recalibration engine to raise PolicyViolationError and write no proposal when synthetic results reach it.

Proof level

Test/code-path integrity only. This does not establish biological activity, assay validity, recalibration validity, or wet-lab readiness.

Limitations

The full repository suite is not green because of the 20 unrelated baseline failures above. Human review remains required for any real wet-lab recalibration.

@streamentry streamentry left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review completed: the diff preserves the fail-closed synthetic-result boundary, makes ignored CLI artifacts self-contained under pytest temporary directories, and prevents proposal creation from synthetic input. No production policy, threshold, candidate artifact, or biological claim changed. Focused calibration tests pass (99); the known 20 baseline failures are disclosed in the PR body. No actionable findings.

@streamentry
streamentry merged commit 0c1deb3 into main Jul 27, 2026
@streamentry
streamentry deleted the agent/synthetic-result-recalibration-fixture-drift branch July 27, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant