Skip to content

Validate precomputed evaluation values against variation types - #40

Merged
typotter merged 2 commits into
mainfrom
tyler/typed-evaluation-schema
Sep 17, 2026
Merged

typotter merged 2 commits into
mainfrom
tyler/typed-evaluation-schema

Conversation

@typotter

@typotter typotter commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

What changed

Schema and documentation changes require precomputed evaluation defaults and expected values to match the declared variation type.

  • Float evaluations accept any JSON number; integer evaluations accept only integral numbers, including 1.0.
  • The existing fixture validator applies these constraints directly from the schema.

Why

A numeric string such as "1.5" previously passed validation for a float evaluation, allowing malformed typed getter inputs into downstream runners.

How to verify

  • Ran the existing fixture validator: all 12 fixtures pass unchanged.
  • Checked the schema against valid and invalid values for both fields across all five variation types.

Impact

For internal, the FFE team catches malformed typed evaluation fixtures before they reach SDK runners, reducing downstream debugging of invalid getter inputs.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The schema and documentation changes are consistent, narrowly scoped, and improve fixture validation without introducing functional code-path risk.

Pull request overview

This PR tightens validation for FLEX precomputed assignment fixtures by updating the JSON Schema (and related docs) so that defaultValue and expectedResult.value must match the declared variationType, preventing malformed typed getter inputs from reaching downstream SDK runners.

Changes:

  • Updated schemas/precomputed-assignment.schema.json to conditionally validate defaultValue and expectedResult.value types based on variationType (boolean/string/integer/float/object).
  • Clarified documentation in precomputed-assignments/README.md and AGENTS.md to explicitly state the required JSON types for these fields.
File summaries
File Description
schemas/precomputed-assignment.schema.json Adds allOf + if/then constraints to enforce defaultValue/expectedResult.value types by variationType.
precomputed-assignments/README.md Documents the new type-matching requirements for evaluation inputs and expected values.
AGENTS.md Updates the fixture-authoring guidance to reflect the stricter typed-value validation rules.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@typotter
typotter marked this pull request as ready for review September 17, 2026 20:28
@typotter
typotter requested a review from a team as a code owner September 17, 2026 20:28
@typotter
typotter requested review from danyal002, leoromanovsky and pavlokhrebto and removed request for a team September 17, 2026 20:29

@leoromanovsky leoromanovsky left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice

@typotter
typotter merged commit 0c46747 into main Sep 17, 2026
3 checks passed
@typotter
typotter deleted the tyler/typed-evaluation-schema branch September 17, 2026 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants