Skip to content

design(atlas): cross-tier vector comparability — score normalization when tiers use different embedding models #6060

Open

Description

@dezren39

Parent: #6019 (semi-automated ingestion operations). Research source: prior-art doc ~/workspace/scratch/atlas-ingestion-brief-2026-11-04/prior-art-research.md (2026-11-04). Follow-up 42 (Cluster 7). Search side: fan-out #5755; rescore #5733/#5741.

Open question (research, flagged not sourced)

If the secret tier is embedded locally (sibling design under #6019) with a different model than the internal tier's route, the tiers' vectors live in different spaces — fan-out merge currently assumes comparable scores across shards. Mixing embedding models across tiers makes cross-tier score comparison meaningless unless the rescore stage normalizes (per-model score calibration, rank-based merge instead of score-based merge, or per-tier result quotas).

Scope

Done looks like

A decision + merge/rescore contract, or a one-model mandate with its cost consequences stated.

Activity

  1. added
    area/infraInfrastructure, deployment, and operations
    kind/designA design proposal (design-architect): multi-step, phased, decision-bearing
    on Nov 5, 2026
  2. developing-today commented on Nov 5, 2026

    @developing-today
    Contributor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/infraInfrastructure, deployment, and operationskind/designA design proposal (design-architect): multi-step, phased, decision-bearing

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions