Repository navigation
fix(core): native embeddings load failure is cached forever — one failed first load bricks the backend until restart #1567
Description
Summary
The native in-process embeddings backend (#1559, step 2 of #1554) caches the model-load promise forever — including when the load fails. One failed first load permanently bricks the native backend for the life of the server process, and every later call replays the original error even after the underlying problem is fixed on disk.
Evidence
packages/core/src/embeddings-native.ts:94-128 (origin/main 3bd4efaaf8):
const loadedModels = new Map<string, Promise<LoadedModel>>()
const loadModel = (snapshotDir: string, onnxFile: string): Promise<LoadedModel> => {
const key = `${snapshotDir}\n${onnxFile}`
const existing = loadedModels.get(key)
if (existing) return existing
const pending = (async (): Promise<LoadedModel> => { ... })()
loadedModels.set(key, pending)
return pending
}The rejected promise is never evicted, so a transient first-load failure (snapshot still syncing, tokenizer.json not yet written, a briefly-missing graph file) is final.
Reproduced this run on the laptop against a clean git archive of origin/main, driving embedNative directly with a temp snapshot dir (no model artifact needed — both failures occur before any native binding loads):
call1 (empty snapshot dir)
=> native-onnx embeddings backend: no tokenizer.json in model snapshot /tmp/embed-snap-XXXXXX
# tokenizer.json then created in that same directory; graph still missing
call2 (tokenizer.json now present, graph still missing)
=> native-onnx embeddings backend: no tokenizer.json in model snapshot /tmp/embed-snap-XXXXXX ← stale cached error
Expected call2 behavior: the load is retried and fails at the next real problem — no ONNX graph at <dir>/onnx/model_int8.onnx — or succeeds once the snapshot is complete.
Why it matters
Step 3 of #1554 flips a registration to native-onnx deliberately. The first embed call after that flip is exactly when a snapshot problem is most likely (wrong path, partial HF download). Today that first failure is unrecoverable without a server restart, and the error message actively misleads: it names a file that may already exist by the time anyone reads the log. The HTTP sidecar this replaces has no equivalent failure mode — a fixed sidecar serves the very next request.
Acceptance criteria
- A failed load is evicted from
loadedModels(only if the map still holds that same promise), so the nextembedNativecall retries the load from disk. - A successful load stays cached and shared exactly as today (one load per (snapshot, graph) pair; concurrent first calls still share one load).
- Regression test that needs no model artifact and no native bindings: first call against a snapshot dir missing
tokenizer.jsonfails with the tokenizer error; aftertokenizer.jsonis created in that dir, a second call fails with the graph error (proving the retry reached the next check) instead of replaying the tokenizer error.packages/core/test/embeddings-native.test.tsis the natural home — outside the snapshot-gated parity describe, since CI has no model file. - Existing embeddings suites stay green:
packages/core/test/embeddings-native.test.ts,packages/core/test/embeddings.test.ts.
Part of the #1554 native-embeddings series; found by the new-issue-creator sweep while verifying #1559 on main.
Activity
dezren39 commentedon Nov 3, 2026
🤖 Agent note · state: filed
loadedModelscache (packages/core/src/embeddings-native.ts:94-128) stores the load promise before it settles and never evicts a rejection, so one failed first load bricks the backend for the process lifetime and replays the stale error. Verified on a clean laptop archive probe of origin/main 3bd4efa by drivingembedNativeagainst a temp snapshot dir: call 1 (no tokenizer.json) failed with the tokenizer error; after tokenizer.json was created in that dir, call 2 returned the same tokenizer error instead of advancing to the missing-graph error.loadedModelsso the next call retries from disk (successful loads stay cached/shared), plus a model-free regression test in packages/core/test/embeddings-native.test.ts proving the retry reaches the next check; done when that test and the existing embeddings suites pass.4 remaining items
dezren39 commentedon Nov 4, 2026
🤖 Agent note · picked up
dezren39 commentedon Nov 4, 2026
🤖 Agent note · parked
dezren39 commentedon Nov 4, 2026
🤖 Agent note · picked up
dezren39 commentedon Nov 4, 2026
🤖 Agent note · parked
dezren39 commentedon Nov 5, 2026
🤖 Agent note · picked up
queue/8843-1567-embed-load, mergeability is MERGEABLE, and its check rollup is empty. No branch, files, or PR were touched. The first queue checkpoint was attempted withorigin=openchamber-scheduled-lanebut timed out.dezren39 commentedon Nov 5, 2026
🤖 Agent note · escalated
user/escalatedwas added. No code or PR changes were made.repo-queue checkpointandrepo-queue block --reason-class escalated-to-usercalls timed out. The configured queue-broker endpoint is unreachable from this session, so the terminal queue state is unconfirmed; the last confirmed response wasclaimedwith lease until 2026-11-05 01:45:33 UTC. Queue state still needs reconciliation when the helper is reachable.Merge pull request #1645 from developing-today-forks/queue/8843-1567-…