FastEmbed/ONNX embedding model is loaded multiple times per process, leaking ~25 GB of native memory #872

Summary

A single long-running basic-memory mcp server process grows to 25+ GB of memory and never releases it, eventually putting the whole machine under memory pressure (heavy swap/compression, UI stalls). This is not explained by the embedding model's normal size (~2.3 GB). The model appears to be loaded several times inside one process, and the native memory of the superseded ONNX sessions is never reclaimed.

Environment

• basic-memory 0.21.5 (pipx/mise install), Python 3.14
• macOS (Apple Silicon, 48 GB RAM)
• Backend: SQLite + sqlite-vec, semantic search enabled
• Embedding: provider fastembed, model jinaai/jina-embeddings-v3, 1024 dims
• fastembed 0.8.0, onnxruntime 1.26.0
• Server: basic-memory mcp --transport streamable-http --port 7331 (one shared daemon, multiple clients)
• 3 projects configured, sync_changes: true (file watch active)

What I observed

After ~30 h uptime, the server process footprint (footprint -p <pid>) looked like this:

phys_footprint:  26 GB
  25 GB   MALLOC_LARGE   (native heap)
 2.1 GB   mapped file    (the ONNX model on disk)
 ~0.6 GB  everything else

So ~25 GB sits in native large allocations — far beyond the model itself, and it is not Python-object memory.

The server log shows the embedding model being loaded multiple times within one continuous process (process start 11:39:18; this is the only process that has the model open, so every line below belongs to it):

11:41:59  FastEmbed model loaded: model_name=jinaai/jina-embeddings-v3 ...
11:44:54  FastEmbed model loaded: ...
12:22:26  FastEmbed model loaded: ...
12:22:36  FastEmbed model loaded: ...

Each "model loaded" line means a fresh TextEmbedding / onnxruntime.InferenceSession was created (~2.3 GB of weights plus its own CPU memory arena). The loads line up with vector-sync batches and incoming project-resolve / MCP requests.

Resident memory grows step-wise during embedding/sync activity and then stays flat — it does not shrink back. Over a day of normal multi-client use it reaches 25+ GB.

Why this matters

On a machine doing other work, this one process can eat roughly half of RAM, forcing the system into constant swap/compression and making everything sluggish. Restarting the server frees the memory instantly, but it grows back with use.

Likely root cause (hypothesis)

embedding_provider_factory.create_embedding_provider() is designed to return a process-wide cached singleton (_EMBEDDING_PROVIDER_CACHE), so the model should load exactly once. In practice it loads many times per process, which suggests the cache is being bypassed — i.e. a new provider instance is created on a cache miss, probably because the provider / search repository is rebuilt per request or per project context instead of reusing the cached provider.

Compounding factor: onnxruntime does not return CPU arena memory to the OS, so every superseded session's memory is retained for the life of the process. The effect is cumulative growth.

Confirmed: repeated model loads in one process, ~25 GB of native MALLOC_LARGE, step-wise retained growth.
Not yet pinned down: the exact code path that bypasses the provider cache.

Suggested directions

• Guarantee exactly one embedding provider / ONNX session per process. Verify the factory cache is actually shared across the request and sync paths, and log a warning if a second session is ever created.
• Consider disabling or capping the ONNX CPU arena (SessionOptions.enable_cpu_mem_arena = False); note that fastembed does not currently expose session options, so this may need an upstream hook or a workaround.
• Add a one-time "model loaded" guard so accidental re-instantiation is immediately visible.

How to reproduce (approximate)

1. Enable semantic search with fastembed + jinaai/jina-embeddings-v3.
2. Run basic-memory mcp --transport streamable-http as a long-lived server with sync_changes: true and a few projects.
3. Drive normal usage (note writes + searches) for a few hours.
4. Watch the server process RSS/footprint and grep the log for FastEmbed model loaded — it shows up more than once per process, and memory climbs accordingly.