Repository navigation
Extraction keeps what the model wrote, the agent answers the new message, local embeddings, and a turn is one readable trace - #229
Merged
Conversation
… people One reply field used to cost a whole extraction. With TemporalValidityMode.Extract a model that knows only the month writes "valid_from": "2026-08" -- valid ISO-8601 that System.Text.Json rejects by design. The typed parse threw, the runner answered "That response was not valid JSON" (it was valid), echoed the whole reply back, the model repeated the date, and after three attempts every entity, fact and preference of the turn was discarded. Measured live: 2 of 4 runs of one sentence stored nothing; after this change 3 of 3 runs that wrote "2026-08" stored everything in one call each. Fixed (defaults): - valid_from / valid_until read leniently: YYYY, YYYY-MM, YYYY-MM-DD, timestamps; a month or year is the start of the period for valid_from and its end for valid_until; always UTC (a date-only value was read at the machine's local midnight); free text only with an explicit year and never day/month-ambiguous; an unreadable date drops that date, never the item, and never throws. - A schema failure is salvaged item by item; the readable items are kept, each dropped one named. - One repair message, replaced each attempt, naming the problem -- no echo of the bad reply. - finish_reason=length retries with twice the output room it used; content_filter is not retried (ContentFilteredException, recognised by the batch splitter as a content rejection). - New entity names in one extraction are embedded in one request; a new entity keeps the vector resolution computed for its name (resolution itself unchanged: the node and the turn's candidate set still see it without a vector until persistence writes it). Added (dark by default): - EntityResolutionOptions.EnablePartialNameMatch: "Priya" resolves to the ONE known "Priya Nair"; ambiguity declines and is recorded (memory.resolve.partial_name_ambiguous). EntityMatchType.PartialName. - MemoryOptions.EmbeddingCacheCapacity: LRU text->vector cache (the stored user message was embedded twice per turn; known names every turn). - memory.extract.attempt spans (outcome, finish reason, tokens, items kept/dropped, error); LlmExtractionOptions.LogRawResponseOnFailure. The LongMemEval call meter now derives parse-retry detection from the runner instead of a copy of its old wording. Two review rounds: round 1 (4 MEDIUM, 8 LOW) and round 2 (1 MEDIUM, 2 LOW) all fixed with tests that fail without the fix. Release build 0 warnings. Unit 5444, extensibility 42, SK 54, LongMemEval 910, integration 504/533 (29 skipped as before). Sample-run gate: MinimalAgent and AgentWithMemory run end-to-end on Bitdeer + local Neo4j. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved moderate findings remain in caching, truncation handling, acknowledgement validation, date parsing, and content-filter handling.
Review effort: Lite
Findings: 1
What changed in this PR
This PR improves LLM extraction resilience and entity-resolution efficiency.
Changes:
- Adds tolerant date parsing, item salvage, targeted retries, and telemetry.
- Adds opt-in partial-name matching, embedding caching, and batched embeddings.
- Expands regression tests and LongMemEval metering.
| File | Summary | Review notes |
|---|---|---|
tools/AgentMemory.LongMemEval/LongMemEvalChatCallMeter.cs |
Updates retry detection. | No final comment. |
tests/AgentMemory.Tests.Unit/Services/EmbeddingVectorCacheTests.cs |
Tests cache behavior. | No final comment. |
tests/AgentMemory.Tests.Unit/Resolution/PartialNameEntityMatcherTests.cs |
Tests partial matching. | No final comment. |
tests/AgentMemory.Tests.Unit/Resolution/BatchNameEmbeddingTests.cs |
Tests batched embeddings. | No final comment. |
tests/AgentMemory.Tests.Unit/Extraction/Sprint1ReviewRegressionTests.cs |
Adds regression coverage. | No final comment. |
tests/AgentMemory.Tests.Unit/Extraction/PeriodDateConverterTests.cs |
Tests date parsing. | No final comment. |
tests/AgentMemory.Tests.Unit/Extraction/LlmUnifiedMemoryExtractorTests.cs |
Updates retry expectations. | No final comment. |
tests/AgentMemory.Tests.Unit/Extraction/ExtractionRepairTests.cs |
Tests repair behavior. | No final comment. |
tests/AgentMemory.Tests.Unit.LongMemEval/LongMemEvalPreparedBatchBehaviorTests.cs |
Updates meter fixtures. | No final comment. |
src/AgentMemory.Extraction.Llm/LlmMultiSessionUnifiedMemoryExtractor.cs |
Handles content-filter rejection. | Moderate: content-filter failures can still trigger batch splitting (1 vote). |
src/AgentMemory.Extraction.Llm/LlmExtractionOptions.cs |
Adds raw-response logging. | Nit: operator documentation is incomplete (1 vote). |
src/AgentMemory.Extraction.Llm/Internal/PeriodDateConverters.cs |
Implements tolerant UTC period parsing. | Moderate: ambiguous hyphenated dates are not rejected (1 vote). |
src/AgentMemory.Extraction.Llm/Internal/LlmResponseModels.cs |
Applies date converters. | No final comment. |
src/AgentMemory.Extraction.Llm/Internal/LlmExtractionRunner.cs |
Adds salvage, repair, retry, and telemetry. | Moderate: length-truncated usable responses may be accepted; null acknowledgements can cause failures (1 and 3 votes). |
src/AgentMemory.Extraction.Llm/Internal/ContentFilteredException.cs |
Defines content-filter failures. | No final comment. |
src/AgentMemory.Extraction.Llm/AgentMemory.Extraction.Llm.csproj |
Exposes internals to tests. | No final comment. |
src/AgentMemory.Core/Services/EmbeddingVectorCache.cs |
Adds bounded LRU caching. | Moderate: mutable cached vectors can be externally corrupted (1 vote). |
src/AgentMemory.Core/Services/EmbeddingOrchestrator.cs |
Integrates embedding caching. | No final comment. |
src/AgentMemory.Core/ServiceCollectionExtensions.cs |
Registers and validates options. | No final comment. |
src/AgentMemory.Core/Resolution/PartialNameEntityMatcher.cs |
Implements partial-name resolution. | No final comment. |
src/AgentMemory.Core/Resolution/IExtractionEntityResolver.cs |
Adds batch embedding preparation. | No final comment. |
src/AgentMemory.Core/Resolution/EmbeddingMemo.cs |
Reuses computed vectors. | No final comment. |
src/AgentMemory.Core/Resolution/CompositeEntityResolver.cs |
Integrates matchers and vector reuse. | No final comment. |
src/AgentMemory.Core/Resolution/CompositeEntityResolver.Batch.cs |
Adds batch preparation. | No final comment. |
src/AgentMemory.Core/Extraction/ExtractionStage.cs |
Invokes batch embedding preparation. | No final comment. |
src/AgentMemory.Abstractions/Options/MemoryOptions.cs |
Adds embedding-cache configuration. | Nit: required operator documentation is incomplete (1 vote). |
src/AgentMemory.Abstractions/Options/ExtractionOptions.cs |
Adds partial-name configuration. | No final comment. |
src/AgentMemory.Abstractions/Domain/Extraction/EntityMatchType.cs |
Adds partial-name match type. | No final comment. |
CHANGELOG.md |
Documents user-visible changes. | Nit: related operator documentation is missing (1 vote). |
.gitignore |
Ignores local DemoBrain files. | No final comment. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+373
to
+374
| response.ProcessedSourceSessions = property.Value.Deserialize<List<string>>(JsonOptions); | ||
| } |
…view fixes
Each model call now runs inside its own memory.extract.attempt span, one level below the
extractor's span. Two meters read the extractor's span from inside the chat client and silently
lost it: the perf counting client (purpose attribution, PERF-W-07 in CI) and the LongMemEval call
meter (token-estimate tags and per-activity retry counting). The runner now exposes
CallerActivity (skips attempt spans) for the meter, and the perf client walks to the nearest
enclosing extraction span instead of matching only the immediate parent.
PERF-W-08 pinned 7 embedded items; 5 is the intended saving (new entities carry the vector
resolution computed, so persistence embeds only facts and preferences). Its provenance check
shared that constant -- a twin that was only right while the two counts matched -- and now has
its own. Local perf + gate PASS on both the zero and remote profiles.
Round-2 review: KeptItems is null-safe ("facts": null with a listener attached threw); a
truncation reporting 0 output tokens sets no limit; day/month-ambiguous numeric dates are not
guessed; TryParse returns exactly IsUsable.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Hook spans parent everything memory does in a turn (memory.hook.recall / memory.hook.ingest), with routing (memory.route), composition (memory.compose) and embeddings (memory.embed) as their own spans. Every recall leg carries its memory type and result count. Database query spans end when their result is read, record the server's timings and the managed transaction's retries, and name every query (unregistered queries get a structural fingerprint instead of one "unknown" bucket). Background access tracking and enrichment run in their own traces, linked to the work that queued them. Owner and application ids are never attributes. Span names and keys live in MemoryTelemetry; docs/observability.md documents the tree, the attributes and the privacy rules. Nothing is recorded without a listener. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
MAF appends a context provider's messages after the request. Recalled conversation turns came back newest first, so they followed the user's new message and the model answered the last (old) user turn it read. MAF also hands a provider only the caller's new messages, so the recalled-history dedup never saw the session's own chat history. Recalled turns now go before the live thread, in the order they happened, behind a one-line framing message; the dedup sees the whole request. On hosts that render memory at the user role, the provider's memory messages move to just before the user's last message (never into a tool exchange). One shared placement rule (RecalledTurns) serves the Neo4j and NAMS providers; NAMS orders turns by createdAt. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AgentMemory.Inference gains an `ollama` provider (http://127.0.0.1:11434/v1, no key, bge-m3 by default; last in auto-detect order), and AI_EMBEDDING_PROVIDER alone now moves embeddings to another provider, taking what is not set from that provider's own variables and defaults, never from the chat provider. AI_EMBEDDING_PROVIDER=ollama puts embeddings on a local bge-m3 while chat stays remote. Ollama's bge-m3 returns the same vectors as BAAI/bge-m3 (measured cosine 1.0000), so an existing store keeps working. Measured per short text: 52 ms on a local GPU, 70 ms CPU-only, 1.3 s typical for the hosted call. 127.0.0.1, not localhost: on Windows localhost costs ~2 s per request. Also: the facade placement test awaits instead of blocking (xUnit1031). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Review fixes for the local-embeddings change: - With AI_EMBEDDING_ENDPOINT set, the key comes only from AI_EMBEDDING_API_KEY (or none, for a keyless provider): a leftover endpoint plus AI_EMBEDDING_PROVIDER=bitdeer no longer sends the Bitdeer key to that host. - Ollama is never auto-detected: OLLAMA_MODEL / OLLAMA_API_KEY are set by other tools and must neither choose a provider nor hide a half-configured one. - A refused endpoint names the variable it came from. - Help text, samples and docs list ollama; AgentEval parity noted. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Access tracking and enrichment consumers inherited the store routing of whichever request first built the singleton, so in a multi-store host application B's access stamps and enrichment went to application A's store. Each batch now carries its application id; the consumer applies it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each fan-out leg's query was embedded in its own request, one after another: three sequential round trips on a two-part question (~110 ms local, ~2.6 s hosted, measured in an 18-turn traced session). The legs now go out together; a failed batch falls back to leg-by-leg. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The owner-scoped similarity scans (27% of recall legs in an 18-turn session), the entity-resolution type lookup, the fact merge-key lookup and vector-index creation reported as structural hashes. They now carry names. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
From a whole-branch defect hunt:
- a failed extraction/persistence and an attempt whose transport retries
ran out now record the exception type and an error status on their span;
- validation refuses PartialNameMatchConfidence >= AutoMergeThreshold
(a merged alias would bypass the ambiguity check);
- a relational name ("X's mom") is not a partial-name candidate for X.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When the whole-document read fails, the salvage pass reads every item again, so a dropped date was recorded twice on the same attempt span. Drops from the whole read are now held and recorded only if it succeeds. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…olution that scales (dark) - The working-memory profile tier is on by default (MinFactMentionCount 1). A new session could not answer "what do you know about me?" without it (a generic question's embedding matches few stored facts); with it, 3 of 3 runs answered with the user's name, job, family and preferences. Cost, now in the perf baseline: +1 read per recall, +5 queries per memorised turn for the rebuild, ~300 prompt tokens. - AgentMemoryChatHistory: MAF's in-memory history without the memory a context provider injected (17 vs 8 messages by the third call). Neo4jChatHistoryProvider stores through the same filter; samples use it. - EntityResolutionOptions.IndexedCandidates (dark): candidates without vectors plus the entity vector index for the semantic stage. At 5,000 people per owner the default read cost 0.8-1.2 s per resolution. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ExtractionOptions.CanonicalFactSubjects: a fact's subject and object take
the resolved entity's name ("Tomás" -> "Tomás Silva"), the words used
kept as subject_surface/object_surface, so mentions of one person merge
into one fact and keep their ABOUT edges. Verified live: "Tomas" and
"Tomás" facts are stored under "Tomás Silva".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…n (dark) Review of the default-on profile tier (1 HIGH, 8 MEDIUM, 6 LOW): - The tier now activates its Neo4j schema extension (the :User.identifier constraint), so its per-owner MERGE is indexed and race-safe. - The block keeps itself current: it records the next validity boundary of its facts and rebuilds on the first read after it; a prune clears the touched owners' blocks; the most recently touched facts win slots while the text keeps a stable order (no reshuffle on re-mention). - Semantic Kernel renders the block when similarity found nothing. - AgentMemoryChatHistory composes with the caller's own filter and never mutates the caller's options. - IndexedCandidates (dark): index hits without vectors are read in full; the turn's merged copy wins over the index's stored one. - The perf baseline keeps its quality annotations on refresh; the eval harness stays pinned to MinFactMentionCount 2; cancelled rebuilds are not logged as failures; salvage records a dropped date only for a kept item. ExtractionOptions.DeduplicateWithinExtraction (dark): a fact at least 0.93 similar to another fact of the same extraction is dropped for the more confident one (duplicates 0.912-0.996, closest distinct pair 0.879). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With auto-extraction the agent's reply was extracted too, so every fact the agent repeated back gained a mention and summary replies re-learned the profile as rephrased duplicates. AgentFrameworkOptions.ExtractFromUserMessagesOnly extracts only the turn's request messages; the reply is still stored. Measured on one six-turn conversation: 11 facts (3 rephrasings, echo-inflated mention counts) against 8 facts at one mention each. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…milarity Profile tier (default on): - A rebuild on read that fails never fails recall; it serves no block (or the stored one with ClearOnRebuildFailure = false). - A prune or a failed rebuild removes the text and marks the block due, so the next read rebuilds it from what remains; an unscoped prune no longer leaves every owner without a profile until they write again. - An empty block keeps its validity boundary, so a block waiting for a future fact is rebuilt when the fact starts. Dark options: - Canonical fact subjects: distinctness is checked on the stored (canonical) key, so two phrasings of one fact take the sequential path instead of a batch that folds them and replays both. - Within-extraction merge requires the same subject, one differing slot, the same numbers and negations and no conflicting validity window; keeps the user's words over the assistant's; carries over dates and source turns; reports the dropped fact as Skipped (FactMergedWithinExtraction). Also: the perf baseline reads its measurement labels from the summary; the CLI documents that the profile schema is always added while the tier is on. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The extractor calls the speaker "user", so one person was stored under two names and the profile listed both. LlmExtractionOptions.CaptureUserName asks every extractor rung for a "user | is named | <name>" fact when the user states their name; ExtractionOptions.ResolveUserToName stores facts whose subject or object is user/I/me under that name (from the same extraction or the owner's latest stored naming fact). One function decides the stored name for both the fact and the batch distinctness guard. Measured on a two-session conversation: 3 facts about "Dana" and 5 about "user" without it; all 8 under "Dana" with it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AgentFrameworkOptions.ExtractInBackground hands a turn's extraction to the new IBackgroundExtraction: ordered per session, concurrent across sessions (BackgroundExtractionConcurrency), carrying the turn's owner and store scope (the caller's execution context is captured, so a host scheduler that does not flow context still writes as the right owner), drained on shutdown (BackgroundDrainTimeout; sync and async dispose). The context provider, the chat-history provider and the facade now share one dispatcher. Measured, 10 live turns per arm with a 6 s pause: median answer 3.5 s -> 1.5 s, p95 9.7 s -> 4.5 s, same facts stored. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… everywhere Default paths: - A rebuild on read that failed is not retried for that owner for a minute (WorkingMemoryRebuildBackoff), instead of a failing write per recall. - The batch distinctness guard compares predicates on the storage key (separators folded), so "works_at" / "works at" take the sequential path. Dark options: - ExtractFromUserMessagesOnly is applied once, in the shared dispatcher, so the chat-history provider and the facade honour it too. - I-5: the name is read with one keyed query (IFactRepository.FindLatestObjectAsync, casing/separators/validity aware; a failed read leaves the words used); "I"/"me" are the user only as a subject, "user" as an object, never the assistant's own first person; the naming fact is never merged away; a fact now under the name also supersedes what was stored as "user"; the naming instruction is described in words (the multi-session rung's schema needs source_session). - Within-extraction merge: what only the assistant said is not carried into the user's words. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AgentTraceRecorder passed no owner unless given one, and the reasoning service does not read the ambient scope, so under strict multi-tenant isolation every trace write inside a provider's turn threw MemoryOwnerScopeRequiredException. It now falls back to IMemoryOwnerContext; an explicit owner still wins. Found by turning on reasoning traces in a live host. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The first wording of CaptureUserName's instruction told the model to "refer to the person speaking to the assistant as user". On a book passage the model then reasoned "fictional text; no user memory" and returned nothing: 6 of 18 chunks of Alice chapter 1 over three runs. It now only asks for the naming fact and says everything else is extracted as before: 1 of 18, as without the option; the naming fact is still captured in conversation. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ExtensibleMemoryContextProvider threads every optional dependency of the base provider through (it is the shipped provider for extensible hosts), and TheDerivedProviderAcceptsEveryDependencyTheBaseDoes caught that the new IBackgroundExtraction was not: ExtractInBackground would have been silently inline on those hosts. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…th I-5) "user -WORKS_AT-> Fabrikam" was dropped at extraction because "user" is not an extracted entity. With ResolveUserToName the extraction stage keeps it, and persistence maps the endpoint to the person the user named (in this turn, or the stored entity of the stored name; one read, best-effort), by the same rule as fact slots. Live: three relationships from "Ana", none skipped. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… the block - WorkingMemoryOptions.RecentStableFactSlots (default 4 of 12): the most recently learned facts that did not win a slot by mentions still get in, so a new job or city is not shut out by facts mentioned more often. 0 restores slots by mentions only; the LongMemEval profile pins 0 (its measured path). - The next-boundary query counts only block-eligible facts (no rebuilds for a fact that cannot be in the block). - CLI invalidate/supersede mark the owner's block due. - MCP recall returns the block (workingMemory); it was read and dropped. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Other_sessions_do_not_wait read Pending right after the second session's work signalled, before the queue's finally had decremented it; it failed on a slow CI runner. It now asserts what it is about: session 2 ran while session 1 was still blocked. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The five public types that gained an optional constructor parameter keep their 1.5.0 signatures as overloads (an added optional parameter is a binary break); the new constructors carry [ActivatorUtilitiesConstructor]. - Background extraction resolves IMemoryService from a fresh scope per item (the turn's scope has usually ended), starts on the thread pool, and is its own trace linked to the turn. - The extraction context window takes only messages from before the targets: extracted later, the newest stored messages can be the next turns. - I-7: the user's person entity is found live (IEntityRepository.FindLiveByNameAsync: never a merged tombstone or an invalidated node, exact name before alias, case-insensitive); no self-loop edges; "me" as a relationship target is the user. - A flaky-test-free assertion set, docs on the queue's limits. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…fault Provenance (external review): each Agent Framework component minted its own id for a caller's message, so no EXTRACTED_FROM edge reached the user's words, and with user-only extraction there were none at all. A request message now gets one id stamped on the ChatMessage (MafTypeMapper.EnsureProviderMessageId); the context provider stores the user's message under it, the chat-history provider stores under the same id, the store MERGEs on it. A real ChatClientAgent turn against Neo4j (user-only on/off x background on/off x with/without the history provider): every node reaches the user's :Message, every id resolves, one node per message; all six fail before the fix. New defaults, all measured live: - ExtractInBackground on, with a next-turn guard (RecallWaitsForPendingExtraction, 2 s; one owner's turns learned in order, owners in parallel) - ExtractFromUserMessagesOnly on - ResolveUserToName + CaptureUserName on - new IgnoreQuestions (on): a question states nothing LongMemEval pins all of them off. Perf baseline: +1 embedding and write per turn (the user's message), 1 source message per extraction. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- One entity-resolution batch per extraction request: new names are embedded in one call, not one each. - ExtractionOptions.SkipPlainQuestions (off by default): a plain question does not pay for an extraction call; a question that names someone, mentions a time or describes what the user owns still does. - Recall fan-out no longer splits a statement; requests, contracted interrogatives and imperatives still fan out. - Reasoning traces are searched only when the formatter shows them. - AgentFrameworkOptions.RecalledMemoryBeforeQuestion (off by default), for models that answer the last message. - Owner-first vector recall (MemoryOptions.OwnerFirstVectorThreshold, default 500): a small owner is scored exactly over its own rows and cannot be crowded out of the global index; a large shared corpus is merged from the index. The owner-bounded fallback seeks owner_key instead of scanning every fact. - ExtractionRequest.ShareWithEveryone: general knowledge stored without an owner, recalled by everyone, resolved against shared entities only; a shared write never supersedes or aggregates tenants' facts. - Two review rounds: every finding fixed with a test that fails before its fix. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9
IShortTermMemoryService.InvalidateMessageAsync (and IMessageRepository.InvalidateAsync) stamp invalidated_at: the message leaves recent messages, message search and whole-session reads, and stays for history (as-of reads of earlier times still return it). Found live: after forgetting a person and every fact about them, the agent still answered from the message that named them. The new members have default implementations that throw, so existing implementations still compile. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9
Conversation reads skip them (retroactive conversation extraction would re-learn and revive forgotten facts; the MCP conversation tool read them back), source quotes skip them (Message.InvalidatedAtUtc), a forgotten last message is not a session preview, and the unscoped message search over-fetches so forgotten messages never leave it short. Docs and changelog say where forgetting applies; InvalidateAsync, like DeleteAsync, is not owner-checked and says so. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9
The over-fetched unscoped message search now always applies LIMIT $limit (it returned the whole candidate pool when no metadata filter was set). A session with nothing live lists last, the MCP conversation list counts live messages only, and only source quotes skip forgotten messages (date grounding still reads their timestamp). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9
- With canonical subjects on, "user" could resolve into the person the user named ("Hi! I'm Dana"), and the
naming fact itself was stored as "Dana | is named | Dana": the name was never found again and the user showed
up beside themselves as someone else. Words that mean the user are renamed only by the user-name rule.
- Sessions list by their last live activity, else when they were created: a session with nothing live left no
longer jumps to the top, and a new empty one no longer drops to the bottom.
- The over-fetch claim is stated precisely; a test pins that a forgotten source still dates its fact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9
Reasoning traces the formatter hides are no longer searched (PERF-R-04/07/08 read the formatter instead of assuming traces are recalled), and one extraction's new names are embedded in one request (PERF-W-08: 2 embedding requests, the same 5 texts). Perf gate: PASS on the zero and remote profiles. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9
joslat
added a commit
that referenced
this pull request
Sep 28, 2026
…tions, recent topics (#231) * Dates and the shared-knowledge budget on by default (37.1) IncludeDates defaults to true on the four renderers that print dates (one guard test for the twins) and SharedRecallBudget to 3: in simulated conversations both kept every answer and improved the dated ones. The model-dependent extraction switches and supersession stay opt-in. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9 * An object the model wrote twice is stored once (37.2) "Daniel | is a chef | chef" rendered "Daniel is a chef chef". A predicate that ends with its own object's words, with or without a leading article, is trimmed at write, for every fact on either write path; whole words only, and never to an empty predicate. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9 * "Lately": what the person has talked about most recently (37.5) WorkingMemoryOptions.RecentTopicsDays (off by default) ends the profile block with one line naming the owner's entities that live facts extracted in the window mention most, counted by distinct messages (EXTRACTED_FROM provenance with the message time), never the person (self words, their stated name). Only what the person said counts. One read per rebuild, only when enabled; trimmed first when the block is over budget. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9 * A question-only turn waits, stored, for the next turn that tells something (37.4) The durable redesign of the deferral pulled in review round 8. The memory service decides (every entry point gets it); the host only consents (AgentFrameworkOptions.DeferQuestionTurns sets ExtractionRequest.DeferIfOnlyAsking). A waiting turn is marked on its stored message: it survives a restart, stays in its store and session, is gone when forgotten, and the marks clear only when the releasing extraction succeeds; at most MaxDeferredTurns wait. The E2 context window reaches up to the last target, so the agent's reply between a waiting question and its telling turn is context. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9 * Docs: every memory option in one table; dates, events, shared knowledge, relationships, changes of mind (37.3) A new docs/configuration/memory-options.md lists each switch with its default; getting-started, agent-framework, memory-map, schema and architecture say what the memory now does and store (occurred_on, date precision, entity owner_key, the predicate trim), with the known limits. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9 * Docs: the ships-dark section says which defaults were later turned on Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9 * Keep the shared-knowledge budget opt-in; record W-08's justified read in the baseline The perf gate caught the cost of SharedRecallBudget on by default: an owner-scoped recall searches twice per memory type (PERF-R-07: 9 -> 16 queries), in every store, shared knowledge or not. It stays opt-in (3 is the measured value) until the shared search is skipped where there is nothing shared. IncludeDates stays on. The baseline records PERF-W-08's 15th query, the restated-relationship read justified in #230, so later changes are gated against the current shape. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9 * Phase 37 review round 1: held turns belong to their owner; Lately counts what the person said Deferral: a waiting turn is marked for its owner (the session when there is none) and read by an index seek; the owner's next telling turn in any session releases it (a session ending on a question no longer strands it; one agent's shared session id never releases Alice's words into Bob's extraction); any failure while deferring extracts the turn now; "please ..." and "help me ..." are told, not held; waiting turns are counted as the user's messages. Lately: distinct facts from live user messages (a released batch and the agent's words no longer inflate it), the naming exclusion by the same keys as the name lookup, case-blind entity match, rebuilt at least daily. The predicate trim moves a dangling article to the object. A dated recalled turn is deduplicated against the live thread by what was said. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9 * The shared budget no longer searches for shared rows that do not exist; on by default (37.1b) With SharedRecallBudget set, every owner-scoped recall searched twice per memory type, shared knowledge or not (the perf gate caught it: PERF-R-07 9 -> 16 queries). ISharedCorpusProbe answers whether the store holds shared memory of a kind (Neo4j: one EXISTS per store and kind, a seek on owner_key for facts and entities; "none" cached 30 s, "some" 10 min); without any, the shared search is skipped, and a probe that cannot answer leaves it in place. With the cost gone where nothing is shared, SharedRecallBudget defaults to 3 again. The perf gate passes unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9 * Integration tests that read a turn's memory at once extract inline Three tests read what a turn stored right after it returned; with ExtractInBackground on by default (since #229) that was a race they lost when run on their own (warm runs won it, so full-suite runs passed). They are about owner stamping, provenance and trust, not background timing. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9 * Shared knowledge written a moment ago is recalled at once The shared-corpus check cached "none" for 30 seconds, so in a store without shared knowledge, a book taught and then asked about straight away was not recalled until the window expired. Every repository path that writes an owner-less fact, entity or preference (single, batch, fused, derived) now tells the check before the write, and it answers "some" at once; a "some" that turns out wrong only costs one empty search. Only a write from another process still waits out the window. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0139u7CNFuidKuPJBRK7auC9 --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Oct 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Why
One reply field used to cost a whole extraction. With
TemporalValidityMode.Extract, a model that knows only the month writes"valid_from": "2026-08", which is valid ISO-8601 but rejected by System.Text.Json by design. The typed parse threw, and the runner answered "That response was not valid JSON", although the reply was valid. It then echoed the whole reply back, and the model repeated the same date. After three attempts, every entity, fact and preference of the turn was discarded.Measured live: before this change, 2 of 4 runs of one sentence stored nothing. After it, 3 of 3 runs that wrote
"2026-08"stored everything in one call each.What changed
Fixed (defaults)
valid_from/valid_untilare read leniently.YYYY,YYYY-MM,YYYY-MM-DDand timestamps.valid_fromand its end forvalid_until.03/04/2026).finish_reason = lengthis retried with twice the output room the reply used.content_filteris not retried; it raisesContentFilteredException, which the batch splitter treats as a content rejection.Added (dark by default)
EntityResolutionOptions.EnablePartialNameMatch: "Priya" resolves to the one known "Priya Nair". If several people match, the matcher declines and recordsmemory.resolve.partial_name_ambiguous. AddsEntityMatchType.PartialName = 4.MemoryOptions.EmbeddingCacheCapacity: an LRU text→vector cache. The stored user message was embedded twice per turn, and known names were re-embedded every turn.memory.extract.attemptspans, tagged with outcome, finish reason, tokens, items kept and dropped, and the error. PlusLlmExtractionOptions.LogRawResponseOnFailure(off by default).Fixed: the agent answered the previous message
MAF appends a context provider's messages after the request. Recalled conversation turns came back newest first, so they followed the user's new message, and the model answered the last (old) user turn it read. Captured live: asked "Priya called me, she wants to meet on Friday", the agent replied "Priya Nair is already on file" (the previous turn's topic). MAF also hands a provider only the caller's new messages, so the recalled-history dedup never saw the session's own chat history.
RecalledTurns) servesNeo4jMemoryContextProviderandNamsMemoryContextProvider; NAMS orders turns bycreatedAt. The facade no longer reverses recents before the budget.docs/agent-framework.mdshows how to keep recalled memory out of MAF's in-memory history.Added: a local embedding model is one variable away (
AgentMemory.Inference)ollamaprovider:http://127.0.0.1:11434/v1(notlocalhost: on Windows that costs ~2 s per request), no key,bge-m3by default,OLLAMA_MODELfor chat. Used only when named, never auto-detected.AI_EMBEDDING_PROVIDERalone now moves embeddings:=ollamaputs them on a local bge-m3 while chat stays remote,=bitdeermoves them back. Missing values come from the named provider's own settings, never from the chat provider, and a key travels with its endpoint.bge-m3returns the same vectors asBAAI/bge-m3(cosine 1.0000), 52 ms per short text on a local GPU, 70 ms CPU-only, against 1.3 s typical (5.4 s p95) hosted. In a live 9-vs-9-turn comparison, median time to answer went from 4.6 s to 2.8 s and retrieval from 1.2 s to 0.21 s.From an 18-turn traced session and a whole-branch defect hunt
PartialNameMatchConfidence >= AutoMergeThreshold), and a relational name ("Priya's mom") is not a partial match for Priya.Tracing: one agent turn is one readable trace (nothing is recorded without a listener)
memory.hook.recall(PRE) andmemory.hook.ingest(POST) parent everything memory does in a turn, tagged with the session, the conversation, and only whether the turn was owner-scoped or routed to an application store (ids are tenant data; no hashes).memory.routerecords the recall policy's decision; the hook span records the temporal as-of and the fan-out plan;memory.composecounts flagged, excluded and deduplicated items.memory.typeandmemory.results.counton each leg, including the repositories' vector and point-in-time searches.memory.embedspans (inputs, cache hits, sent).db.result.read = none | partial.db.attempts/db.retries.unregistered:<index-or-label>:<hash>(literals normalised) instead of oneunknownbucket: a live session had 340 unknown; now 0. Cypher passed to the graph-query service is never given a structural name.MemoryTelemetry(constants public, helpers internal). Newdocs/observability.mddocuments the span tree, the attributes and the privacy rules.The LongMemEval call meter now takes its parse-retry check from the runner instead of keeping a copy of the old wording.
Measurement note: retry attempts send different bytes. A run whose every reply parsed on the first attempt is unaffected.
Changed: the working-memory profile tier is on by default
WorkingMemory.Enabled = falserestores the previous behaviour.working-memory, the:User.identifierconstraint) whenever the tier is on.WorkingMemoryOptions.RecentStableFactSlots(4 of 12): the most recently learned facts still get in when the other slots hold facts mentioned more often. CLIinvalidate/supersedemark the block due; MCP recall returns it (workingMemory).AgentMemoryChatHistorykeeps recalled memory out of MAF's own chat history (CreateInMemoryProvider,ExcludeInjectedContext).Added: the answer does not wait for memorising (
AgentFrameworkOptions.ExtractInBackground, dark)IBackgroundExtraction: ordered per session, concurrent across sessions, carrying the turn's owner and store scope, drained on shutdown. The context provider, the chat-history provider and the facade share one dispatcher.Added: learning quality (all dark)
ExtractFromUserMessagesOnly: learn from what the user said, not from the agent repeating it back. One conversation: 11 facts with echo-inflated mention counts, against 8 facts at one mention each.ResolveUserToName+CaptureUserName: what the user says about themselves is stored under their name. Two sessions: 3 facts about "Dana" and 5 about "user" without it; all 8 under "Dana" with it. The name is read with one keyed query (IFactRepository.FindLatestObjectAsync). Relationships from "user" start at the user's person entity (before: dropped at extraction).CanonicalFactSubjects: facts named by the resolved entity ("Tomás" and "Tomás Silva" become one subject).DeduplicateWithinExtraction: one fact per statement within an extraction, only for same-subject, one-slot rephrasings with the same numbers, negations and validity (similarity alone merged "2 kids" and "3 kids").EntityResolutionOptions.IndexedCandidates: resolution candidates from the vector index instead of every entity of the type with its vector. At 5,000 people: 1.1–1.7 s to 0.17–0.24 s.Fixed:
AgentTraceRecorderrecords traces under the turn's owner. Under strict isolation every trace write inside a provider's turn failed ("StartTraceAsync requires an owner scope"); it now falls back to the ambient owner.Review and verification
Eleven review rounds, all findings fixed or documented. Since the last update: the profile flip and indexed candidates (round 1: 1 HIGH, 8 MEDIUM; round 2: 5 MEDIUM, 11 LOW; round 3: 9 MEDIUM, 7 LOW); background extraction and relationships from the user (round 4: 4 MEDIUM, including a binary break of five public constructors against 1.5.0, now kept as overloads, and queued work using the turn's ended scope; 6 LOW). Earlier: the recalled-turn placement fix (1 HIGH, 4 MEDIUM, 13 LOW over two rounds), local embeddings (1 MEDIUM), extraction (5 MEDIUM, 10 LOW), tracing (5 MEDIUM, 15 LOW). Each fix has a test that fails without it (mutation-checked). Re-measuring book ingestion also caught a prompt regression in
CaptureUserName(book chunks returned nothing), fixed.Build: Release, 0 warnings.
Tests:
Sample-run gate (at
7924251): MinimalAgent and AgentWithMemory run end to end on Bitdeer and local Neo4j, including cross-session recall and the strict-isolation fail-closed check.AgentWithMemory also runs with
AI_EMBEDDING_PROVIDER=ollama(chat on Bitdeer, embeddings on a local bge-m3).Perf gates: zero-latency and remote-latency profiles PASS against
eng/perf/baselines/hermetic-S.json.Sprint 8: what an end-to-end trace said, and forgetting that sticks
From a traced end-to-end run (the agent model is ≈85% of the answer; recall ≈180 ms):
ExtractionOptions.SkipPlainQuestions(off by default): a plain question ("Where does my brother live?") no longer pays ≈1 s and ≈450 tokens for an extraction that finds nothing. A question that names someone, mentions a time or a number, or describes what the user owns ("…for my celiac son?") is still read.AgentFrameworkOptions.RecalledMemoryBeforeQuestion(off by default), for small models that answer only the last message.Owner-first vector recall (
MemoryOptions.OwnerFirstVectorThreshold, default 500; 0 turns it off):owner_id; shared rows come viaowner_key.ExtractionRequest.ShareWithEveryone: general knowledge (a book, a manual) stored without an owner.Forget a message without deleting it:
IShortTermMemoryService.InvalidateMessageAsync. Found live: after forgetting a person and every fact about them, the agent still answered from the message that named them.invalidated_atis skipped by:limit);Message.InvalidatedAtUtcsays when.Fixed: the fact that names the user keeps the subject
user. With canonical subjects on, "Hi! I'm Dana" resolved "user" into the person Dana, and stored the naming fact asDana | is named | Dana. The name was never found again.Review:
Measured (banded ×3, same five turns, all facts right): GLM-5.3-Flash is 1.16 s to first word and 1.53 s to answer with reasoning off. Reasoning off and low differ by less than the noise on Flash; low adds tail spikes on the larger models. The default model stays.
Checks at
3c01061: Unit 5,690 · Extensibility 42 · SemanticKernel 54 · LongMemEval 910 · Integration 546 (29 skipped), all passing. Perf gate PASS on the zero and remote profiles; the scenarios now expect this sprint's savings (reasoning traces the formatter hides are not searched; one embedding request for a request's new names). Sample-run gate: every sample builds, and MinimalAgent and AgentWithMemory run end to end on live Neo4j and a hosted model.🤖 Generated with Claude Code