Skip to content

fix(core): date semantic freshness by the latest embedded note - #2064

Open
phernandez wants to merge 2 commits into
mainfrom
semantic-freshness-latest-embedded
Open

phernandez wants to merge 2 commits into
mainfrom
semantic-freshness-latest-embedded

Conversation

@phernandez

Copy link
Copy Markdown
Member

Why

semantic_index_current_as_of was the oldest last-modified time among notes whose embeddings lag. A single note that never catches up dated the whole index back to its own edit. Production's dev project has said "Semantic results current as of 2026-09-07" on every search for two months.

What changed

  • Freshness (repository/semantic_index_freshness.py): the value is now the newest updated_at among markdown notes whose embeddings are current (embedded_checksum = checksum and no deferral marker). It is still reported only while some note is behind, and that check still reads the partial index ix_entity_project_embedding_not_current. When nothing is embedded yet, it falls back to the oldest note behind, because null would claim every note is current.
  • The text note now reads "Notes still waiting for embeddings are found by text search but may be missing from semantic matches." The schema description and bm-search(3) are updated to match.
  • Project list footer: removed "Calls without one use the default project", which is wrong in 1.0. The footer now names the project parameter and its forms.

Tests

  • New test_one_stuck_note_does_not_date_the_index_back. The multi-note repository test now expects the latest embedded change.
  • The MCP freshness tests hold every seeded note, so the text output keeps a fixed timestamp.
  • just fast-check is clean.
  • Freshness, search API and bm_search tests pass on SQLite (67) and Postgres (69 passed, 1 skipped: the SQLite-only plan test).

🤖 Generated with Claude Code

https://claude.ai/code/session_01CMYVL4TcbWdXhdvAzxfqjo

phernandez and others added 2 commits November 5, 2026 22:45
1.0 requires a project on every note, folder, context and schema call, so
"Calls without one use the default project" was wrong. The footer now names
the `project` parameter and its accepted forms, and points search at
`projects` and `all_projects`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CMYVL4TcbWdXhdvAzxfqjo
Signed-off-by: phernandez <paul@basicmachines.co>
semantic_index_current_as_of was the oldest last-modified time among notes
whose embeddings lag. One note that never catches up (a failing embed, a
stuck shard) dated the whole index back to its own edit: production's dev
project has reported "current as of 2026-09-07" on every search for two
months.

The value is now the newest last-modified time among notes whose
embeddings are current, still reported only while some note is behind.
The "is anything behind" check keeps reading the partial index. When
nothing is embedded yet, it falls back to the oldest note behind, since
null would claim every note is current.

The text note now says notes still waiting for embeddings may be missing
from semantic matches, rather than notes changed after the timestamp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CMYVL4TcbWdXhdvAzxfqjo
Signed-off-by: phernandez <paul@basicmachines.co>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-06T04:47:19.196591Z 736143d PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 736143d496

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +49 to +50
return select(func.max(Entity.updated_at)).where(
Entity.project_id.in_(scope.project_ids),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update cross-database aggregation for the new freshness meaning

When a search spans multiple workspace databases, mcp/tools/search.py:679 still merges these values using min(semantic_current_as_of), which matched the former oldest-behind meaning but not the new latest-embedded meaning. For example, if both databases have pending notes and their latest embedded changes are September 7 and November 6, the merged response reports September 7 despite reflecting a November 6 change. The same projects searched within one database now report November 6, so freshness depends on database placement and cross-workspace searches retain the misleading old date this change aims to fix. Update the fan-out aggregation and add a multi-database regression test alongside this semantic change.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant