[alex09x]

Problem

prod-code answered workspace/symbol, references and the other index questions while the language server was still loading or indexing: empty, or with whatever it had indexed so far, and nothing said so (#391). v0.3.14's #381 only guessed, retrying an empty answer from an engine younger than 30 s, and a partial answer passed as the whole one. The servers do report when they are ready; the gateway's engines never asked for it (window.workDoneProgress) and ignored $/progress.

A probe on a Linux build node, asking workspace/symbol every 250 ms on generated projects:

clangd, 2,000 files      progress 'indexing' 0.10s .. 4.32s; 7 empty answers, first hit at 1.81s (partial)
gopls, 2,000 files       progress 'Setting up workspace' 0.15s .. 0.35s; holds the question, 0 empty
basedpyright, 20,000     no progress; log 'Found 20000 source files' at 0.97s; 2 empty answers before it
tsc --lsp, 2,000 files   no progress; holds the question, 0 empty

Change

• prod_code_protocol::readiness: a Readiness tracker fed with every message from a server. It follows what the server reports:

  • work-done progress it creates, begins, reports and ends;
  • for ReadySignal::Log, the log line that says it is set up;
  • for progress servers, a 500 ms window after initialized, in which work that has not begun yet is still expected.

  wait(max) returns when the server is ready, or returns what it is still doing when the time is up. needs_index lists the index questions.

• Engines (gopls, generic):

  • they declare window.workDoneProgress and feed the tracker;
  • send_request of an index question waits, up to 30 s (GenericLspConfig::index_wait), and attaches the unfinished work to the answer as prodCodeIndexing.

  Signals per server:

  • clangd and sourcekit-lsp: progress;
  • basedpyright and pyright: the Found N source files / No source files found. line;
  • native tsc: holds its own questions;
  • anything else: unknown, answers taken as they come.

• Gateway:

  • turns prodCodeIndexing into a prod-code/indexing notification sent just before the answer;
  • the handshake gains index_gated: true for the in-process Rust engine, gopls, and servers whose readiness is known.

• Client:

  • the MCP session records the notes, and every tool prints them under its answer: "(the language server was still indexing (…) for N s when asked: this answer may be incomplete)";
  • workspace_symbol_search makes no retry for a gated engine;
  • index questions get a 120 s budget.

• Fixed on the way (each surfaced by the gateway's live Go test once progress was declared):

  • a server request taken for an answer. gopls numbers its own requests from 1, as the engine and the client number theirs. The engines matched window/workDoneProgress/create to a pending question. The gateway passed gopls's requests on, and the client took workspace/configuration (id 2) for the answer to its workspace/symbol (id 2). Only a message without a method is an answer now, in both engines, the MCP session, the CLI's own query path and the bench. The gateway no longer passes a server's own requests on for engines that answer them.
  • a copy named like a hidden directory. The live test's checkout is a tempfile directory, .tmpXXXX, and its copy took that name. Go tools skip directories beginning with . or _, so gopls's first workspace/symbol came back null and only the opened file was found, 50 ms later. A probe showed only the module root's own name matters. sanitize_identifier now names such a copy dot-… / under-….

Reproduce and verify

A C++ checkout of 900 files that each include <regex>, <iostream> and <map> and call func0(). Two identical copies, each asked prod-code refs --symbol func0 three times, 5 s apart, from its first contact with a cold clangd on the same Linux build node. 0.3.14 used the production gateway and client; this branch used its debug gateway on a spare port and the dev client:

0.3.14:  run 1: 0 references in 1.3 s   ("No references found.")
         run 2: 315 references in 0.2 s
         run 3: 715 references in 0.2 s
branch:  run 1: 899 references in 14.9 s
         run 2: 899 references in 0.3 s
         run 3: 899 references in 0.3 s

In the branch gateway's log, the first workspace/symbol took 14,250 ms (waiting for clangd's index) and the references after it 17 ms. With 2,000 one-line files, both versions answer 997 references: that is clangd's own 1,000-result limit, so that project could not tell them apart.

New tests. The two server-request tests were run against the code without their fix and failed as described; the others cover behaviour that did not exist before:

• readiness unit tests (7): work begun or created but not ended is busy; a wait ends when the work ends or when the time is up; a numeric token, and an end without a begin, are handled; the pyright line; unknown servers never wait; Busy on the wire.
• a_question_from_the_index_waits_until_the_server_has_indexed and a_server_still_indexing_when_the_wait_ends_is_asked_with_a_note (generic engine, fake server that indexes).
• a_server_request_with_the_id_of_a_question_is_not_its_answer (generic engine). Without the fix, the answer is the server's create request.
• the_first_symbol_search_of_a_fresh_gopls_finds_the_function (Go engine, real gopls).
• an_answer_given_while_the_server_indexes_says_so and a_server_request_with_the_questions_id_is_not_its_answer (MCP). Without the client fix: "No references found."
• an_empty_symbol_search_is_asked_again_only_of_an_engine_still_loading gains the gated case: one search, however young the engine.
• a_server_request_is_told_from_a_notification and a_copy_is_never_named_as_a_hidden_directory (gateway).
• the_handshake_says_how_old_the_engine_is asserts index_gated on a real gateway.
• the_gateway_forwards_to_a_child_language_server (live, real gopls) passes with no retry at all: 3 of 3 alone, and in the full workspace run.

Checks

Linux build node (x86_64), the whole workspace in one run:

$ cargo fmt --all -- --check                               -> exit 0
$ cargo clippy --workspace --all-targets -- -D warnings    -> Finished, exit 0
$ cargo test --workspace --no-fail-fast
34 test targets: test result: ok (no failures), prod-code-gateway --test live 17 passed

Closes #391

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-26T05:28:29.073180Z	3a081af	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3a081afff9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Surface indexing notifications in direct CLI queries

When prod-code implementations is asked while the server is still indexing after the 30-second gate, the gateway sends a prod-code/indexing notification immediately before the response. This loop rejects every method-bearing message and discards that notification, then returns the potentially partial result without the warning that the MCP path displays. Retain and render the indexing notification while waiting for response ID 2.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Drain indexing notes from every checkout queried

For code_schema_rename with repos, rename_across runs LSP renames with each secondary repository as the session root, so record_indexing_note stores any timeout warning under that repository. This function drains only the primary root, leaving secondary-repository warnings out of the response and allowing a partial cross-repository rename to be reported or applied silently. Associate notes with the request result or collect them for every queried root.

Useful? React with 👍 / 👎.