fix(db): SIGSEGV when fetch/query race a segment switch; unblock reader concurrency

[YongqiYin]

Closes: #714

What

Both read entry points crash with SIGSEGV on Linux when a writer crosses a
segment switch (details in #714). fetch() faults inside the in-memory table
rebuild; query() faults on the same store, reached through the planner's
fan-out (VectorRecallNode::collect_batch → SegmentImpl::fetch → fetch_normal → convertToTable). Two races and two read-path changes:

1. Segment list — get_all_segments() takes write_mtx_ shared
   (collection.cc), as stats() already does; _unsafe variant added for
   callers already holding the lock. Probe on main: 99.7% of doc_ids_
   push_backs executed while a reader was inside this function.
2. Segment teardown — finish_memory_components() /
   init_memory_components() take seg_col_mtx_ exclusively, and the two
   combined-vector-indexer accessors take it shared; dump() takes seg_mtx_
   exclusively. Readers used to reach memory_store_ / persist_stores_ /
   the indexer maps while flush() was republishing them under a different
   lock, so a reader landing between memory_store_->close() and
   persist_stores_.push_back() found neither and dropped that block's rows
   with no error and no log. Locking the rebuild covers all three flush()
   entry points (buffer-full insert, segment switch, close).
3. Point-read fast path — MemForwardStore::convertToTable() materializes
   only the batch holding the requested rows instead of rebuilding the whole
   in-memory store: O(store) → O(1), and it no longer walks the stack the
   crash faults in.
4. Reader locks — seg_mtx_ and cache_mtx_ become shared_mutex;
   read-only accessors take them shared (verified member by member).
   close() now takes cache_mtx_ exclusively, with flush_locked()
   extracted so it does not lock twice.

Also from review: the three convertToTable helpers moved to snake_case, and
tests/db/concurrent_fetch_test.cc was renamed concurrent_read_test.cc now
that it covers both read paths.

Test plan

tests/db/concurrent_read_test.cc, one writer crossing frequent segment
switches throughout:

• fetch: 4/8 readers comparing every field of every fetched doc against
  the deterministic generator output. 3/3 SIGSEGV on unfixed main, passes
  with the fix (2.4M field comparisons on Linux, 1.6M on macOS, 0 errors).
• query: 4 queriers running KNN with an output field, asserting no errors,
  no empty results, no missing field values, and that each value matches the
  pk it came back with. 3/3 SIGSEGV on unfixed main, passes with the fix
  (19.5k queries on Linux, 16.5k on macOS, 0 errors).

Two settings are load-bearing: max_doc_count_per_segment = 4,000
(schema-validated ≥1,000) so switches happen inside the run window, and the
writer is capped at 50k docs and asserted to be alive (writer_errors plus
a minimum docs_written) — otherwise a starved writer silently turns the run
into a no-writer one and the test passes without exercising the crash path.

Full suites: 222 tests on macOS arm64, 224 on Linux x86_64.

Measured impact (64-core Linux; same-binary repeat variance <4%)

metric	before	after
point-read latency (500→4,000 docs in the in-memory store)	472→1,982µs, grows with doc count (O(N))	flat ~180µs at any doc count (O(1))
fetch throughput 1→8 threads, no writer	flat (no reader-reader parallelism)	5.8x scaling (5.6k→32k reads/s)
fetch throughput 1→8 threads, with writer	crashes	6.4x scaling (8.7k→56k reads/s)
query throughput 1→8 threads, no writer	1.15x (597→685 q/s) — materialization serialized under the exclusive lock	3.4x (600→2,038 q/s)
writer throughput as readers 0→8	starves (290 docs/s, macOS) / crashes (Linux)	flat (−2.4%)

Known trade-offs

• Reads now wait for the in-flight write batch. Before this PR, readers
  fetched the segment list with no lock at all — that unlocked read is exactly
  the race being fixed. After it, get_all_segments() takes write_mtx_
  shared, so a reader waits for the current write_impl batch. This is the
  only newly-blocked path at the collection layer: create_index /
  add_column already blocked readers for the whole DDL task (they hold the
  schema lock exclusively), and per-insert exclusion already existed at the
  segment layer.
• Reads now wait for a segment teardown or rebuild. Item 2 blocks readers
  for the duration of dump() (avg 16ms, max 24ms, 26 occurrences over an
  8s window in the regression test) and of a memory-component rebuild.
  Previously they were not blocked — they read state that was being torn down.
  Measured cost is inside the ±1% run-to-run noise on every workload above.
• Writers can be passed by concurrent readers. With std::shared_mutex, a
  writer waiting for a segment lock must wait out the readers currently inside
  it. The wait stays bounded: new readers cannot arrive (the writer holds the
  collection's exclusive write_mtx_) and each in-flight read is now an O(1)
  lookup. Writer throughput stays flat as readers go 0→8 (−2.4%).

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[YongqiYin]

评审期间上游合入了 #722（synchronize reads of writing segment）和 #726（synchronize flat cache and
collection block switching），修的是本 PR 同一批竞态，但方向相反：

• 上游：query/fetch 全程持 shared write_mtx_，相当于查询期间冻结整个集合。语义简单、一致性强，
  但读取进行期间写者无法推进。
• 本 PR：仅在取段列表时短暂持 shared write_mtx_，之后由段级 shared 锁保证段内容一致，读写并发。

把两套方案在同一台机器上做了对照实测。

测试条件

tests/db/concurrent_read_test.cc，（Linux，64 核），每组 3 次，8 秒窗口：

• 写侧：1 个线程按 100 条/批持续 insert，上限 50,000 条；max_doc_count_per_segment=4000，
  运行期间会反复跨段切换
• 读侧：4 个线程做点查（逐字段校验内容），或 4 个线程做 KNN topk=10
• 读写同时进行，下表三个指标取自同一次运行

结果

指标	上游锁方案	本 PR	倍数
fetch 吞吐（4 读者 + 1 写者）	5,135–5,352/s	88,564–89,380/s	~17x
query 吞吐（4 查询者 + 1 写者）	1,672–1,718/s	2,441–2,455/s	~1.4x
写者吞吐（与上面读负载并发）	12 docs/s——8 秒内只完成 1 批 100 条	6,250 docs/s——写满 50,000 条上限	~520x

写者停滞的原因是读者饥饿：glibc 的 rwlock 读者优先，读者全程持 shared write_mtx_ 时，持续的读流量
会让排他写者长时间拿不到锁。macOS 的 libc++ 写者优先，因此只在 Linux 复现。该现象已用纯上游代码
（不含本 PR 任何改动）独立复现 3/3。

正确性对齐

#722 移除了 collection_test.cc 中对 fetch table failed 瞬态的容忍，断言更严格。本 PR 在该测试下
Linux 5/5、macOS 8/8 通过；新增的高压并发测试约 5.7 万次 query 也是 tolerated=0——段锁合并后
internal_insert 的元数据发布与读者严格互斥，顺带消除了这个瞬态。

合并的处理

保留本 PR 的锁设计，并吸收上游的覆盖面：补上 get_vector_indexer() /
get_quant_vector_indexer() 的 shared 锁。

[egolearner]

private？

[egolearner]

[P1] The reader lock does not cover all writers of the segment metadata. In CollectionImpl::switch_to_new_segment_for_writing(), remove_writing_forward_block() (collection.cc:1730) runs after dump() has released seg_mtx_. A concurrent query can therefore pass has_writing_forward_block() in the CombinedVectorColumnIndexer constructor and race with the optional reset before value() or the BlockMeta copy. This is a data race with potential exceptions/crashes, not merely a weakly consistent read. Could we move this metadata removal into a Segment operation protected by the exclusive seg_mtx_?