Skip to content

ORM: declined index-union on the non-cursor select() path anchors no index, with no guard and no available bound #445

Closed

Description

@MikeyZhang75

Summary

PR #425 fixed the headline case — compiled probes now reach the stream builders through _buildPlanStream. A narrower slice survives: when the index union is declined, _buildPlanStream anchors to no index at all and streams the whole table, and on the non-cursor select() path there is no guard and no way to bound it.

Measured on origin/main, 120-row table, only the last row matching:

db.query.users.select()
  .where({ status: { in: [65 values] } })
  .map((r) => r.name)
  .limit(1)
  .execute()
// -> 120 documents read
db.query.users.findMany({ where: { status: { in: [65 values] } }, limit: 1 })
// -> 1 document read

Same result, 120× the reads, purely because the pipeline path was taken.

Root cause — three things combine

1. Every fallback rung is gated on !hasProbeUnionPlan. packages/kitcn/src/orm/query.ts:3122:

    if (queryConfig.index && !hasProbeUnionPlan) {

with the same !hasProbeUnionPlan on the configuredIndex and orderIndexName rungs below. So a plan that has probeFilters but whose union was declined falls past every rung and walks by_creation_time with no range.

The union is declined at query.ts:3027 when there are more than MAX_INDEX_UNION_PROBES (64, where-clause-compiler.ts:98) probes, or when the orderBy sits outside the probes' eq-pinned index prefix.

2. The new strict guard only covers cursor pagination. query.ts:6202:

          const rejectedProbeUnion =
            isCursorPaginated &&

so a non-cursor select() + stage read with a declined union gets neither a throw nor a warning.

3. maxScan is unreachable on that path. query-builder.ts:304 types it never on SelectPipelineBaseConfig, and query.ts:5911 rejects it at runtime for non-cursor reads. .limit(n) does not help either — the stream take(n) keeps pulling until it finds n matches.

Adding a cursor is the only escape: select().paginate({ cursor, limit, maxScan }) is correctly guarded and bounded (verified — 10 reads with maxScan: 10, and the un-maxScanned call throws).

Suggested fix

Let the declined-union case fall through to the ordinary index rungs instead of being gated out — a declined union should degrade to "anchor the best single index available", not to "anchor nothing".

Then either extend the rejectedProbeUnion guard to the non-cursor path, or make maxScan reachable there so the caller has an option at all.

Existing tests

#425's convex/orm/index-union-pagination.test.ts (418 lines, uses the scanned counter) covers non-paginated findMany, cursor, endCursor, residual filters, and four declined-union strict-maxScan cases. No test exercises the non-cursor select() / pipeline path with a declined union — that is the gap.

Environment

  • kitcn 0.32.1
  • convex 1.44.0
  • verified and measured against origin/main at 14bab503

Activity

  1. self-assigned this
    on Sep 5, 2026
  2. zbeyens commented on Sep 7, 2026

    @zbeyens
    Member

    Fixed in #456 (merged).

    Verify with an indexed filter containing more than 64 values:

    • Compare a limited select() query with the equivalent findMany; both should return the expected matches without reading the whole table.
    • Page through the results and confirm no matches are duplicated or skipped.
    • Add a contains condition and a small limit; reads should stop after enough matches when index order satisfies the sort.
    • For newest-first pagination across values, include orderBy and maxScan.
  3. added a commit that references this issue on Sep 13, 2026
    5b0bd5a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions