Skip to content

ORM: cross-index top-level OR plans a bare table scan instead of unioning index ranges #441

Open

Description

@MikeyZhang75

Summary

A top-level or() whose disjuncts touch different indexed fields compiles to strategy: 'none', selectedIndex: null, and executes as a full table scan — even when every disjunct is individually indexed.

The same-field case works. That makes this a sharp, surprising cliff: or(eq(status,'a'), eq(status,'b')) is index-backed, or(eq(status,'a'), eq(age,99)) scans the table.

Measured: 302 documents walked to return 2.

Root cause

packages/kitcn/src/orm/where-clause-compiler.ts:44 still carries a singular index:

selectedIndex: IndexLike | null

and the executor's CompiledQueryPlan mirrors it with a single optional index, whose own comment at packages/kitcn/src/orm/query.ts:600 scopes the union to in/ne/notIn/same-field OR plans.

So the compiler has no way to express "probe A on by_status, probe B on by_age", and falls back to no index at all.

Not fixed by the recent work

The prerequisite named in the audit did not happen. #425 changed exactly 17 lines of where-clause-compiler.ts, all of them a constant rename plus export (MAX_PROMOTED_PROBES → export MAX_INDEX_UNION_PROBES) and the reformat of its one call site. It never went near OR compilation. #426 is the only other commit touching that file and only added order-aware index choice in findLeadingIndex.

Why this is now well-scoped

#425 already built the executor half. _buildPlanStream can merge probe streams — it just cannot address more than one index. The remaining work is to let the plan carry per-probe index info.

Suggested fix

  1. Widen the exported WhereClauseResult so probes carry their own index — replace the singular selectedIndex: IndexLike | null at where-clause-compiler.ts:44 with per-probe { index: IndexLike; filters: FilterExpression<boolean>[] } entries.
  2. Teach _buildPlanStream to merge streams anchored on different indexes.

This is a breaking change to a public type. WhereClauseResult is exported, and update.ts / delete.ts each run their own probe fan-out off their own compiler instance, so both need updating in the same change.

Testing note

This was historically unpinnable — the read-count harness could not see rows rejected behind a .filter(). #399 fixed that: convex/setup.testing.ts now exposes a scanned counter, and convex/orm/read-counting.test.ts already demonstrates a cross-index OR reporting documents: 2, scanned: 302. So a read-bound regression test is now expressible — assert on scanned, seeded well past the bound.

Existing tests

None pin cross-index OR read bounds. #425's new convex/orm/index-union-pagination.test.ts covers in() / cursor pagination only.

Environment

  • kitcn 0.32.1
  • convex 1.44.0
  • verified against origin/main at 14bab503

Activity

  1. self-assigned this
    on Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions