Skip to content

ORM: index-union findMany drops its take() bound entirely under RLS or a residual filter #442

Closed

Description

@MikeyZhang75

Summary

The index-union (multiProbe) findMany lane drops its take() bound entirely whenever either:

  • RLS is enabled on the table, or
  • any post-filter is not Convex-enforceable (e.g. contains)

Each probe then falls back to .collect() and scans its whole probed range.

Measured on origin/main with the scanned counter, findMany({ where: { ownerId: { in: [a, b] } }, limit: 3 }):

Condition Scanned
No RLS, no residual filter 6
RLS enabled 500
Residual filter (contains) 500

The RLS leg is the severe one: turning on a security feature silently removes a read bound, and nothing at the call site indicates it.

Root cause

packages/kitcn/src/orm/query.ts:6871-6872:

this.rls?.mode !== 'skip' && isRlsEnabled(this.tableConfig.table as any)

sets probeHasPostFetchMembership, which nulls probeBound at :6884, which forces .collect() at :6917.

probeHasResidualFilter at :6854-6855 does the same for the whole fan-out whenever any post-filter is not enforceable.

Not fixed by the recent work

#425 rewired the stream sites (cursor pagination, _buildPlanStream). The non-paginated multiProbe branch returns early at query.ts:6835 and never reaches a stream, so it was untouched.

#426 relaxed only the ordering leg of this guard — probeOrderDirection now comes from orderPushdownDirection and the !hasSecondaryOrders clause is gone, so multi-field orderBy no longer costs the bound. That is orthogonal. The residual-filter and RLS legs are byte-identical to the audited code.

Suggested fix

Route the non-paginated multiProbe branch (query.ts:6835) through _buildResidualFilterStream. Post-#425 that already builds a merged probe-union stream via _buildPlanStream and sizes take() by matches rather than scanned rows — which is exactly what the single-index lane at query.ts:7344 already does.

The bound is recoverable because the stream counts survivors; the current code drops it only because a plain db.query fan-out cannot.

Tests to add

Two read-bound cases, asserting on reads.scanned:

  • RLS + in() + limit
  • residual filter (contains) + in() + limit

Both should hold the same absolute bound at two table sizes.

Existing tests

None pin the broken behavior. convex/orm/index-union-pagination.test.ts:240 covers non-paginated multiProbe only where probeBound is already defined (no RLS, no residual filter).

Environment

  • kitcn 0.32.1
  • convex 1.44.0
  • verified and measured against origin/main at 14bab503

Activity

  1. self-assigned this
    on Sep 5, 2026
  2. MikeyZhang75 commented on Sep 6, 2026

    @MikeyZhang75
    CollaboratorAuthor

    Fixed in #455.

    An index-union findMany with a limit now keeps its read bound when the table has RLS enabled, when the where carries a filter that cannot be pushed into Convex such as contains, or when it filters through a relation. Previously each of those made every probe read its whole range.

    To verify:

    1. Seed a table twice — once with 200 rows, once with 500 — spread across two owners, with an index on the owner field.
    2. Run findMany({ where: { ownerId: { in: [a, b] } }, limit: 3 }) against a build with RLS enabled on that table, and again with name: { contains: '...' } added to the where.
    3. Count documents read. It should be the same small number at both table sizes (6 for limit: 3 over two probes), not a number that grows with the table.
    4. Check the page still fills when rows in front of the probes are hidden by the policy or rejected by the filter, and that the returned rows and their order are the same as before.

    Not covered by this change: ne, notIn and isNotNull combined with an orderBy that no index can serve still read their whole complement range. That is a different cost regime and is tracked separately.

  3. zbeyens commented on Sep 7, 2026

    @zbeyens
    Member

    Fixed in #455. Package release skipped.

    QA: run an indexed findMany with two in values and limit: 3, first under
    RLS and then with a contains filter. With matching rows, expect three results
    and six document reads at both 200 and 500 rows. Verify offsets count visible
    matches, mixed-direction ordering stays correct, and relation filters still
    reject requests exceeding relationFanOutMaxKeys.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions