Skip to content

ORM: no per-transaction memo lifetime — FK probes, CLEARING probes, and relation targets re-read once per row #406

Closed

Description

@MikeyZhang75

Summary

The ORM has four memo lifetimes. The one several read-amplification defects need — per transaction — does not exist, so three separate call sites re-issue an identical read once per row instead of once per statement.

This is an enabler issue. On its own it changes no behaviour; it exists so the three consumers below stop being three independent hacks.

Current memo lifetimes

Lifetime Mechanism Site
isolate WeakMap keyed on schema identity orm/database.ts:105-106, orm/query.ts:156
per-query instance field orm/query.ts (_countIndexReadinessByKey)
per-statement local let orm/update.ts:718 (foreignKeysProbed)
per-relation-load local Map orm/query.ts (relationCountExecutionCache)
per-transaction absent —

The anchor already exists

orm/lifecycle.ts:23:

const ORMLIFECYCLE_INNER_DB = Symbol.for('kitcn:OrmLifecycleInnerDB');

read back at :66-69:

export const getOrmLifecycleInnerDb = (
  ...
  const inner = (db as any)[ORMLIFECYCLE_INNER_DB];

and installed at :675:

Object.defineProperty(wrappedDb, ORMLIFECYCLE_INNER_DB, {

That handle is a stable per-transaction identity — exactly the key a WeakMap<innerDb, Cache> needs.

Consumers this unblocks

  1. insert() foreign-key re-probe — one db.get(foreignId) per inserted row for the same parent. update() already memoizes this per statement (orm/update.ts:718); insert() never got the equivalent. Measured 8 reads for 8 rows sharing one FK, where 1 suffices.
  2. assertAggregateIndexesWritable (orm/aggregate-index/runtime.ts:3088) — re-probes the by_table_status CLEARING range once per written row. Measured exactly 1:1 with row count (12 rows → 12 probes, 40 → 40), against an empty range every time.
  3. Relation-where target resolution — the same target document is re-resolved per parent rather than once per page. Measured 1050 db.get for 2 distinct targets.

The trap

prependWriteBarrier's closure (orm/lifecycle.ts:754) is constructed inside createOrmDbLifecycle, which createOrm calls at module scope (orm/create-orm.ts:301). A naive let probed = false there has isolate lifetime, not transaction lifetime, and would permanently disable the write barrier for the whole isolate — a correctness regression, not just a missed optimization.

The memo must key on the stable per-transaction writer reached via getOrmLifecycleInnerDb, never on the closure.

Suggested fix

Introduce one per-transaction cache owner keyed on the inner writer:

const perTxn = new WeakMap<object, TxnCache>();

with a small typed accessor so the three consumers share one mechanism rather than each inventing a local Map. Land the primitive with one consumer to prove it, then port the other two.

Staleness note

Consumer 2 is safe to memoize: every COUNT_STATUS_CLEARING write happens inside backfill mutations (orm/aggregate-index/backfill.ts) that write only aggregate_* tables, so a user write transaction cannot flip a state row to CLEARING mid-loop.

Environment

  • kitcn 0.25.10
  • convex 1.44.0
  • verified against origin/main at 00f7f121

Activity

  1. self-assigned this
    on Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions