Fix unsigned index key order for high values #5338

Status and target

Deferred to v5.1.0 by Ivan on 2026-10-08. Research and local reproduction are available; no production fix or PR has been implemented. This issue tracks audit A2: Fix unsigned index key order for high values.

Source: Platform v5 devnet audit, finding #9 (aggregates/t-u8.mjs). Independently reproduced on platform commit ad63a4861b6df3ff0a7e8de6879e10bc93221179 with pinned GroveDB 9791d277207998b7789c85b0745e99d37040a12e.

Reproduction and impact

Create a contract with a grade integer property bounded from 0 to 255, sized integer types enabled, and an ascending index configured as countable and rangeCountable. Store grades [50, 110, 130, 180, 220].

Proof queries match their originating requests and authenticate the actual database root. This is incorrect numeric-to-byte query interpretation, not a forged Merkle proof.

Local boundary probes also reproduce incorrect ranges, ascending/descending order and limits for U8/U16/U32/U64 around 2^(width-1). Equality and IN membership remain correct for homogeneous encodings, but IN numeric ordering can be wrong. Signed controls and a PV8 I64 query control pass. U128 has an encoder-level finding only; normal-schema query reachability is not established.

Cause and compatibility constraints

The unsigned DPP document-property index encoders invert the top bit, producing physical order [upper half, lower half]. That transformation is appropriate for signed integers, not unsigned ones. Sized integer types are enabled by default starting at PV9.

Simply removing the flip would change every persisted unsigned key, including low values, and can break equality, uniqueness, proofs and existing writers. Do not globally change signed, Date/system-height, or Drive system-tree encodings as a side effect. Actual constructed Withdrawals system contracts use sized types disabled and I64 fields at PV8/9/13/14; schema bounds alone do not establish U32 encoding.

A separate historical schema-update defect is a migration prerequisite: PV9/PV13 accept U8-to-U16 width changes without reindexing. An existing grade-7 row then disappears from new-schema equality queries; its primary bytes still authenticate and decode under the old schema but fail under the new one. PV14 rejects the update. A repair cannot assume all persisted keys/documents match the current schema. Historical repair is not implicitly included in this issue.

Design work remaining for 5.1

Evaluate and choose a reviewed approach before implementation:

• External query correction over unchanged storage, including complete numeric ordering, global limits/cursors, aggregate proofs and client compatibility.
• Client composition of existing two-sided half-bounded requests as a narrower alternative. Feasibility is not yet demonstrated; consistent roots/heights, round trips, compound-query limits and raw clients remaining affected must be addressed.
• Authenticated per-contract encoding version for prevention, plus a separately designed migration if existing data is to be rewritten.
• Explicit containment if a complete correction cannot yet ship; do not describe partial coverage or refusal as a full fix.

Preserve consensus-facing builders and historical replay. Any migration needs retained-state inventory, deterministic bounded progress/retry/rollback, reference/index-only handling and explicit collision/undecodable-state policy. No silently dropped data or arbitrary winners.

Acceptance

• Reproduce then fix numeric range/count/order/limit regressions across reachable unsigned widths, with signed and historical controls.
• Verify original-request coverage, numeric result oracles and authenticated roots; exercise missing/overlapping/reordered proof components where applicable.
• Cover pagination/cursors, compound index behavior and supported aggregate forms; explicitly document unsupported/residual cases.
• Establish persisted-state, writer/uniqueness and old/new client compatibility for the chosen approach.
• Obtain independent design/code review and passing final-head CI.

The local research snapshot ran 11 tests: seven controls/observations passed and four unsigned correctness assertions failed as expected. Working spec, review, raw logs and untracked probe are preserved in the audit workspace; they are research artifacts, not a shipped fix. The proved-document range failure was observed at Drive only; an SDK node-ban consequence has not been measured.