Repository navigation
feat(platform)!: integerRange index buckets an integer property into windows - #5170
Merged
Merged
Conversation
…to windows
Adds the integerRange index keyword to the v3 document meta-schema
(protocol version 14): { on, range, step, phase } buckets the index's
first property, a required integer property of at most 64 bits, into
windows starting at phase + k * step. A start below the lowest value the
property's integer type holds is clamped to it, so every value lands in
at least one window. The parsed grid carries the source's key encoding,
resolved from the schema on every parse.
IndexBucketing is the shared view of a timeRange or integerRange grid;
IndexLevel now stores it. indexOnly proof indexes, refersTo lookups and
propertyConstraints answering indexes exclude integer-bucketed indexes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Carries the dpp integerRange keyword through the stack: - Drive walkers (insert v2, delete v2, update v1), indexOnly paths, cost estimation and the storage layout read one IndexBucketing view; the uniqueness probe (v1) looks in the candidate's window and lets a document change its value within its own window. - IN_INTEGER_RANGE = 12 with a typed IntegerRangeSelection operand on the v1 getDocuments wire, resolved to a window-start equality from the query alone on the server and in every SDK proof verifier; the resolution provenance now carries either kind of grid. - DocumentQuery::with_integer_range[_grid] in dash-platform-queries and an integerRange member on the wasm-sdk document, aggregate, ranked and having queries; documentTypeLayout describes integer windows. Review fixes: the grid rules, overlap cap and grid selection shared by both kinds; a nested source must sit in required objects; the level key must fit 255 bytes; indexOnly types refuse integerRange (rows differing only in the bucketed integer would collide per window); a width change of the source is reported as the property's integer-encoding change; documentCreateCost prices every window; the composite and chained SDK guards refuse integer selections; refusals name the right operator. Regenerated dapi-grpc clients and grovedb-structure.json; book chapter contract-keywords/integer-range.md and v14 note 57. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Warning Review limit reachedNext included review available in 2 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Repository: dashpay/platform/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (92)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
the
waiting-bots
Collaborator
|
🕓 Queued for automated review — 3rd in line, estimated start in ~20 min (commit 6de8c9f)
|
…oupings-93dcc1 Resolves contested.md (integerRange mention next to moderatorAbilities.delete) and v14.rs (#5158 keeps note 57; integer-range indexes become note 58). Generated clients regenerate identically on the merged proto. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
🌳 GroveDB structure This pull request changes the described GroveDB structure. Open it in the structure viewer: new nodes glow, removed ones stay as ghosts, and the tour walks through each change. Changed (2 nodes)
Compared |
Contributor
|
📖 Book Preview built successfully. Download the preview from the workflow artifacts. Updated at 2026-09-29T05:05:38.732Z |
…oupings-93dcc1 Brings in $moderatedAt/$moderatedBy (#5161) and skipIfAbsent at any position (#5162). Conflicts resolved: - try_from_schema common: resolve the integerRange key type and the skipIfAbsent set on every parse. - insert/delete walkers v2: import both bucket_index_keys and level_reaches_entry. - update walker v1: the bucketed set diff keeps skipIfAbsent's empty side for a version the index skips, for time and integer grids. - layout: skip notes read the bucketing source. - indexOnly synthesis: upstream's index_admissible_for_query already keeps bucketed indexes off the terminal route. - v14 notes: skipIfAbsent 59 (it shared 58 with $moderatedAt), integer-range indexes 60. Adds an e2e test of a listing moving into and out of every window of a skipIfAbsent integer-range index, and book text for the combination. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
QuantumExplorer
added a commit
that referenced
this pull request
Sep 29, 2026
romchornyi
pushed a commit
that referenced
this pull request
Oct 2, 2026
v5.0-dev's DocumentQuery gained integer_range_clauses (#5170); the two literals this branch adds leave it empty, as every other DPNS query does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
11 of 15 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Basic explanation
What this does: A contract can already group an index by time windows (
timeRange: "posts per hour"). This adds the same thing for whole numbers:integerRangegroups an index by windows of an integer property, such as a price or a score, so "listings priced 600 to 899, per category" or "players per score band of 100" is one lookup instead of a scan. A query picks one window by where it starts, and counts, sums and rankings on the index then apply to that window, with proofs.Value: Contracts get banded counts, totals and leaderboards over any integer property with the same proof guarantees as time windows, and unique indexes can express "one per band" (for example one bid per owner per price band of 1000).
Risks: Medium-low. It is consensus code (a new index keyword, new walker branches, a new query operator), but it is all new at protocol version 14, which is not yet on mainnet: no stored contract can carry
integerRange, and every shared code path takes its old branch when an index has none. The write walkers now read a shared "bucketing" type instead of the time grid directly; the existing time-range e2e suite passes unchanged. Because the grid rules are now shared, sometimeRangerefusal messages are reworded (and a contract breaking several rules may see a different one first); that is also protocol version 14 only. Two calls for the reviewer: indexOnly types refuseintegerRange(see below), and integer windows reuse the time grid's overlap cap (max_time_range_overlap_factor, 24) rather than a separate limit. The Swift and Kotlin example apps do not display the new keyword yet.Issue being fixed or feature implemented
Time windows (
timeRange) only bucket system timestamps. There was no way to group documents by bands of an integer property, so per-band counts or rankings needed one index entry per exact value and a scan over them.What was done?
Contract grammar (document meta-schema v3, protocol version 14):
onis the index's first property: a required integer property of at most 64 bits (the width and sign come from itsminimum/maximum, as for any integer property). A nested source must sit inside required objects, and the grid-qualified level key must fit GroveDB's 255 bytes.phase + k * stepfor every integerk(negative included) and cover[start, start + range).rangeis a multiple ofstep,range / stepis capped by the existing overlap cap (24),phase < step. Nottl.price#300#100(the time-range level key format).range == step. The source may change on update: the uniqueness probe (v1) looks in the candidate's window and lets a document change its value within its own window.skipIfAbsentproperties (the bucketed one is required, so it never is): a document missing one writes no entry in any window, and an update that adds or drops it moves the document into or out of all of its windows.contested,nullSearchable: false,preallocated,timeRange, a ranking at the bucketed level, as arefersTolookup target, as apropertyConstraintsanswering index, and on indexOnly document types (an indexOnly entry is keyed by the index's values and terminal, so rows that differ only in the bucketed integer would collide in the windows they share).timeRangeandintegerRange.Before / after, per window count:
Before, with a plain
(price, category)index, a "books between 600 and 899" count is a range count over every distinct price. After, with the index above:reads one window key and returns a proved count.
Query surface:
platform.proto:WhereOperator.IN_INTEGER_RANGE = 12with a typedIntegerRangeSelection { start, grid }operand inWhereClause.integer_range.resolve_integer_range_bucket_clauseturns a selection into a window-start equality, from the query alone (no clock). The resolution provenance (ResolvedTimeRange) now carries anIndexBucketing(time or integer grid), so every index picker keeps raw queries off both kinds of bucketed index through the one admissibility rule.IN_TIME_RANGE; chained and composite requests refuse it as they refuse time selections.dash-platform-queries):DocumentQuery::with_integer_range/with_integer_range_grid, encoded on the v1 wire and resolved identically in every proof verifier.integerRange: [{ field, start, grid? }]member on document, count/sum/average, ranked and having queries (startmay be a decimal string aboveNumber.MAX_SAFE_INTEGER).documentTypeLayoutdescribes integer windows (integerRangeBucketkey,integerRangeOverlapnote).Shared code:
IndexBucketing(dpp) is the one view of a time or integer grid: source, level key, overlap factor and the entry-key fan-out.IndexLevelstores it; the insert (v2), delete (v2) and update (v1) walkers, the indexOnly paths and cost estimation read it. TTL handling still reads the time grid only.ttlprices every window an integer-range index writes, anddocumentCreateCostsamples an integer-range source clear of the clamped bottom window so it prices every window.Docs and generated files: new book chapter
contract-keywords/integer-range.md, keyword table rows, cross-references, meta-schema descriptions, and note 60 inv14.rs(this also renumbers theskipIfAbsentnote, which shared number 58 with the$moderatedAtnote on the base branch, to 59). The GroveDB structure description mentions integer windows (regeneratedgrovedb-structure.json), and the dapi-grpc clients are regenerated for the new operator.In-place changes to shipped generations
None of these can change behaviour for a contract that exists today:
integerRangeparses only in the generation-3 grammar (protocol version 14, not yet on mainnet), soIndex::integer_rangeisNoneon every stored contract.try_from_schemageneration 3,Indexparsing: new keyword, admitted only by generation 3.validate_uniqueness_of_datav1 (selected by protocol versions 10 to 13, and by 14 through the v2 wrapper): the integer-window probe and the exemption of a changed integer-range source fromallow_original = falseonly fire whenindex.integer_rangeis set, which no contract before 14 can have. The edited lines say so; the existing PV13 testsvalidate_uniqueness_of_data_v1_*pin the unchanged outcome.single_in_path_queryv0 (every protocol version): the cursor check readsIndex::bucketing()instead oftime_range; both areNonefor every contract before 14. The edited lines say so.propertyConstraintsanswering index:!is_bucketed()equals the oldtime_range.is_none()without integer grids.How Has This Been Tested?
skipIfAbsentinteger-range index as an update adds and drops the skipped property (mutation-checked: letting the new side ignore the skip leaves a null-keyed entry and fails the test).FromProofentry points (each document counted once), a proof of one window refused for another, and malformed selections refused.dppanddrivelib suites, drive-abciquery::document_query,dash-platform-queries(including a composite page refusing a window selection before sending),wasm-sdklib tests, dapi-grpc JS tests; clippy on the touched crates.Breaking Changes
Consensus: a new index keyword and query operator at protocol version 14 (unreleased).
Checklist:
structure.rs, regeneratedgrovedb-structure.json, and checked the structure viewer link posted on this pull request (described and regenerated; the viewer link is not checked yet)🤖 Generated with Claude Code
PR Hygiene ·
6de8c9f/skip-botsproceeds without the ones not yet reported/self-reviewedonce the bots are donedpp— you own itrs-drive-abci— you own itrs-drive— you own itrs-platform-wallet(packages/rs-platform-wallet/src/wallet/identity/network/contact_info.rs,packages/rs-platform-wallet/src/wallet/identity/network/dpns_marketplace.rs,packages/rs-platform-wallet/src/wallet/identity/network/profile.rs) — HashEngineering or ZocoLini or llbartekll or romchornyirust-sdk(packages/rs-sdk/src/platform/dashpay/contact_request_queries.rs,packages/rs-sdk/src/platform/dpns_usernames/mod.rs,packages/rs-sdk/src/platform/dpns_usernames/queries.rsand 1 more) — lklimek or shumkovjs-wasm-sdk(packages/wasm-sdk/src/document_type_layout.rs,packages/wasm-sdk/src/dpns.rs,packages/wasm-sdk/src/queries/chained_document.rsand 3 more) — shumkovWhen every box is checked the
PR Hygienecheck passes and this can merge.