Use Pebble batch directly in SS

[alrevuelta]

• Batching is used when inserting keys into SS. Meaning we aggregate tons on updates into a single batch.
• However the current code was using an intermediate buffer for the batch. This means that we were allocating a buffer for the whole batch (which can be eg 500k of keys) to then copy that again to Pebble "native" batch.
• This produced many many unnecessary memory allocations.
• This PR reduces allocations (for a 500k keys batch) from 1,011,200 to only 11,300 and insertion time (of an ordered batch) from 90ms to 79 ms.
• It simply uses the native Pebble batch type directly, without using an intermediate batch.

[cursor]

PR Summary

Medium Risk
Touches the core state-store write path and batch lifecycle; behavior changes include no pre-write sort and stricter batch reuse rules, though tests cover round-trips and apply semantics.

Overview
State store batching no longer buffers operations in an intermediate slice before copying into Pebble. Batch now wraps pebble.Batch and MVCC-encodes each Set/Delete/HardDelete via deferred writes, which cuts allocations on large changesets (per PR description: ~1M → ~11k for ~500k keys) and slightly speeds ordered applies.

NewBatch reserves byte capacity via changesetBatchSize (replacing op-count changesetPairs); ApplyChangesetSync uses that sizing and defer Close. Batches are single-use: Write commits and releases; Close drops without commit; further ops return errBatchClosed. RawBatch, pre-commit sorting (sortBatchOps, SortChangesetPairs), and RawSnapshotNode are removed—changesets are no longer reordered in place. Import/delete paths use unsized batches with explicit Close; DeleteKeysAtVersion returns wrapped errors instead of printing to stdout.

Tests/benchmarks were updated for zero-allocation Set, batch sizing, single-use semantics, and unchanged input pairs after apply.

^{Reviewed by Cursor Bugbot for commit 2be5dc4. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	✅ passed	✅ passed	✅ passed	Sep 23, 2026, 9:59 AM

[codecov]

Codecov Report

❌ Patch coverage is 69.66292% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.32%. Comparing base (e4fe987) to head (2be5dc4).

Files with missing lines	Patch %	Lines
sei-db/db_engine/pebbledb/mvcc/db.go	14.81%	23 Missing ⚠️
sei-db/db_engine/pebbledb/mvcc/batch.go	93.54%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4300      +/-   ##
==========================================
- Coverage   67.42%   66.32%   -1.10%     
==========================================
  Files        2179     2058     -121     
  Lines      167292   155265   -12027     
==========================================
- Hits       112797   102985    -9812     
+ Misses      54485    52270    -2215     
  Partials       10       10              

Flag	Coverage Δ
sei-chain-pr	77.77% <69.66%> (?)
sei-db	74.50% <ø> (ø)
sei-db-state-db	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
sei-db/db_engine/types/types.go	37.50% <ø> (ø)
sei-db/db_engine/pebbledb/mvcc/batch.go	92.68% <93.54%> (+18.73%)	⬆️
sei-db/db_engine/pebbledb/mvcc/db.go	77.35% <14.81%> (-0.73%)	⬇️

... and 121 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 8246527. Configure here.}

[seidroid]

Swapping the intermediate op buffer for a native pebble.Batch is a solid allocation win, but moving the sort out of the batch and into ApplyChangesetSync now mutates caller-owned changeset slices from the async writer goroutine, racing the synchronous commit-store apply and hash logging. The new Write()-closes-the-batch lifecycle is also undocumented and leaves one error path operating on a released pebble batch.

Findings: 1 blocking | 5 non-blocking | 3 posted inline

Blockers

• None at the file/PR level.
• 1 blocking issue(s) flagged inline on specific lines.

Non-blocking

• [suggestion] NewBatch now acquires a pebble.Batch eagerly, but only Write() releases it. Every early return err in ApplyChangesetSync (db.go:710, 713) and the error exits in DeleteKeysAtVersion abandon the batch without Close(), so it is never returned to pebble's batchPool. A defer that closes the batch unless Write() already did (or a Close()/Release() method on Batch) would make the lifecycle a property of the type rather than a convention each caller has to remember.
• [suggestion] Sorting is now done only in ApplyChangesetSync; Import lost it. Previously writeBatchOps sorted every batch before commit, so each ImportCommitBatchSize chunk hit the memtable in comparer order. Import feeds from a channel consumed by ImportNumWorkers goroutines, so its batches are no longer ordered — by this PR's own premise (ordered insertion is measurably faster) that is a regression on the import path. Worth either sorting in the import worker or noting the measured impact.
• [suggestion] Test coverage for the two behaviours the refactor actually changed is missing: (1) nothing asserts that ApplyChangesetSync leaves its input []*proto.NamedChangeSet unmodified — a test on that would have caught the in-place sort above, and it is cheap (snapshot the pair order, apply, compare); (2) nothing pins that a Batch is single-use after Write(). TestSortBatchOps (which checked the sort agreed with MVCCComparer in both version directions) was replaced by TestSortChangesetPairs, which only re-tests bytes.Compare on three keys and covers none of the new surface.
• 2 suggestion(s)/nit(s) flagged inline on specific lines.

[alrevuelta]

@seidroid review

[seidroid]

Replacing the intermediate []batchOp buffer with a native pebble.Batch is a clean allocation win, and the three findings from the previous review (the cross-goroutine sort race, the undocumented single-use lifecycle, and the Batch godoc) are all genuinely fixed — with Close()/errBatchClosed, errors.Join cleanup at every exit, and new tests pinning both. Two non-blocking points remain: the batch is no longer sorted before it reaches Pebble on any path, and a new godoc overstates what DeleteKeysAtVersion removes.

Findings: 0 blocking | 4 non-blocking | 1 posted inline

Blockers

• None at the file/PR level.

Non-blocking

• [suggestion] Encoding directly into the pebble.Batch removes the sort that writeBatchOps previously applied on every write path, and nothing replaces it (SortChangesetPairs is deleted with no caller). Base always handed Pebble a batch in comparer order; head hands it whatever order the caller produced, and PopChangeSet produces write order, not key order. Pebble's memtable apply uses arenaskl.Inserter, whose splice cache only pays off for ascending input, so unordered application loses that. The PR describes the 90ms→79ms number as "insertion time (of an ordered batch)", i.e. the case where the sort was already a no-op scan — the unordered case, which is what ApplyChangesetSync, Import and DeleteKeysAtVersion actually feed, is the one that could regress. BenchmarkBatchWrite already builds random (unordered) EVM keys, so a base-vs-head run of it would settle this; worth posting that number, or sorting at the producer, before merging.
• [suggestion] Import now passes bufSize: 0 (db.go:1272, db.go:1295), so each 10k-record batch grows its buffer by doubling from scratch. Base sized the ops slice by ImportCommitBatchSize and then derived an exact pebble buffer from it via pebbleBatchBufSize, so the import path previously reserved once. Given the PR's premise is eliminating regrowth, an estimated batchRecordSize-based reservation (or reusing one batch via Reset across chunks) would keep that property on the import path.
• 1 suggestion(s)/nit(s) flagged inline on specific lines.
• 1 non-blocking pre-existing issue(s) listed below under pre-existing issues.

Pre-existing issues

• [suggestion] DeleteKeysAtVersion (sei-db/db_engine/pebbledb/mvcc/db.go:1379) drives its deletes from RawIterate, which continues past any entry whose value is tombstoned (db.go:1359). Keys deleted at the target version therefore keep their tombstone records after the "physical delete" runs. Present on the base branch.

Superseded: latest AI review found no blocking issues.