Skip to content

perf(evmonlyapp): hash a block's transactions before taking the sender lock - #4235

Merged
bdchatham merged 4 commits into
giga-1from
brandon/serial-2-take-senders
Sep 17, 2026
Merged

bdchatham merged 4 commits into
giga-1from
brandon/serial-2-take-senders

Conversation

@bdchatham

@bdchatham bdchatham commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

One of four independent PRs from the serial-budget survey (#4234, #4235, #4236, #4237); each targets giga-1 on its own.

takeSenders keccaked every raw transaction inside the checkedSenders lock, on the execute loop. The hashing is the whole cost of the pass, and CheckTx writes that same map about 64,500 times a second — so admission stalled for the length of a block's worth of hashes.

The hashes are now taken across the pool first, and the lock covers only the map lookups.

Measured

Measured locally with a throwaway BenchmarkHashRawTxs (not included in this PR). 1,848 transactions, 10-core machine:

time
keccak on the loop (what it did) 791 µs — 428 ns/tx
across the pool (what it does) 231 µs

560 µs off a 28.83 ms loop, ~1.9%. A validator at GOMAXPROCS 32 has more of the pool to spread it over, so this is the floor. It costs one 59 KB slice of hashes per block.

Two things the survey did not have: the keccak is 428 ns/tx, slightly above its 300–420 ns estimate; and the lock was held across the whole pass, which it did not mention and which is the part that touches admission latency rather than throughput.

Correctness

  • Lookups still run in index order, so a block carrying the same transaction twice resolves the first and leaves the second unknown — unchanged.
  • TestHashRawTxsMatchesKeccak256Hash pins hashRawTxs to crypto.Keccak256Hash over 0, 1, 2, 17 and 1,848 inputs. This matters more than it looks: the map is keyed by tx.Hash(), so a hash that differs by one byte would silently lose every sender and make the executor recover all 1,848 signatures instead.
  • One hasher is reset per worker rather than allocated per transaction.
  • Ranges are contiguous, not strided, so workers do not share cache lines while writing.
  • -race clean on the package.

Link to Devin session: https://app.devin.ai/sessions/0952787e9df84a7994fda23fea5db400
Open in Devin Desktop: https://app.devin.ai/desktop/session/0952787e9df84a7994fda23fea5db400?variant=devin
Requested by: @bdchatham

@cursor

cursor Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

PR Summary

Low Risk
Behavior change is limited to when work runs and how hashes are computed; correctness hinges on Keccak parity with the prior per-tx hashing, which the new test exercises across small and large batch sizes.

Overview
Moves Keccak hashing out of the checkedSenders mutex so FinalizeBlock no longer holds that lock while hashing every raw transaction—a path that contended with high-frequency CheckTx writes to the same map.

takeSenders now calls hashRawTxs first, then locks only for map lookup/delete. hashRawTxs parallelizes hashing across GOMAXPROCS when there are more than 64 txs per worker, using a shared hashRawTxRange helper that resets one Keccak hasher per goroutine instead of allocating per tx.

Adds TestHashRawTxsMatchesKeccak256Hash so batch hashes stay identical to crypto.Keccak256Hash, which is required for sender-cache keys to match CheckTx.

Reviewed by Cursor Bugbot for commit 23e892a. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedSep 17, 2026, 1:30 PM

@codecov

codecov Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.70%. Comparing base (7e3ca24) to head (23e892a).
⚠️ Report is 26 commits behind head on giga-1.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           giga-1    #4235       +/-   ##
===========================================
+ Coverage   65.55%   78.70%   +13.14%     
===========================================
  Files        2081        7     -2074     
  Lines      157460      756   -156704     
===========================================
- Hits       103222      595   -102627     
+ Misses      54097      161    -53936     
+ Partials      141        0      -141     
Flag Coverage Δ
sei-chain ?
sei-chain-pr 83.62% <100.00%> (?)
sei-db 74.50% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sei-tendermint/internal/evmonlyapp/app.go 82.63% <100.00%> (+0.86%) ⬆️

... and 2074 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@seidroid seidroid Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving the keccak pass out of the checkedSenders lock and across the pool is correct and behaviour-preserving: hashes stay index-aligned, lookups still run in order so the duplicate-transaction case is unchanged, workers write disjoint contiguous ranges of hashes, and TestHashRawTxsMatchesKeccak256Hash pins the Reset/Write/Read hasher reuse to crypto.Keccak256Hash (the equality that would otherwise silently drop every sender). Two non-blocking notes: the chunking degenerates to one goroutine per transaction on blocks smaller than GOMAXPROCS, and the new godoc paragraphs carry rationale that AGENTS.md asks be kept inline.

Findings: 0 blocking | 2 non-blocking | 2 posted inline

Blockers

  • None at the file/PR level.

Non-blocking

  • None at the file/PR level.
  • 2 suggestion(s)/nit(s) flagged inline on specific lines.

Comment thread sei-tendermint/internal/evmonlyapp/app.go Outdated
Comment thread sei-tendermint/internal/evmonlyapp/app.go Outdated
@devin-ai-integration
devin-ai-integration Bot force-pushed the brandon/serial-2-take-senders branch 2 times, most recently from 851ae16 to d0397fd Compare September 17, 2026 13:01
bdchatham and others added 4 commits September 17, 2026 13:28
…r lock

takeSenders keccaked every raw transaction inside the checkedSenders lock, on
the execute loop. The hashing is the whole cost of the pass, and CheckTx writes
the same map about 64,500 times a second, so admission stalled for the length
of a block's worth of hashes.

The hashes are now taken across the pool first, and the lock covers only the
map lookups. Measured over 1,848 transactions on a 10-core machine:

    keccak on the loop   791 us  (428 ns/tx)
    across the pool      231 us

That is 560 us off a 28.83 ms loop, and a validator at GOMAXPROCS 32 has more
of the pool to spread it over. It costs one 59 KB slice of hashes per block.

The lookups still run in index order, so a block carrying the same transaction
twice resolves the first and leaves the second unknown, as before. One hasher is
reset per worker rather than allocated per transaction, and
TestHashRawTxsMatchesKeccak256Hash pins the result to crypto.Keccak256Hash: a
hash that differs by a byte would silently lose every sender and make the
executor recover them all again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration
devin-ai-integration Bot force-pushed the brandon/serial-2-take-senders branch from 9c4915a to 23e892a Compare September 17, 2026 13:28
@devin-ai-integration
devin-ai-integration Bot changed the base branch from brandon/serial-1-parse-workers to giga-1 September 17, 2026 13:28
@bdchatham
bdchatham requested a review from masih September 17, 2026 13:43
@bdchatham
bdchatham added this pull request to the merge queue Sep 17, 2026
Merged via the queue into giga-1 with commit abeab64 Sep 17, 2026
71 of 72 checks passed
@bdchatham
bdchatham deleted the brandon/serial-2-take-senders branch September 17, 2026 15:30
revofusion pushed a commit to revofusion/sei-chain that referenced this pull request Sep 29, 2026
…r lock (sei-protocol#4364)

Backports sei-protocol#4235 from
`giga-1` to `main` (PLT-1303, part of PLT-1297). `takeSenders` keccaks
every raw transaction while holding the `checkedSenders` lock on the
execute loop, and `CheckTx` writes that same cache constantly, so
admission stalls for the length of a block's worth of hashes (~790 µs
for 1,848 txs on the original measurement).

`takeSenders` now computes all hashes first with `hashRawTxs`, which
splits the block across up to `GOMAXPROCS` workers (at least
`minTxsPerHashWorker` = 64 txs each, so small blocks stay on one
goroutine), and the lock covers only the `senderCache.take` lookups.
Main has since replaced the plain map with the two-generation
`senderCache`, so the port keeps that structure and only moves the
hashing out of the lock. Unlike the `giga-1` version, the fan-out uses
`scope.Parallel` instead of a bare `sync.WaitGroup` + `go func`, per the
sei-tendermint conventions.

No consensus or state impact: lookups still run in block index order, so
a duplicate tx in a block still resolves only the first occurrence.
`TestHashRawTxsMatchesKeccak256Hash` pins `hashRawTxs` to
`crypto.Keccak256Hash` across the single-worker and multi-worker paths;
the evmonlyapp package tests pass under `-race`, and `make lint` / `make
fmtcheck` are clean.

Link to Devin session:
https://app.devin.ai/sessions/d808db90106341278344623de10b92c0
Open in Devin Desktop:
https://app.devin.ai/desktop/session/d808db90106341278344623de10b92c0?variant=devin
Requested by: @bdchatham

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants