Repository navigation
autobahn: split the execute loop's storage phase into vault commit / app commit / push app hash / prune - #4263
Conversation
…commit / push app hash / prune Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
PR SummaryLow Risk Overview A second Reviewed by Cursor Bugbot for commit 6f52f0d. Bugbot is set up for automated code reviews on this repo. Configure here. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## giga-1 #4263 +/- ##
===========================================
+ Coverage 65.55% 83.63% +18.08%
===========================================
Files 2081 22 -2059
Lines 157460 2047 -155413
===========================================
- Hits 103222 1712 -101510
+ Misses 54097 334 -53763
+ Partials 141 1 -140
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
|
The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).
|
There was a problem hiding this comment.
Measurement-only change that adds a second PhaseTimer (autobahn_storage_tail) splitting the Autobahn execute loop's storage phase into five sub-phases. The mechanism is correct — Reset() before waiting means idle time is charged to no phase, and the lazy-init/mutex pattern mirrors the existing MainLoop timer — but two of the sub-phases absorb bookkeeping work the PR description says they exclude.
Findings: 0 blocking | 1 non-blocking | 1 posted inline
Blockers
- None at the file/PR level.
Non-blocking
- None at the file/PR level.
- 1 suggestion(s)/nit(s) flagged inline on specific lines.
…own phase Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…app commit / push app hash / prune (#4263) The Autobahn execute loop's `storage` phase (`autobahn_main_loop_phase_duration_seconds_total{phase="storage"}`) is ~0.20–0.28 s/s on testnet-2 today, but it covers several very different steps run serially after `FinalizeBlock` returns: `commitAppHashToVault` (a pebble `Set` with `pebble.Sync` when `Fsync` is on), `app.Commit` (cursor promotion; the state commit itself is pipelined since #4254), `r.data.PushAppHash`, and then `PruneBefore` / `hashVault.Prune` in the loop. Which of them dominates decides the next optimization, and whether it touches the equivocation guard's durability at all, so this PR only measures. It adds a second phase timer on the same `autobahn` meter, `autobahn_storage_tail_phase_duration_seconds_total{phase=...}`, with phases `vault_commit`, `app_commit`, `push_app_hash`, `bookkeeping`, `prune_data` and `prune_vault`. Each named phase covers exactly the call it names: the loop's own work between them (`committeeWeights`, `finalizeBlockGasUsed`/`PushGasUsed`, the executed-block window) is charged to `bookkeeping`, and `EndStoragePhase()` closes the timer before the loop goes back to waiting for the next block, so idle time is charged to none of them. Counter only, one series per phase; the existing main-loop timer is untouched and the new phases together are a slice of its `storage` phase. No behavior change. Focused `-race` router tests, lint, vet and `make fmtcheck` are clean; the numbers themselves come from testnet-2 after rollout. Link to Devin session: https://app.devin.ai/sessions/ff612badcded4aa5914ea408dbb41888 Open in Devin Desktop: https://app.devin.ai/desktop/session/ff612badcded4aa5914ea408dbb41888?variant=devin Requested by: @bdchatham --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit 368a799)
…app commit / push app hash / prune (#4263) The Autobahn execute loop's `storage` phase (`autobahn_main_loop_phase_duration_seconds_total{phase="storage"}`) is ~0.20–0.28 s/s on testnet-2 today, but it covers several very different steps run serially after `FinalizeBlock` returns: `commitAppHashToVault` (a pebble `Set` with `pebble.Sync` when `Fsync` is on), `app.Commit` (cursor promotion; the state commit itself is pipelined since #4254), `r.data.PushAppHash`, and then `PruneBefore` / `hashVault.Prune` in the loop. Which of them dominates decides the next optimization, and whether it touches the equivocation guard's durability at all, so this PR only measures. It adds a second phase timer on the same `autobahn` meter, `autobahn_storage_tail_phase_duration_seconds_total{phase=...}`, with phases `vault_commit`, `app_commit`, `push_app_hash`, `bookkeeping`, `prune_data` and `prune_vault`. Each named phase covers exactly the call it names: the loop's own work between them (`committeeWeights`, `finalizeBlockGasUsed`/`PushGasUsed`, the executed-block window) is charged to `bookkeeping`, and `EndStoragePhase()` closes the timer before the loop goes back to waiting for the next block, so idle time is charged to none of them. Counter only, one series per phase; the existing main-loop timer is untouched and the new phases together are a slice of its `storage` phase. No behavior change. Focused `-race` router tests, lint, vet and `make fmtcheck` are clean; the numbers themselves come from testnet-2 after rollout. Link to Devin session: https://app.devin.ai/sessions/ff612badcded4aa5914ea408dbb41888 Open in Devin Desktop: https://app.devin.ai/desktop/session/ff612badcded4aa5914ea408dbb41888?variant=devin Requested by: @bdchatham --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit 368a799)
…ei-protocol#4330) ## Summary - Backport Autobahn execute-loop phase metrics from giga-1 (`sei-protocol#4172` Go path, plus `sei-protocol#4263` storage-tail split) so `/metrics` can show consensus / execution / storage, then vault commit / app commit / push app hash / prune. - Count txs packed into locally produced lane blocks (`tendermint_internal_autobahn_avail_produced_txs`, `sei-protocol#4180`). - Time `FinalizeBlock` stages around the EVM-only executor (`sei-protocol#4259`). Rebased onto current `main`. Regenerated `sei-tendermint/internal/autobahn/avail/metrics/metrics.gen.go`; it was already in sync. ## Test plan - [x] `go test ./giga/metrics/... ./giga/evmonly/...` - [x] `go test ./internal/autobahn/avail/... ./internal/evmonlyapp/...` (from `sei-tendermint`) - [ ] On an Autobahn node, confirm `sei_chain_autobahn_main_loop_phase_duration_seconds_total`, `sei_chain_autobahn_storage_tail_phase_duration_seconds_total`, and `tendermint_internal_autobahn_avail_produced_txs` appear on `/metrics` Made with [Cursor](https://cursor.com) Closes CON-426 Closes PLT-1309 Closes PLT-1308 --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: FromTheRain <bdchatham@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Philip Su <philip@seinetwork.io>
The Autobahn execute loop's
storagephase (autobahn_main_loop_phase_duration_seconds_total{phase="storage"}) is ~0.20–0.28 s/s on testnet-2 today, but it covers several very different steps run serially afterFinalizeBlockreturns:commitAppHashToVault(a pebbleSetwithpebble.SyncwhenFsyncis on),app.Commit(cursor promotion; the state commit itself is pipelined since #4254),r.data.PushAppHash, and thenPruneBefore/hashVault.Prunein the loop. Which of them dominates decides the next optimization, and whether it touches the equivocation guard's durability at all, so this PR only measures.It adds a second phase timer on the same
autobahnmeter,autobahn_storage_tail_phase_duration_seconds_total{phase=...}, with phasesvault_commit,app_commit,push_app_hash,bookkeeping,prune_dataandprune_vault. Each named phase covers exactly the call it names: the loop's own work between them (committeeWeights,finalizeBlockGasUsed/PushGasUsed, the executed-block window) is charged tobookkeeping, andEndStoragePhase()closes the timer before the loop goes back to waiting for the next block, so idle time is charged to none of them. Counter only, one series per phase; the existing main-loop timer is untouched and the new phases together are a slice of itsstoragephase.No behavior change. Focused
-racerouter tests, lint, vet andmake fmtcheckare clean; the numbers themselves come from testnet-2 after rollout.Link to Devin session: https://app.devin.ai/sessions/ff612badcded4aa5914ea408dbb41888
Open in Devin Desktop: https://app.devin.ai/desktop/session/ff612badcded4aa5914ea408dbb41888?variant=devin
Requested by: @bdchatham