Skip to content

autobahn: split the execute loop's storage phase into vault commit / app commit / push app hash / prune - #4263

Merged
bdchatham merged 3 commits into
giga-1from
devin/1789850569-storage-tail-timing
Sep 20, 2026
Merged

bdchatham merged 3 commits into
giga-1from
devin/1789850569-storage-tail-timing

Conversation

@bdchatham

@bdchatham bdchatham commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

The Autobahn execute loop's storage phase (autobahn_main_loop_phase_duration_seconds_total{phase="storage"}) is ~0.20–0.28 s/s on testnet-2 today, but it covers several very different steps run serially after FinalizeBlock returns: commitAppHashToVault (a pebble Set with pebble.Sync when Fsync is on), app.Commit (cursor promotion; the state commit itself is pipelined since #4254), r.data.PushAppHash, and then PruneBefore / hashVault.Prune in the loop. Which of them dominates decides the next optimization, and whether it touches the equivocation guard's durability at all, so this PR only measures.

It adds a second phase timer on the same autobahn meter, autobahn_storage_tail_phase_duration_seconds_total{phase=...}, with phases vault_commit, app_commit, push_app_hash, bookkeeping, prune_data and prune_vault. Each named phase covers exactly the call it names: the loop's own work between them (committeeWeights, finalizeBlockGasUsed/PushGasUsed, the executed-block window) is charged to bookkeeping, and EndStoragePhase() closes the timer before the loop goes back to waiting for the next block, so idle time is charged to none of them. Counter only, one series per phase; the existing main-loop timer is untouched and the new phases together are a slice of its storage phase.

No behavior change. Focused -race router tests, lint, vet and make fmtcheck are clean; the numbers themselves come from testnet-2 after rollout.

Link to Devin session: https://app.devin.ai/sessions/ff612badcded4aa5914ea408dbb41888
Open in Devin Desktop: https://app.devin.ai/desktop/session/ff612badcded4aa5914ea408dbb41888?variant=devin
Requested by: @bdchatham

…commit / push app hash / prune

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@cursor

cursor Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

PR Summary

Low Risk
Metrics-only instrumentation in the execute loop; no changes to commit, vault, or pruning behavior.

Overview
Adds finer-grained Prometheus timing for Autobahn’s post-FinalizeBlock “storage tail” so operators can see which serial step dominates the existing coarse storage phase.

A second PhaseTimer on the autobahn meter (autobahn_storage_tail_phase_duration_seconds_total) records phases vault_commit, app_commit, push_app_hash, bookkeeping, prune_data, and prune_vault. The giga router sets these around commitAppHashToVault, app.Commit, PushAppHash, committee/gas bookkeeping, and data/vault pruning; EndStoragePhase() resets the timer before the loop waits on the next block so idle time is not attributed to any sub-phase. The existing autobahn_main_loop timer is unchanged.

Reviewed by Cursor Bugbot for commit 6f52f0d. Bugbot is set up for automated code reviews on this repo. Configure here.

@codecov

codecov Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.63%. Comparing base (7e3ca24) to head (6f52f0d).
⚠️ Report is 42 commits behind head on giga-1.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           giga-1    #4263       +/-   ##
===========================================
+ Coverage   65.55%   83.63%   +18.08%     
===========================================
  Files        2081       22     -2059     
  Lines      157460     2047   -155413     
===========================================
- Hits       103222     1712   -101510     
+ Misses      54097      334    -53763     
+ Partials      141        1      -140     
Flag Coverage Δ
sei-chain ?
sei-chain-pr 83.63% <100.00%> (?)
sei-db ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
giga/metrics/autobahn_loop.go 92.59% <100.00%> (ø)
sei-tendermint/internal/p2p/giga_router_common.go 78.41% <100.00%> (+0.21%) ⬆️

... and 2063 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedSep 19, 2026, 9:50 PM

@seidroid seidroid Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Measurement-only change that adds a second PhaseTimer (autobahn_storage_tail) splitting the Autobahn execute loop's storage phase into five sub-phases. The mechanism is correct — Reset() before waiting means idle time is charged to no phase, and the lazy-init/mutex pattern mirrors the existing MainLoop timer — but two of the sub-phases absorb bookkeeping work the PR description says they exclude.

Findings: 0 blocking | 1 non-blocking | 1 posted inline

Blockers

  • None at the file/PR level.

Non-blocking

  • None at the file/PR level.
  • 1 suggestion(s)/nit(s) flagged inline on specific lines.

Comment thread sei-tendermint/internal/p2p/giga_router_common.go
…own phase

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@bdchatham
bdchatham added this pull request to the merge queue Sep 20, 2026
Merged via the queue into giga-1 with commit 368a799 Sep 20, 2026
62 checks passed
@bdchatham
bdchatham deleted the devin/1789850569-storage-tail-timing branch September 20, 2026 02:35
wen-coding pushed a commit that referenced this pull request Sep 24, 2026
Backport of the giga/metrics package and its execute-loop phase hooks from
giga-1 commit 0484ca1 (#4172). Dashboard and autobahn-e2e tooling changes
are left out; #4263 builds on this timer.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
wen-coding pushed a commit that referenced this pull request Sep 24, 2026
…app commit / push app hash / prune (#4263)

The Autobahn execute loop's `storage` phase
(`autobahn_main_loop_phase_duration_seconds_total{phase="storage"}`) is
~0.20–0.28 s/s on testnet-2 today, but it covers several very different
steps run serially after `FinalizeBlock` returns: `commitAppHashToVault`
(a pebble `Set` with `pebble.Sync` when `Fsync` is on), `app.Commit`
(cursor promotion; the state commit itself is pipelined since #4254),
`r.data.PushAppHash`, and then `PruneBefore` / `hashVault.Prune` in the
loop. Which of them dominates decides the next optimization, and whether
it touches the equivocation guard's durability at all, so this PR only
measures.

It adds a second phase timer on the same `autobahn` meter,
`autobahn_storage_tail_phase_duration_seconds_total{phase=...}`, with
phases `vault_commit`, `app_commit`, `push_app_hash`, `bookkeeping`,
`prune_data` and `prune_vault`. Each named phase covers exactly the call
it names: the loop's own work between them (`committeeWeights`,
`finalizeBlockGasUsed`/`PushGasUsed`, the executed-block window) is
charged to `bookkeeping`, and `EndStoragePhase()` closes the timer
before the loop goes back to waiting for the next block, so idle time is
charged to none of them. Counter only, one series per phase; the
existing main-loop timer is untouched and the new phases together are a
slice of its `storage` phase.

No behavior change. Focused `-race` router tests, lint, vet and `make
fmtcheck` are clean; the numbers themselves come from testnet-2 after
rollout.

Link to Devin session:
https://app.devin.ai/sessions/ff612badcded4aa5914ea408dbb41888
Open in Devin Desktop:
https://app.devin.ai/desktop/session/ff612badcded4aa5914ea408dbb41888?variant=devin
Requested by: @bdchatham

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 368a799)
wen-coding pushed a commit that referenced this pull request Sep 25, 2026
…app commit / push app hash / prune (#4263)

The Autobahn execute loop's `storage` phase
(`autobahn_main_loop_phase_duration_seconds_total{phase="storage"}`) is
~0.20–0.28 s/s on testnet-2 today, but it covers several very different
steps run serially after `FinalizeBlock` returns: `commitAppHashToVault`
(a pebble `Set` with `pebble.Sync` when `Fsync` is on), `app.Commit`
(cursor promotion; the state commit itself is pipelined since #4254),
`r.data.PushAppHash`, and then `PruneBefore` / `hashVault.Prune` in the
loop. Which of them dominates decides the next optimization, and whether
it touches the equivocation guard's durability at all, so this PR only
measures.

It adds a second phase timer on the same `autobahn` meter,
`autobahn_storage_tail_phase_duration_seconds_total{phase=...}`, with
phases `vault_commit`, `app_commit`, `push_app_hash`, `bookkeeping`,
`prune_data` and `prune_vault`. Each named phase covers exactly the call
it names: the loop's own work between them (`committeeWeights`,
`finalizeBlockGasUsed`/`PushGasUsed`, the executed-block window) is
charged to `bookkeeping`, and `EndStoragePhase()` closes the timer
before the loop goes back to waiting for the next block, so idle time is
charged to none of them. Counter only, one series per phase; the
existing main-loop timer is untouched and the new phases together are a
slice of its `storage` phase.

No behavior change. Focused `-race` router tests, lint, vet and `make
fmtcheck` are clean; the numbers themselves come from testnet-2 after
rollout.

Link to Devin session:
https://app.devin.ai/sessions/ff612badcded4aa5914ea408dbb41888
Open in Devin Desktop:
https://app.devin.ai/desktop/session/ff612badcded4aa5914ea408dbb41888?variant=devin
Requested by: @bdchatham

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 368a799)
revofusion pushed a commit to revofusion/sei-chain that referenced this pull request Sep 25, 2026
…ei-protocol#4330)

## Summary
- Backport Autobahn execute-loop phase metrics from giga-1 (`sei-protocol#4172` Go
path, plus `sei-protocol#4263` storage-tail split) so `/metrics` can show consensus
/ execution / storage, then vault commit / app commit / push app hash /
prune.
- Count txs packed into locally produced lane blocks
(`tendermint_internal_autobahn_avail_produced_txs`, `sei-protocol#4180`).
- Time `FinalizeBlock` stages around the EVM-only executor (`sei-protocol#4259`).

Rebased onto current `main`. Regenerated
`sei-tendermint/internal/autobahn/avail/metrics/metrics.gen.go`; it was
already in sync.

## Test plan
- [x] `go test ./giga/metrics/... ./giga/evmonly/...`
- [x] `go test ./internal/autobahn/avail/... ./internal/evmonlyapp/...`
(from `sei-tendermint`)
- [ ] On an Autobahn node, confirm
`sei_chain_autobahn_main_loop_phase_duration_seconds_total`,
`sei_chain_autobahn_storage_tail_phase_duration_seconds_total`, and
`tendermint_internal_autobahn_avail_produced_txs` appear on `/metrics`


Made with [Cursor](https://cursor.com)

Closes CON-426
Closes PLT-1309
Closes PLT-1308

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: FromTheRain <bdchatham@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Philip Su <philip@seinetwork.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant