pow%perf: provide alternate software AES backends for Groestl and SHAvite, precompute final block schedule, run NTT of 16-bit lanes for SIMD, unroll and inline BMW by kwvg · Pull Request #80 · dashpay/base-sdk

kwvg · 2026-10-22T07:40:36Z

Motivation

The system requirements imposed by dash-pow are aimed at full nodes and other server-side applications but are unacceptable for mobile devices

For instance, targeting iOS sets +apple-a7 by default which supports AES acceleration (source) but targeting Android sets +v8a which treats it as optional (the whole situation around ARM extensions is troublesome on the best of days, see BoringSSL guidance for a general summary, source).

A minimum requirement for transitioning away from sphlib-based FFIs is performance at-par or better than the existing implementation. This means we had to rework the portable (i.e. software-only) backend to achieve performance improvements to justify the switchover for downstream consumers.

Additional Information

The Groestl software backend builds the fused round column-first, holding the state as sixteen u64 columns and running each round with one table of SubBytes and the MixBytes column, rotated by 8r bits for row r.
- This is the single-table variant of the Groestl specification (source, section 8.1.2), where the circulant MixBytes matrix makes T1 through T7 byte rotations of T0, and the T-table method itself originates in the Rijndael proposal (source, section 5.2.1).
- The hardware backend keeps the row-first construction of the specification's AES-NI strategy (source, section 8.1.1) and of Byte Slicing Groestl (source).
The SHAvite-3 software AES rounds run as T-table lookups on plain u32 words, with each key after the first entering as the key operand of the round before it, which is the keyed T-table round of the Rijndael proposal (source, section 5.2.1).
The SHAvite-3 message expansion is generated inside the round loop from an eight-bundle window instead of a precomputed 112-entry schedule. The recurrence defined in the SHAvite-3 submission only reaches eight bundles back and the rounds consume bundles in order (source, section 4.2.2), so the window holds the whole live schedule.
Skein unrolls Threefish, four rounds per subkey injection, with the word permutation turned into register renaming and every rotation an immediate. The Skein paper describes exactly this, noting that the period-4 permutation lets an implementation use different registers each round instead of shuffling words (source, sections 7.1 and 8.3).
SIMD precomputes the message schedule of the length-only final block of a 64-byte message, skipping that block's NTT. The block is fixed because it carries only the bit count (source, section 1.2.4). The schedule is generated at compile time with a const fn.
SIMD runs its 256-point NTT over Z/257 on 16-bit lanes as eight parallel FFT-8 passes with transposes, reducing only where a lane could leave i16. SWIFFT is the earliest source of the method, with eight 8-point FFTs across SSE2 registers of signed 16-bit lanes, twiddles as shifts by 2, 4 and 6, and (x & 255) - (x >> 8) reducing only a few intermediate values into [-127, 383] (source, source). The SIMD submission adopts the same field for that reason (source, section 2.1).
BMW writes its message expansion and folding with literal indices, so every index and add/sub sign of the expansion defined in Blue Midnight Wish is a compile-time constant (source, listed only).

How Has This Been Tested?

just fuzz::test comb

Checklist

I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have added or updated relevant unit/integration/functional tests
I have made corresponding changes to the documentation
I did not use agentic assistance (i.e. LLMs, AI) except as disclosed and can answer questions, comments and concerns without said assistance. I assume authorship of the code submitted.
I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

Agent disclosure

Code generated with Claude Code using Opus 5.5, medium effort and Fable 5.1, xhigh effort. Reviewed with Codex using GPT-6-Astra, extra high effort as adversarial review.

coderabbitai · 2026-10-22T07:40:46Z

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 9 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Source mode: Auto
Repository: dashpay/base-sdk
Branch: develop
File: .coderabbit.yml
Commit: e95acf0
Layers: Repository YAML
Review profile: CHILL
Plan: Advanced
Run ID: bc2aa507-8c66-4dc3-a357-e1d3ecd85743

📥 Commits

Reviewing files that changed from the base of the PR and between d03126f and 4dc9ef8.

📒 Files selected for processing: 4

pkgs/pow/src/consts/simd.rs
pkgs/pow/src/groestl/aes_cols.rs
pkgs/pow/src/shavite/common.rs
pkgs/pow/src/simd.rs

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Source mode: Auto
Repository: dashpay/base-sdk
Branch: develop
File: .coderabbit.yml
Commit: 7798ecf
Layers: Repository YAML
Review profile: CHILL
Plan: Advanced
Run ID: d0090980-3828-4e9e-bc65-177095b7d522

📥 Commits

Reviewing files that changed from the base of the PR and between 7798ecf and d03126f.

📒 Files selected for processing: 28

pkgs/pow/src/aes/mod.rs
pkgs/pow/src/aes/scalar.rs
pkgs/pow/src/blake.rs
pkgs/pow/src/bmw.rs
pkgs/pow/src/consts/bmw.rs
pkgs/pow/src/consts/groestl.rs
pkgs/pow/src/consts/simd.rs
pkgs/pow/src/cubehash.rs
pkgs/pow/src/echo.rs
pkgs/pow/src/groestl/aes_cols.rs
pkgs/pow/src/groestl/aes_rows.rs
pkgs/pow/src/groestl/common.rs
pkgs/pow/src/groestl/mod.rs
pkgs/pow/src/jh.rs
pkgs/pow/src/keccak.rs
pkgs/pow/src/luffa.rs
pkgs/pow/src/shavite.rs
pkgs/pow/src/shavite/aes_lanes.rs
pkgs/pow/src/shavite/aes_words.rs
pkgs/pow/src/shavite/common.rs
pkgs/pow/src/shavite/mod.rs
pkgs/pow/src/simd.rs
pkgs/pow/src/skein.rs
pkgs/pow/src/util/arx.rs
pkgs/pow/src/util/macros.rs
pkgs/pow/src/util/memops.rs
pkgs/pow/src/util/mod.rs
pkgs/pow/src/util/threefish.rs

💤 Files with no reviewable changes: 1

pkgs/pow/src/shavite.rs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Summary

Summary by CodeRabbit

New Features
- Added Groestl-512 hashing support.
- Added SHAvite-3-512 hashing support.
Improvements
- Updated several hashing algorithms and AES operations while preserving their existing outputs.
- Improved processing of certain message blocks, including final blocks.

The pull request adds streaming Groestl-512 and SHAvite-3-512 implementations with software and hardware AES backends. It replaces the SIMD transform and schedule extraction pipeline and adds a specialized path for exact 64-byte messages. BMW and Threefish compression now use compile-time expansion. Shared macros and word-loading helpers support these changes. Several hash source headers receive attribution updates, and Skein uses the shared word-loading helper.

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant Groestl512
  participant Buffer
  participant Core
  participant Backend
  Caller->>Groestl512: update input
  Groestl512->>Buffer: buffer input
  Buffer->>Core: compress full blocks
  Core->>Backend: compress blocks and transform state
  Groestl512->>Core: finalize padded input
  Core->>Backend: apply output transform and extract digest
  Groestl512-->>Caller: return digest

sequenceDiagram
  participant Caller
  participant Shavite512
  participant Buffer
  participant Core
  participant AESBackend
  Caller->>Shavite512: update input
  Shavite512->>Buffer: buffer input
  Buffer->>Core: compress full blocks
  Core->>AESBackend: apply AES rounds during compression
  Shavite512->>Core: finalize buffered input
  Core-->>Shavite512: produce digest
  Shavite512-->>Caller: return digest

Suggested reviewers: thepastaclaw

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to d0312

This change swaps in faster software backends for several hash functions and leaves public hashing behavior unchanged. Nothing in the supplied review evidence shows a concrete defect, so the remaining risk is only routine confirmation that the known-answer and fuzz tests pass in CI.

Pre-merge checks: 4

Check name	Status	Explanation
Title check		The title accurately summarizes the main performance changes, including alternate software AES backends, SIMD optimization, final-block precomputation, and BMW unrolling. It is long but specific and c…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Description check		The pull request description directly explains the portable backend performance goal and describes the Groestl, SHAvite, SIMD, BMW, and Skein changes listed in the changeset.

Full details: Title check

Explanation

The title accurately summarizes the main performance changes, including alternate software AES backends, SIMD optimization, final-block precomputation, and BMW unrolling. It is long but specific and clearly related to the changeset.

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-10-22T07:41:00Z

Note

This pull request has no conflicts! 🎊 🎉 🎊

thepastaclaw · 2026-10-22T07:45:24Z

✅ Final review complete — no blockers (commit d03126f) · triage: critical

thepastaclaw

Final validation — Phase 1 + Phase 2

Static verification found no blocking defects and retained one nonblocking suggestion to exercise upper counter words in the rewritten SHAvite schedule. Exact-head CI passed MSRV feature tests, native/portable sphlib replay, stable platform builds, cross-builds, and code-lint steps; the overall Lint job failed only because its PR-title check received the nonconforming title “Refine hash.” No local builds or tests were run.

🟡 1 suggestion(s)

Review provenance

Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: security-auditor); reviewer 4: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 5: muse-spark-1.3-contributor (agent: phase1-reviewer, role: rust-quality); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

Triage: critical by gpt-6.1-sol (effort low) — The diff makes extensive, intricate changes directly to cryptographic hash implementations, including AES backends in pkgs/pow/src/groestl/ and pkgs/pow/src/shavite/, NTT arithmetic in pkgs/pow/src/simd.rs, and the schedule in pkgs/pow/src/bmw.rs.
Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer, muse-spark-1.3-contributor — rust-quality (completed, effort xhigh); agent phase1-reviewer
Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
Phase 2 reviewers: gpt-6.1-sol — general (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — rust-quality (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — security-auditor (completed, effort xhigh); agent phase2-reviewer

🤖 Prompt for all review comments with AI agents

These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `pkgs/pow/src/shavite/common.rs`:
- [SUGGESTION] pkgs/pow/src/shavite/common.rs:36-40: Cover upper counter words in the rewritten rolling schedule
  Add a compression-level known-answer vector with distinct, nonzero values in all four HAIFA counter words, and exercise it under both AES backends. The NIST vectors cover messages through 255 bytes, the largest committed SHAvite corpus file is 55,341 bytes, and the differential sweep stops at 512 bytes, so ordinary CI cannot distinguish permutations of the upper three counter words. The ignored counter_word_carries test reaches only the first 32-bit carry even when explicitly enabled. A vector checked against Dash Core's compression function would validate the rewritten injection ordering and its propagation through all four rolling-schedule checkpoints without hashing enormous inputs. The current assignments match Core on inspection; this is regression coverage, not a demonstrated hash defect. Reference: https://github.com/dashpay/dash/blob/v24.0.0-rc.3/src/crypto/x11/shavite.cpp#L142-L173.

`git diff --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space`

thepastaclaw · 2026-10-22T11:56:04Z

@kwvg on it — I've queued this as job cj-20261022-115603-c577 and will reply here or link a PR when there's something to look at.

thepastaclaw

Re-review — Final validation — Phase 1 + Phase 2

The exact head implements performance-only X11 backend and schedule rewrites without an identified consensus, safety, or compatibility defect. The previously noted SHAvite counter-word coverage gap is fixed by a direct compression known-answer test using distinct nonzero values in all four counter words. All listed build, test, lint, and cross-platform checks succeeded at this head; CodeRabbit remains pending but has supplied no findings.

🔴 0 blocking | 🟡 0 suggestion(s) | 💬 0 nitpick(s)

Review provenance

Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: security-auditor); reviewer 4: gpt-6-astra (agent: phase2-reviewer, role: general); reviewer 5: gpt-6-astra (agent: phase2-reviewer, role: rust-quality); reviewer 6: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 7: gpt-6-astra (agent: phase2-reviewer, role: security-auditor); reviewer 8: muse-spark-1.3-contributor (agent: phase1-reviewer, role: rust-quality); reviewer 9: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 10: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); reviewer 11: gpt-6.1-sol (agent: phase2-reviewer, role: security-auditor); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

Triage: critical by gpt-6.1-sol (effort low) — The diff intricately rewrites cryptographic proof-of-work implementations, including Groestl software AES in pkgs/pow/src/groestl/aes_cols.rs, SHAvite scheduling in pkgs/pow/src/shavite/common.rs, and SIMD NTT arithmetic in pkgs/pow/src/simd.rs, where correctness affects consensus hashes.
Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer, muse-spark-1.3-contributor — rust-quality (completed, effort xhigh); agent phase1-reviewer
Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
Phase 2 reviewers: gpt-6.1-sol — general (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — rust-quality (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — security-auditor (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — general (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — rust-quality (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — security-auditor (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — general (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — rust-quality (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — security-auditor (completed, effort xhigh); agent phase2-reviewer
Model comparison: every Phase-2 reviewer also ran on gpt-6-astra; the verifier weighed both sets without knowing which model wrote which

🤖 Prompt for all review comments with AI agents

These findings are from an automated code review. Verify the current code and confirm that no unresolved issues remain.

No unresolved findings remain from the prior review on this head.

…ations to `const` expressions (#91) * [63a9ee8] pow%refac(echo): load the message words with `load_u32s_le` (Kittywhiskers Van Gogh) * [01e798e] pow%refac(blake): load message words with `util::memops` (Kittywhiskers Van Gogh) * [f926cdf] pow%refac(luffa): share the scalar and vector step bodies via macros (Kittywhiskers Van Gogh) * [f22220c] pow%refac(util): drop unused definitions (Kittywhiskers Van Gogh) * [42a58c5] pow%refac(luffa): pack round constants with `const_array!` (Kittywhiskers Van Gogh) * [88f292c] pow%refac(jh): fold the bit-group swaps into one function (Kittywhiskers Van Gogh) * [87a1e6b] pow%refac(jh): generate tables with `const_array!` (Kittywhiskers Van Gogh) * [cbda3b0] pow%doc: reword comments (Kittywhiskers Van Gogh) * [f320850] pow%refac(cubehash): build round index tables as xor masks (Kittywhiskers Van Gogh) * [15c9fc1] pow%refac(jh): derive `IP_8` and `IV` from the spec permutation (Kittywhiskers Van Gogh) * [a8d99bd] pow%refac(echo): derive the big ShiftRows permutation as 5i mod 16 (Kittywhiskers Van Gogh) * [a061d37] pow%refac(blake): build sigma rounds 10 to 15 from rows r mod 10 (Kittywhiskers Van Gogh) * [a61617c] pow%refac(skein): derive the IV by UBI of the config block (Kittywhiskers Van Gogh) * [d49f11d] pow%refac(shavite): load the message window with `load_u32s_le` (Kittywhiskers Van Gogh) Description: ````` ## Additional Information * Depends on #80 ## Breaking Changes None expected ## How Has This Been Tested? ``` just comb ``` ## Checklist - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have added or updated relevant unit/integration/functional tests - [ ] I have made corresponding changes to the documentation - [x] I did not use agentic assistance (i.e. LLMs, AI) except as [disclosed](#agent-disclosure) and can answer questions, comments and concerns without said assistance. I assume authorship of the code submitted. - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ### Agent disclosure Code generated with Claude Code using Opus 5.5, medium effort. Reviewed with Codex using GPT-6-Astra, extra high effort as adversarial review and Claude Code with Fable 5.1, xhigh effort. `````

kwvg added this to the 0.2 milestone Oct 22, 2026

kwvg self-assigned this Oct 22, 2026

kwvg added this to base-sdk v0.2 Oct 22, 2026

kwvg changed the title ~~Refine hash~~ pow%perf: provide alternate software AES backends for Groestl and SHAvite, precompute final block schedule, run NTT of 16-bit lanes for SIMD, unroll and inline BMW Oct 22, 2026

kwvg moved this to Documentation in base-sdk v0.2 Oct 22, 2026

kwvg moved this from Documentation to Crypto in base-sdk v0.2 Oct 22, 2026

thepastaclaw reviewed Oct 22, 2026

View reviewed changes

Comment thread pkgs/pow/src/shavite/common.rs Resolved

github-actions Bot added the topic:Crypto Pull requests that primarily concern the dash-{pkc,pow} crate. label Oct 22, 2026

kwvg added 2 commits October 22, 2026 17:03

pow%doc: declare sphlib, SUPERCOP and block-ciphers adaptations

e8a89a6

pow%refac(groestl): split the module into common code and a row backend

fa64368

`git diff --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space`

kwvg force-pushed the refine_hash branch from 26115af to d03126f Compare October 22, 2026 11:38

kwvg marked this pull request as ready for review October 22, 2026 11:54

kwvg added 8 commits October 22, 2026 18:11

pow%perf(groestl): build the fused table column-first for software AES

4eb5873

pow%refac(shavite): split common code from the vector AES backend

2ba23de

pow%perf(shavite): run table rounds on plain words where AES is software

8c0c613

pow%perf(shavite): generate the key schedule inside the round loop

daf34cb

pow%perf(skein): unroll Threefish

0fac873

pow%perf(simd): precompute the schedule of the 64-byte final block

9897bf2

pow%perf(simd): run the NTT on 16-bit lanes

236b191

pow%perf(bmw): unroll the schedule with literal indices, inline helpers

4dc9ef8

thepastaclaw approved these changes Oct 22, 2026

View reviewed changes

kwvg force-pushed the refine_hash branch from d03126f to 4dc9ef8 Compare October 22, 2026 12:44

kwvg merged commit d41bb4c into dashpay:develop Oct 22, 2026
13 of 15 checks passed

This was referenced Oct 22, 2026

feat(hashes)!: switch proof of work backend to dash-pow, bump Base SDK crates to dashpay/base-sdk@addedfa dashpay/rust-dashcore#1214

Merged

pow%refac: flesh out util:memops, magic blob reductions, move derivations to const expressions #91

Merged

Navigation Menu

Repository navigation

pow%perf: provide alternate software AES backends for Groestl and SHAvite, precompute final block schedule, run NTT of 16-bit lanes for SIMD, unroll and inline BMW - #80

pow%perf: provide alternate software AES backends for Groestl and SHAvite, precompute final block schedule, run NTT of 16-bit lanes for SIMD, unroll and inline BMW#80
kwvg merged 10 commits into
dashpay:developfrom
kwvg:refine_hash

kwvg commented Oct 22, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 22, 2026 •

edited

Loading

Review limit reached

Summary by CodeRabbit

Sequence Diagram(s)

Uh oh!

github-actions Bot commented Oct 22, 2026

Uh oh!

thepastaclaw commented Oct 22, 2026 •

edited

Loading

Uh oh!

thepastaclaw left a comment

Uh oh!

Uh oh!

thepastaclaw commented Oct 22, 2026

Uh oh!

thepastaclaw left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

kwvg commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Additional Information

How Has This Been Tested?

Checklist

Agent disclosure

Uh oh!

coderabbitai Bot commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Summary by CodeRabbit

Sequence Diagram(s)

Uh oh!

github-actions Bot commented Oct 22, 2026

Uh oh!

thepastaclaw commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thepastaclaw left a comment

Choose a reason for hiding this comment

Final validation — Phase 1 + Phase 2

Uh oh!

Uh oh!

thepastaclaw commented Oct 22, 2026

Uh oh!

thepastaclaw left a comment

Choose a reason for hiding this comment

Re-review — Final validation — Phase 1 + Phase 2

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kwvg commented Oct 22, 2026 •

edited

Loading

coderabbitai Bot commented Oct 22, 2026 •

edited

Loading

thepastaclaw commented Oct 22, 2026 •

edited

Loading