Skip to content

JDS refactor - #299

Merged
plebhash merged 17 commits into
stratum-mining:mainfrom
plebhash:2026-02-03-jd-server-sv2
Mar 19, 2026
Merged

JDS refactor#299
plebhash merged 17 commits into
stratum-mining:mainfrom
plebhash:2026-02-03-jd-server-sv2

Conversation

@plebhash

@plebhash plebhash commented Feb 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

close #24

  • jd_server_sv2
  • bitcoin_core_sv2
  • pool_sv2 leveraging jd_server_sv2
  • integration tests

@plebhash
plebhash force-pushed the 2026-02-03-jd-server-sv2 branch 5 times, most recently from e7d7bdd to 40f2a41 Compare February 27, 2026 14:31
@GitGab19 GitGab19 mentioned this pull request Feb 27, 2026
Closed
@plebhash
plebhash force-pushed the 2026-02-03-jd-server-sv2 branch 3 times, most recently from 06d8930 to 1d84c3d Compare March 6, 2026 18:19
bit-aloo
bit-aloo reviewed Mar 7, 2026
View reviewed changes
Comment thread pool-apps/jd-server/src/lib/job_declarator/downstream/mod.rs Outdated
Comment thread pool-apps/jd-server/src/lib/job_declarator/downstream/mod.rs Outdated
Comment thread pool-apps/jd-server/src/lib/job_declarator/mod.rs Outdated
Comment thread pool-apps/jd-server/src/lib/job_declarator/mod.rs Outdated
Comment thread pool-apps/jd-server/src/lib/job_declarator/mod.rs Outdated
Comment thread pool-apps/jd-server/Cargo.toml Outdated
Comment thread pool-apps/jd-server/Cargo.toml
Comment thread stratum-apps/Cargo.toml Outdated
@plebhash plebhash mentioned this pull request Mar 8, 2026
Open
@plebhash
plebhash force-pushed the 2026-02-03-jd-server-sv2 branch 4 times, most recently from 2ba3e49 to e8a35c3 Compare March 8, 2026 22:06
plebhash
plebhash commented Mar 8, 2026
View reviewed changes
Comment thread docker/config/pool-jds-config.toml.template

@plebhash plebhash Mar 8, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renaming pool-config.toml.template -> pool-jds-config.toml.template

and making embedded-jds as default

not sure if it's the right move, @lucasbalieiro @GitGab19 please sanity check?

@lucasbalieiro lucasbalieiro Mar 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is gonna break other things on the build process.

And also will probably need to adapt some tricks to run the apps with/without JDP. As we do with the --profile nowadays

If it gets too complex to explain, I'll post a commit so you can cherry-pick it.

I'm still checking the fallout

@plebhash plebhash Mar 10, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah please share commit

plebhash
plebhash commented Mar 8, 2026
View reviewed changes
Comment thread docker/config/pool-jds-config.toml.template
# Protocol Extensions Configuration
# Extensions that the pool supports (will accept if requested by clients)
# Comment/uncomment to enable/disable specific extensions:
supported_extensions = [

@plebhash plebhash Mar 8, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if this is needed

@plebhash plebhash Mar 10, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confirmed this is not needed

@plebhash
plebhash force-pushed the 2026-02-03-jd-server-sv2 branch from e8a35c3 to cbfa504 Compare March 8, 2026 22:19
@plebhash

plebhash commented Mar 8, 2026

Copy link
Copy Markdown
Member Author

@lucasbalieiro can you please do a sanity check on cbfa504?

@plebhash

plebhash commented Mar 8, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

so far I haven't added any new integration tests... might add some before we merge this (open to suggestions)

but marking it as ready for review... hopefully CI should be all green now

@plebhash
plebhash marked this pull request as ready for review March 8, 2026 22:20
@plebhash
plebhash force-pushed the 2026-02-03-jd-server-sv2 branch from cbfa504 to afae806 Compare March 10, 2026 14:53
lucasbalieiro
lucasbalieiro reviewed Mar 10, 2026
View reviewed changes
Comment thread pool-apps/pool/config-examples/mainnet/pool-jds-config-bitcoin-core-ipc-example.toml Outdated
@plebhash
plebhash force-pushed the 2026-02-03-jd-server-sv2 branch 4 times, most recently from 2dadb44 to ea5cc2b Compare March 10, 2026 15:22
@plebhash

plebhash commented Mar 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

the issue @lucasbalieiro reported on today's dev call is a potential race-condition bug on libmultiprocess

In this PR, pool_sv2 now spawns both BitcoinCoreSv2TDP and BitcoinCoreSv2JDP in the same process (two independent IPC client lifecycles). This pattern seems relevant to triggering this bug.

I created this gist and asked @Sjors to reproduce, triage and sanity-check before we report to libmultiprocess repo

it's non-deterministic and more easily reproducible via docker than cargo run

moving forward, we have a few options:

  • introduce aggressive refactors on bitcoin_core_sv2 (plus jd_server_sv2, pool_sv2, translator_sv2 and integration_tests_sv2 as a consequence) with a workaround that makes sure this bug is not triggered
  • harden Docker lifecycle/restart behavior to reduce bug likelihood in normal runs (partial mitigation, not root fix)

personally speaking, I'd lean towards the second option

whenever it gets fixed (either on v31 or some backport v30.3 or v31.1), hopefully we won't have to worry about it anymore

@GitGab19

GitGab19 commented Mar 11, 2026

Copy link
Copy Markdown
Member

I also think we could proceed with the second option, even though it's still not clear to me how we can harden it on docker side (cc @lucasbalieiro).

@Sjors

Sjors commented Mar 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

The gist refers to the 30.x Bitcoin Core branch, which has known crash issues (when the client disconnects mid wait). The fixes are in master, and I'm not sure if they will be back-ported or if it's better to wait for v31.

Maybe use master on CI until a 31.x branch exists?

plebhash and others added 11 commits March 18, 2026 20:50
@plebhash
adapt integration tests to new jd_server_sv2
3e4a47d
@plebhash
bitcoin_core_sv2::template_distribution_protocol::TemplateData::get_t…
481637a 
…x_data skips coinbase tx
@plebhash
adapt jds_ask_for_missing_transactions for new JDS functionality
e3cc60e
@plebhash
disable propagated_from_jds_to_tp
4ab42b9
@plebhash
fix docker config drift validation after jd_server_sv2 refactor
68a31a6
@plebhash
add jds_reject_setup_connection_with_non_job_declaration_protocol Int…
df46a08 
…egration Test
@plebhash
add jds_reject_setup_connection_without_declare_tx_data_flag Integrat…
953fb63 
…ion Test
@plebhash
add jds_reject_declare_mining_job_with_invalid_mining_job_token Integ…
b3c10fa 
…ration Test
@plebhash
add pool_rejects_reused_set_custom_mining_job_token Integration Test
2e91c70
@plebhash
add pool_without_jds_rejects_set_custom_mining_job Integration Test
909e4e4
@lucasbalieiro @plebhash
adapt the docker setup to run the pool with embeded JDS
9664800
@plebhash
plebhash force-pushed the 2026-02-03-jd-server-sv2 branch from f55eb8a to 9664800 Compare March 18, 2026 23:50
bit-aloo
bit-aloo approved these changes Mar 19, 2026
View reviewed changes

@bit-aloo bit-aloo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK

@Sjors

Sjors commented Mar 19, 2026

Copy link
Copy Markdown
Contributor

If DeclareMiningJob had a reference to current chain tip header [...], we could get more refined error_codes

You could still do that, albeit less elegant: if the node rejects the block, check the tip again.

@plebhash
plebhash merged commit 883e0e1 into stratum-mining:main Mar 19, 2026
12 checks passed
@plebhash
plebhash deleted the 2026-02-03-jd-server-sv2 branch March 19, 2026 12:25
This was referenced Mar 20, 2026
Closed
Merged
Closed
This was referenced Mar 23, 2026
Closed
Closed
Closed
Closed
@plebhash plebhash mentioned this pull request Mar 24, 2026
Merged
@gimballock gimballock mentioned this pull request Apr 5, 2026
Merged
@plebhash plebhash mentioned this pull request Apr 8, 2026
Open
@GitGab19 GitGab19 mentioned this pull request May 11, 2026
Closed
@plebhash plebhash mentioned this pull request May 25, 2026
Merged
@lucasbalieiro lucasbalieiro mentioned this pull request Jul 3, 2026
Merged
@lucasbalieiro lucasbalieiro mentioned this pull request Oct 7, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@GitGab19 GitGab19 GitGab19 left review comments
@bit-aloo bit-aloo bit-aloo approved these changes
+1 more reviewer
@lucasbalieiro lucasbalieiro lucasbalieiro left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Refactor: JDS

5 participants

@plebhash @GitGab19 @Sjors @lucasbalieiro @bit-aloo