Repository navigation
asynchronous distributed commit - #1963
aakash10292 wants to merge 18 commits into
Conversation
|
@roborivers /runtests please. |
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Code review suggestions:
@@ db/db_tunables.h @@
+ Please update the tunable documentation (docs/pages/config/config_files.md) accordingly.
@@ db/seqnum_wait.c @@
+ Please remove debugging statements, and replace fprintf/printf with logmsg.
@@ db/seqnum_wait.c @@
+ Please replace any pthread_xxx() function in your diff with corresponding Pthread_xxx() wrapper.
|
Please add a description to what this does |
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Coding style check: Error. Use clang-format or view diff ⚠.
Smoke testing: Success ✓.
Cbuild submission: Success ✓.
Regression testing: 3/325 tests failed ⚠.
The first 10 failing tests are:
tunables
selectv_rcode_serialretry_generated
selectv_rcode_serialize_reads_like_writes_generated
|
This PR introduces a new feature that enables asynchronous wait for distributed commit. |
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Sysbench Transactions Per Second (commit: 0725cb39):
@@ sysbench/bulk_insert @@
-6.98%
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Code review suggestions:
@@ db/db_tunables.h @@
+ Please update the tunable documentation (docs/pages/config/config_files.md) accordingly.
@@ db/seqnum_wait.c @@
+ Please remove debugging statements, and replace fprintf/printf with logmsg.
@@ db/seqnum_wait.c @@
+ Please replace any pthread_xxx() function in your diff with corresponding Pthread_xxx() wrapper.
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Coding style check: Error. Use clang-format or view diff ⚠.
Smoke testing: Error ⚠ .
Cbuild submission: Success ✓.
Regression testing: 4/325 tests failed ⚠.
The first 10 failing tests are:
restart_socksql
replay_eventlog
tunables
incrementalcopy
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Sysbench Transactions Per Second (commit: 47196279):
@@ sysbench/select_random_ranges @@
+4.82%
@@ sysbench/oltp_read_only @@
-4.72%
There was a problem hiding this comment.
Why do you split the static inline version into a .h and a .c?
There was a problem hiding this comment.
This function should still be static.
There was a problem hiding this comment.
These should be moved to bdb_api.h.
4719627 to
4c91833
Compare
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Code review suggestions:
@@ db/db_tunables.h @@
+ Please update the tunable documentation (docs/pages/config/config_files.md) accordingly.
@@ db/seqnum_wait.c @@
+ Please remove debugging statements, and replace fprintf/printf with logmsg.
@@ db/seqnum_wait.c @@
+ Please replace any pthread_xxx() function in your diff with corresponding Pthread_xxx() wrapper.
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Coding style check: Error. Use clang-format or view diff ⚠.
Smoke testing: Success ✓.
Cbuild submission: Error ⚠ .
Regression testing: 0/0 tests failed ⚠.
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Code review suggestions:
@@ db/db_tunables.h @@
+ Please update the tunable documentation (docs/pages/config/config_files.md) accordingly.
@@ db/seqnum_wait.c @@
+ Please remove debugging statements, and replace fprintf/printf with logmsg.
@@ db/seqnum_wait.c @@
+ Please replace any pthread_xxx() function in your diff with corresponding Pthread_xxx() wrapper.
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Coding style check: Error. Use clang-format or view diff ⚠.
Smoke testing: Success ✓.
Cbuild submission: Error ⚠ .
Regression testing: 0/0 tests failed ⚠.
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Sysbench Transactions Per Second (commit: 4c91833d):
@@ sysbench/oltp_read_only @@
-6.42%
4c91833 to
5c90d40
Compare
There was a problem hiding this comment.
Yes, rectified it
405beae to
c709139
Compare
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Code review suggestions:
@@ db/db_tunables.h @@
+ Please update the tunable documentation (docs/pages/config/config_files.md) accordingly.
@@ db/seqnum_wait.c @@
+ Please remove debugging statements, and replace fprintf/printf with logmsg.
@@ db/seqnum_wait.c @@
+ Please replace any pthread_xxx() function in your diff with corresponding Pthread_xxx() wrapper.
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Coding style check: Error. Use clang-format or view diff ⚠.
Smoke testing: Success ✓.
Cbuild submission: Success ✓.
Regression testing: 4/325 tests failed ⚠.
The first 10 failing tests are:
dohsql_leaks
selectv_rcode_force_commit_generated
tunables
incrementalcopy
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Code review suggestions:
@@ db/db_tunables.h @@
+ Please update the tunable documentation (docs/pages/config/config_files.md) accordingly.
@@ db/seqnum_wait.c @@
+ Please remove debugging statements, and replace fprintf/printf with logmsg.
@@ db/seqnum_wait.c @@
+ Please replace any pthread_xxx() function in your diff with corresponding Pthread_xxx() wrapper.
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Coding style check: Error. Use clang-format or view diff ⚠.
Smoke testing: Success ✓.
Cbuild submission: Success ✓.
Regression testing: 3/325 tests failed ⚠.
The first 10 failing tests are:
selectv_rcode
tunables
incrementalcopy
roborivers
left a comment
roborivers
left a comment
There was a problem hiding this comment.
Sysbench Transactions Per Second (commit: 815b6f73):
@@ sysbench/select_random_ranges @@
+11.52%
@@ sysbench/oltp_read_only @@
+9.35%
815b6f7 to
3ecc8ce
Compare
3ecc8ce to
20daf11
Compare
57479a5 to
0b7492c
Compare
36c68f9 to
eea2abd
Compare
d1c238a to
15d00e3
Compare
Signed-off-by: aakash10292 <aakash10292@gmail.com>
Signed-off-by: aakash10292 <aakash10292@gmail.com>
Signed-off-by: aakash10292 <aakash10292@gmail.com>
Signed-off-by: aakash10292 <aakash10292@gmail.com>
This reverts commit 0750ba34f35eacca52f6e596ffd70a30d71f4a4d. Signed-off-by: aakash10292 <aakash10292@gmail.com>
Signed-off-by: aakash10292 <aakash10292@gmail.com>
Signed-off-by: aakash10292 <aakash10292@gmail.com>
Signed-off-by: aakash10292 <aakash10292@gmail.com>
Signed-off-by: aakash10292 <aakash10292@gmail.com>
… INTERNAL_FORWARD Signed-off-by: aakash10292 <aakash10292@gmail.com>
Signed-off-by: aakash10292 <aakash10292@gmail.com>
This reverts commit ebf113b10476c8001bbaea885aaefe5ca5f18490. Signed-off-by: aakash10292 <aakash10292@gmail.com>
Signed-off-by: aakash10292 <aakash10292@gmail.com>
Signed-off-by: aakash10292 <aakash10292@gmail.com>
Signed-off-by: aakash10292 <aakash10292@gmail.com>
Signed-off-by: aakash10292 <aakash10292@gmail.com>
15d00e3 to
737df5b
Compare
Port of bloomberg#1963 onto current main. Block processors commit locally and then block in trans_wait_for_seqnum_int until every replicant acks, so with the default 8 writer threads only 8 commits can be in flight regardless of client concurrency. This hands the wait to a dedicated seqnum-wait thread and returns the block processor to the queue immediately; the ack is still performed, so replicants stay bounded. Off by default (async_dist_commit). The thread always starts but stays idle until work is queued, so the tunable can be flipped at runtime. Rather than carry bloomberg#1963's copy of bdb_wait_for_seqnum_from_all_int, which predated durable-majority accounting, gbl_2pc, non_durable_retry and set_durable_lsn, the shared logic is factored out of main into bdb_wait_for_seqnum_finish and bdb_wait_for_seqnum_mark_incoherent. Signed-off-by: Emelia Lei <wlei29@bloomberg.net>
Port of bloomberg#1963 onto current main. Block processors commit locally and then block in trans_wait_for_seqnum_int until every replicant acks, so with the default 8 writer threads only 8 commits can be in flight regardless of client concurrency. This hands the wait to a dedicated seqnum-wait thread and returns the block processor to the queue immediately; the ack is still performed, so replicants stay bounded. Off by default (async_dist_commit). The thread always starts but stays idle until work is queued, so the tunable can be flipped at runtime. Rather than carry bloomberg#1963's copy of bdb_wait_for_seqnum_from_all_int, which predated durable-majority accounting, gbl_2pc, non_durable_retry and set_durable_lsn, the shared logic is factored out of main into bdb_wait_for_seqnum_finish and bdb_wait_for_seqnum_mark_incoherent. Signed-off-by: Emelia Lei <wlei29@bloomberg.net>
This PR introduces a new feature that enables asynchronous wait for distributed commit.
Currently, Block processor threads on master wait sequentially for replicants to catch upto a particular LSN. Now, instead of the Block processor threads needlessly spending time on busy waits, the "waiting for replicants" work is offloaded to a separate thread that waits on distributed commit and signals clients with the respective results.