Skip to content

feat(cluster)!: add per-topic replication factor - #4692

Merged
spetz merged 10 commits into
masterfrom
replication_factor
Nov 1, 2026
Merged

spetz merged 10 commits into
masterfrom
replication_factor

Conversation

@numinnex

Copy link
Copy Markdown
Contributor

Closes #4663

@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Oct 30, 2026
@numinnex
numinnex marked this pull request as draft October 30, 2026 20:04
@github-actions github-actions Bot removed the S-waiting-on-review PR is waiting on a reviewer label Oct 30, 2026
Comment thread core/server/src/partition_helpers.rs Resolved
Comment thread core/server/README.md Resolved
Comment thread core/metadata/src/impls/metadata.rs Resolved
Comment thread core/simulator/src/lib.rs Resolved
Comment thread core/server/README.md Resolved
Comment thread core/metadata/src/stm/stream.rs Resolved
@codecov

codecov Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.61358% with 777 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.74%. Comparing base (f9ba48a) to head (e460ea2).

Files with missing lines Patch % Lines
core/server/src/consumer_group/liveness.rs 87.01% 118 Missing and 17 partials ⚠️
core/simulator/src/lib.rs 79.42% 90 Missing and 3 partials ⚠️
core/consensus/src/impls.rs 73.70% 59 Missing and 2 partials ⚠️
core/metadata/src/stm/partition_directory.rs 82.90% 58 Missing and 2 partials ⚠️
core/metadata/src/stm/lifecycle.rs 70.87% 50 Missing and 3 partials ⚠️
core/server/src/partition_reconciler.rs 82.85% 47 Missing and 1 partial ⚠️
core/metadata/src/stm/stream.rs 94.45% 29 Missing and 12 partials ⚠️
core/server/src/partition_helpers.rs 87.62% 21 Missing and 3 partials ⚠️
core/configs/src/server_config/cluster.rs 82.53% 21 Missing and 1 partial ⚠️
core/metadata/src/impls/metadata.rs 91.13% 14 Missing and 7 partials ⚠️
... and 37 more

❗ There is a different number of reports uploaded between BASE (f9ba48a) and HEAD (e460ea2). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (f9ba48a) HEAD (e460ea2)
rust 4 3
Additional details and impacted files
@@              Coverage Diff              @@
##             master    #4692       +/-   ##
=============================================
- Coverage     90.91%   80.74%   -10.18%     
- Complexity     3857     3860        +3     
=============================================
  Files          1551     1555        +4     
  Lines        395231   400900     +5669     
  Branches     329371   334848     +5477     
=============================================
- Hits         359336   323701    -35635     
- Misses        28672    69907    +41235     
- Partials       7223     7292       +69     
Components Coverage Δ
Rust Core 79.74% <87.47%> (-12.43%) ⬇️
Java SDK 79.37% <92.00%> (+0.01%) ⬆️
C# SDK 84.26% <92.50%> (-0.16%) ⬇️
Python SDK 86.50% <ø> (-0.13%) ⬇️
PHP SDK 92.33% <ø> (-0.30%) ⬇️
Node SDK 96.58% <100.00%> (+<0.01%) ⬆️
Go SDK 71.82% <96.29%> (+0.22%) ⬆️
C++ SDK 86.84% <90.47%> (+<0.01%) ⬆️
Files with missing lines Coverage Δ
...binary_protocol/src/consensus/primary_rebalance.rs 68.01% <100.00%> (-31.99%) ⬇️
...s/partitions/create_partitions_with_assignments.rs 92.05% <100.00%> (-1.75%) ⬇️
core/binary_protocol/src/version.rs 80.41% <100.00%> (-19.59%) ⬇️
core/common/src/consumer_group_client_state.rs 97.95% <100.00%> (-1.46%) ⬇️
core/common/src/types/options/codec.rs 76.92% <100.00%> (+0.66%) ⬆️
core/common/src/types/options/key.rs 92.30% <ø> (ø)
core/common/src/types/options/mod.rs 83.07% <100.00%> (-12.00%) ⬇️
core/configs/src/server_config/defaults.rs 100.00% <100.00%> (ø)
core/configs/src/server_config/validators.rs 76.81% <100.00%> (-17.23%) ⬇️
core/consensus/src/client_table.rs 77.71% <100.00%> (-14.04%) ⬇️
... and 101 more

... and 472 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@spetz spetz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outside this diff and also on master: a replaced member of a group with three or more members rejoins with an empty log as Init, because its initialization record went with the disk and build_partition_fresh decides restarted = replica_count > 1 && !first_initialization (core/server/src/partition_helpers.rs:998, the same line at the merge base). It starts Normal and votes in view changes, since only Status::Recovering is barred (core/consensus/src/impls.rs:3906, core/consensus/src/impls.rs:4113). With three members both quorums are 2, so once the primary fails, the wiped member and a member that lagged can complete a view change without an op that the primary and the wiped member committed, and the new view drops it. The comment above core/server/src/partition_helpers.rs:269 already says that an Init join is safe only for a replica that never served the group.

Comment thread core/server/src/partition_helpers.rs Resolved
Comment thread core/metadata/src/impls/placement.rs Resolved
Comment thread core/server/README.md Resolved
Comment thread core/metadata/src/impls/metadata.rs Resolved
Comment thread core/shard/src/lib.rs Resolved
Comment thread foreign/python/src/resource_options.rs Resolved
Comment thread core/partitions/src/iggy_partition.rs Resolved
Comment thread core/server/src/partition_helpers.rs Resolved
Comment thread core/server/README.md Resolved
Comment thread core/server/src/partition_helpers.rs Resolved
Comment thread core/shard/src/primary_rebalance.rs Resolved
Comment thread core/metadata/src/stm/stream.rs Resolved
Comment thread core/shard/src/primary_rebalance.rs Resolved
Comment thread core/server/src/namespace.rs Resolved
Comment thread core/metadata/src/impls/placement.rs Resolved
Comment thread foreign/swift/Sources/Iggy/Iggy.swift Resolved
Comment thread foreign/php/src/client.rs Resolved
@numinnex
numinnex marked this pull request as ready for review October 31, 2026 19:18
@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Oct 31, 2026
Comment thread core/partitions/src/state_transfer.rs Resolved
Comment thread core/server/src/partition_reconciler.rs Resolved
Comment thread core/metadata/src/stm/stream.rs Resolved
Comment thread core/shard/src/primary_rebalance.rs Resolved
Comment thread core/metadata/src/impls/placement.rs Resolved
Comment thread core/server/README.md Resolved
@github-actions github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Oct 31, 2026
@spetz
spetz merged commit fb7f727 into master Nov 1, 2026
117 checks passed
@spetz
spetz deleted the replication_factor branch November 1, 2026 06:29
@github-actions github-actions Bot removed the S-waiting-on-author PR is waiting on author response label Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support configurable per-topic replication factors

3 participants