Skip to content

[v25.3.x] kafka/server: cap fetch memory allocation at max message size limit - #30314

Merged
ballard26 merged 1 commit into
redpanda-data:v25.3.xfrom
vbotbuildovich:backport-pr-30023-v25.3.x-890
Aug 12, 2026
Merged

[v25.3.x] kafka/server: cap fetch memory allocation at max message size limit#30314
ballard26 merged 1 commit into
redpanda-data:v25.3.xfrom
vbotbuildovich:backport-pr-30023-v25.3.x-890

Conversation

@vbotbuildovich

@vbotbuildovich vbotbuildovich commented Apr 28, 2026

Copy link
Copy Markdown
Collaborator

Backport of PR #30023

@vbotbuildovich vbotbuildovich added this to the v25.3.x-next milestone Apr 28, 2026
@vbotbuildovich vbotbuildovich added the kind/backport PRs targeting a stable branch label Apr 28, 2026
@vbotbuildovich

vbotbuildovich commented Apr 28, 2026

Copy link
Copy Markdown
Collaborator Author

Retry command for Build#83728

please wait until all jobs are finished before running the slash command

/ci-repeat 1
skip-redpanda-build
skip-units
skip-rebase
tests/rptest/tests/data_transforms_test.py::DataTransformsLoggingTest.test_log_topic_integrity

@vbotbuildovich

vbotbuildovich commented Apr 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

CI test results

test results on build#83728
test_status test_class test_method test_arguments test_kind job_url passed reason test_history
FLAKY(FAIL) DataTransformsLoggingTest test_log_topic_integrity null integration https://buildkite.com/redpanda/redpanda/builds/83728#019dd217-ac36-4c85-a98b-030ee469a8d2 8/11 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=DataTransformsLoggingTest&test_method=test_log_topic_integrity
test results on build#88644
test_status test_class test_method test_arguments test_kind job_url passed reason test_history
FLAKY(PASS) AuditLogTestSchemaRegistryACLs test_sr_audit_authz {"audit_transport_mode": "kclient", "endpoint_name": "GET_SUBJECT_VERSIONS"} integration https://buildkite.com/redpanda/redpanda/builds/88644#019ff6c6-11de-49c8-9168-4aa6c255c0a5 10/11 Test PASSES after retries.No significant increase in flaky rate(baseline=0.0000, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=AuditLogTestSchemaRegistryACLs&test_method=test_sr_audit_authz

ballard26
ballard26 approved these changes Aug 12, 2026
View reviewed changes

@ballard26 ballard26 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ballard26
ballard26 enabled auto-merge August 12, 2026 15:51
@ballard26
kafka/server: cap fetch memory allocation at max message size limit
0a50158 
Use kafka_max_message_size_upper_limit_bytes to clamp max_bytes and
max_batch_size in fetch memory allocation instead of the full shard
fetch memory budget. This prevents a single partition with an inflated
max.message.bytes (e.g. UINT32_MAX) from monopolizing the entire
shard's fetch memory, which can starve other partitions and cause
fetch timeouts.

(cherry picked from commit 2b2c4b3)
@ballard26
ballard26 force-pushed the backport-pr-30023-v25.3.x-890 branch from 1801678 to 0a50158 Compare August 12, 2026 16:00
@ballard26
ballard26 merged commit 9bde24f into redpanda-data:v25.3.x Aug 12, 2026
16 checks passed
@tyson-redpanda tyson-redpanda modified the milestones: v25.3.x-next, v25.3.17 Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ballard26 ballard26 ballard26 approved these changes

Assignees

No one assigned

Labels

area/build area/redpanda kind/backport PRs targeting a stable branch

Projects

None yet

Milestone

v25.3.17

Development

Successfully merging this pull request may close these issues.

3 participants

@vbotbuildovich @ballard26 @tyson-redpanda