Skip to content

RocketMQ: Requeue does nothing, so the pump's delivery budget can never be exhausted #4353

Closed

Description

@iancooper

Summary

RocketMessageConsumer.Requeue is a no-op. A message whose handler keeps deferring is therefore
redelivered indefinitely and is never dead-lettered — the pump's delivery budget (RequeueCount)
cannot run down, and no Reject is ever issued.

Found by the FR-23 conformance behaviour (requeue budget exhausted to DLQ) in #4240 / #4297, which
drives a real Brighter pump. The cell stays Deferred on this.

The code

src/Paramore.Brighter.MessagingGateway.RocketMQ/RocketMessageConsumer.cs:179 — the method resolves
the MessageView from the bag and returns true. That is all of it; the one call that would act on
the broker is commented out:

public bool Requeue(Message message, TimeSpan? delay = null)
{
    if (!message.Header.Bag.TryGetValue("ReceiptHandle", out var handler) || handler is not MessageView view)
    {
        return false;
    }

    // Waiting for next RocketMQ C# version, due an issue on ChangeInvisibleDuration
    // consumer.ChangeInvisibleDuration(view, TimeSpan.Zero);
    return true;
}

So the message simply stays invisible until its lease lapses and the broker re-serves the stored
copy. HandledCount is read from the message's published properties (ReadHandledCount, :422) and
written only on send (RocketMqMessageProducer.cs:157), so every redelivery arrives reading the value
that was originally published. The pump bumps it to 1, HandledCountReached(RequeueCount) is never
true, and nothing reaches the DLQ.

Measured, and it is not a timing problem

RocketMQ redelivers only when its 10 s invisibility lease lapses, so a budget of 3 could plausibly
outrun the conformance suite's 30 s ceiling. It does not:

ceiling result
30 s (both variants) MT_NONE — nothing on the DLQ
150 s (Reactor, ~15 redeliveries against a budget of 3) still MT_NONE

The budget does not run down slowly. It does not run down at all.

Why this is the most complete instance of a family

This is the third transport where FR-23 has found the same root cause — a requeue that does not
persist the delivery count cannot exhaust a Brighter-side budget
:

transport requeue mechanism consequence
AWS SQS (#4341) ChangeMessageVisibility budget inert, but maxReceiveCount redrive dead-letters the message anyway
GCP Pub/Sub ModifyAckDeadline(…, 0) same, with MaxDeliveryAttempts as the redrive
RocketMQ nothing at all no broker-side redrive is wired either, so the message is never dead-lettered by anyone

The transports whose FR-23 cells pass (Redis, Kafka ×3, MSSQL, Postgres, RMQ ×3) all republish on
requeue, carrying the updated header — which is exactly why their budgets run down.

Suggested fix

Either restore ChangeInvisibleDuration once the RocketMQ C# client issue the comment refers to is
resolved and make the redelivered message carry an incremented delivery count, or map RocketMQ's own
delivery-attempt property into HandledCount on receive so the budget runs off the broker's counter.
The second needs no client fix.

Impact

Any RocketMQ consumer whose handler defers loops for ever. Blocks 1 FR-23 conformance cell.

Activity

  1. self-assigned this
    on Sep 12, 2026
  2. iancooper commented on Sep 24, 2026

    @iancooper
    MemberAuthor

    This issue is covered by spec 0037, Delivery count and rejection routing (family head #4341). Requirements are approved: requirements.md.

    RocketMQ is conditional by design (R-14). If a redelivered message presents a strictly increasing broker delivery-attempt value without ChangeInvisibleDuration, RocketMQ ships in this spec and FR-23 moves to Fixed. If it does not, the transport stops at bound but unimplemented. The FR-23 cell is then re-pointed at the specific upstream ChangeInvisibleDuration blocker, and this issue is updated with the measurement that decided it (AC-23 to AC-25). Both outcomes count as done.

  3. iancooper commented on Oct 1, 2026

    @iancooper
    MemberAuthor

    Measurement for spec 0037, task 7.1 (AC-23), 2026-10-01. Broker apache/rocketmq:5.5.0, RocketMQ.Client 5.2.1, requeueCount: 3, Reactor, a 10 s invisibility lease, and no Requeue/Nack/ChangeInvisibleDuration call. The consumer was disposed and recreated in the same group before delivery 3.

    MessageView.DeliveryAttempt on deliveries 1, 2, 3: 1, 2, 3. Run 1 was on a fresh topic (t+0.1 s / 13.3 s / 25.4 s). Run 2 was on a store reset with down -v (t+0.2 s / 12.3 s / 24.3 s).

    The value is strictly increasing and survives a new client, so it is broker-supplied. R-14's condition holds: AC-24 is taken. Brighter will derive HandledCount from the broker's counter on receive, so the pump's budget can run down even though Requeue stays a broker no-op. The FR-23 cell is expected to move to Fixed (#4353) at task 7.13.

    Also measured: SimpleConsumer.ChangeInvisibleDuration(view, TimeSpan.Zero) works on client 5.2.1. The message was redelivered in 2–3 s instead of after the 10 s lease, and DeliveryAttempt still advanced. So the "waiting for next RocketMQ C# version" comment on Requeue is out of date (see also #4263, which uses it in Nack). Enabling it is a separate follow-up.

    Fixture: tests/Paramore.Brighter.RocketMQ.Tests/MessagingGateway/Reactor/RocketMqDeliveryAttemptMeasurementTests.cs (Skip-marked). Ledger: specs/0036-universal-transport-conformance-tests/conformance-status.md, RocketMQ section.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions