CurrentSession (Mode B) reminders silently burn the 1h execution budget when the target session is mid-turn

[Aaronontheweb]

Summary

CurrentSession-delivered reminders ("Mode B") can silently burn the full 1-hour execution budget when the target session is mid-turn. The reminder message is accepted by the channel gateway, buffered behind an in-flight LLM call, then absorbed into the running human turn as plain text — it never runs as its own turn, so the delivery-confirmation observer never fires, and the execution actor sits until its absolute 1h timeout. Retries repeat the same behavior.

A second, related defect: the error Mode B unsupported origin channel type: Slack is misleading — it is thrown when the origin channel's gateway actor is not yet registered (typically right after a daemon restart), not when the channel type is unsupported. This burns a failure attempt (and increments consecutiveFailures, auto-disable at 5) for a purely transient startup race.

Observed behavior

• A session scheduled a one-shot CurrentSession self-follow-up ("reply to this session"), firing ~4 minutes later.
• At fire time, the target session was mid-turn: 40+ LLM iterations with subagent calls and approval-gated tool calls in flight.
• Session log showed Buffering user message (LLM call in progress) at the reminder fire timestamp, followed by turn_mid_loop_buffer_drain count=1 iteration=31 on the human turn's ID — the reminder was drained into the active turn as an ordinary user message.
• No TurnId for the reminder ever appeared. The delivery observer (keyed reminderId:fireTimeMs) never matched, so no ReminderDeliveryResult was produced — the observer is orphaned.
• Both 1h timers (ExecutionAttemptTimeout and DeliveryObservedTimeout) fired at the same instant; the recorded error was Reminder execution exceeded 01:00:00. (durationMs ≈ 3,600,000).
• The retry 1 hour later (attempt 2/5) hit the identical wall: same busy session, same buffer/drain, same 1h burn.
• After the human turn completed, a replacement reminder fired and completed successfully in seconds — confirming the failure is purely "the session never stopped being busy", not scheduler or delivery config.

Also observed on the startup-race variant:

• A reminder fired ~1s after daemon start, failed in 8ms with Mode B unsupported origin channel type: Slack; the Slack gateway registered ~0.5s later; the next delivery attempt 2 minutes later succeeded.
• Same signature seen across two separate daemon restarts, including on a recurring cron reminder — each restart that coincides with a fire burns an attempt. Recurring reminders are one bad-luck restart away from hitting the 5-failure auto-disable.

Root cause

Mode B (CurrentSession) delivery hands the reminder to the target session's shared serial input queue and acks before the turn runs:

• ReminderExecutionActor.InitializeCurrentSessionAsync dispatches a trusted-session-turn message to the channel gateway (src/Netclaw.Actors/Reminders/ReminderExecutionActor.cs:146).
• The binding actor writes into the same input queue as human messages and registers the delivery observer keyed reminderId:fireTimeMs (src/Netclaw.Channels.Slack/SlackThreadBindingActor.cs:250-322).
• If a human turn's LLM call is in flight, the reminder message is buffered — and the mid-loop drain in LlmSessionActor adds buffered content to the current turn (LlmSessionActor.cs:996-1006). The reminder never becomes its own turn, so TurnCompleted.SourceReminderId is null and the observer key never matches (LlmSessionActor.cs:2108).
• The execution actor's only backstops are both 1h: ExecutionAttemptTimeout (absolute, armed in PreStart) and DeliveryObservedTimeout — identical wall-clock, so the failure is uniformly mislabeled Reminder execution exceeded 01:00:00. (ReminderExecutionActor.cs:60,140-143,343-351).

The startup race:

• ResolveGatewayFor returns null when the gateway actor is not yet registered; the code reports it as Mode B unsupported origin channel type: {type} (ReminderExecutionActor.cs:272-277,401-413).
• Gateway registration happens only after the channel's auth.Test succeeds at connect (src/Netclaw.Channels.Slack/SlackChannel.cs:227-257) — a few hundred ms after the execution actor starts in the observed cases.
• This transient condition increments ConsecutiveFailures and counts toward the 5-failure auto-disable (ReminderManagerActor.cs:33,873-956).

Impact

• Every self-scheduled CurrentSession follow-up that lands while its session is busy burns a full hour of execution-budget and then shows a misleading timeout error; with retries, a single stuck session can burn hours of scheduler capacity.
• Delivery observers are orphaned on each occurrence (a leak).
• The startup race can auto-disable healthy recurring reminders and produces a misleading "unsupported channel" error that sends operators chasing config that is fine.

Suggested fixes (in order of leverage)

1. Defer, don't absorb. In LlmSessionActor's mid-loop drain, a buffered message carrying a ReminderId should be deferred to the next turn (its own turn, own SourceReminderId) instead of AddUserMessage'd into the current one. Smallest change that makes the existing delivery contract true.
2. Fail fast when busy. Have the binding actor return a distinct ack (CommandNack/CommandDeferred, reason "session busy") when the message would be buffered, so the execution actor fails in seconds and the scheduler redelivers — instead of a silent 1h burn.
3. Mode B stall backstop. Mode A has ExecutionStallTimeout (20 min, output-reset); Mode B's only backstop equals the absolute limit. A shorter, progress-aware delivery-observation timeout would surface "delivery not observed" and release the duplicate-execution guard sooner.
4. Approval exemption for trusted reminder turns. The trusted-turn source already carries Principal=VerifiedAutomation + Boundary=trusted-instance; route reminder-origin tool calls around the interactive approval gate so a RequiresApproval stall cannot strand the turn.
5. Leak fix. When a drain absorbs a message whose ReminderId is registered in the delivery observers, remove the entry and signal Delivered:false immediately — fail in milliseconds, not an hour.
6. Startup race. Bounded in-actor wait (1-2s poll, ~30-60s cap) for gateway registration in InitializeCurrentSessionAsync, and correct the error message to e.g. Mode B origin channel gateway not registered for channel type: {type}.

Verification

• Reproduce: create a CurrentSession one-shot reminder into a session, start a long human turn, fire the reminder; assert it either completes as its own turn or fails fast — never burns 1h.
• Unit: fake binding actor that acks DeliverTrustedSessionTurn but never sends ReminderDeliveryResult; assert settlement at DeliveryObservedTimeout with a "delivery not observed" error.
• Unit: mid-loop arrival of a ReminderId-carrying message; assert it is deferred to its own turn and TurnCompleted.SourceReminderId equals the reminder key.
• Stress: N concurrent fires into a busy session; assert no execution exceeds stall-class bounds and the observer registry returns to empty.