You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Engine: a hung LLM stream never times out — the turn and its busy registration leak forever #1739
The episode (2026-10-09, live-diagnosed on shard 3): a session's turn started normally (streams at 09:46-09:47Z, step 3), then went silent with the engine otherwise healthy — /config 5ms, idle CPU, and crucially no provider socket left: the baseten connection died without an error ever surfacing to the stream await. The turn never unwound: the run-coordinator held the session busy for 8 hours while its work had long finished, the panel rendered eternal "thinking," and only an explicit POST /session/{id}/abort cleared it (cancel + the errored message at 17:32:25Z).
The gap: the transport work bounded the provider response (#1716's 64MB cap) and the question wait is now cancellable (#1730), but the LLM stream await itself has no idle/read timeout: a provider socket that dies without a clean FIN/EOF (tunnel drop, NAT timeout, LB idle-kill) leaves the read parked forever, and with it the turn, its coordinator registration, and the session's UI state.
Fix shape: an idle timeout on the stream read (configurable, generous — long generations legitimately pause between chunks; reasoning deltas arrive continuously, so a 60-120s idle ceiling is safe) that fails the step with a transport error — the existing error path then unwinds the turn, publishes the failure state, and frees the coordinator. This is the last member of the lifecycle family: tools (#1708), questions (#1730), and now the stream itself.
Reproduction sketch: start a turn, kill the provider connection mid-stream without a clean close (e.g. drop the TCP connection via a firewall rule or a proxy that blackholes it), observe the session stay busy indefinitely and the step never error.
Part of: the #775/#1717 lifecycle seams. Operational note: the frontdoor's liveness watchdogs catch wedged engines, not wedged turns — a session-level stuck-busy detector in the frontdoor (busy with zero log activity + zero event flow for N minutes → surface a warning card) would bridge the gap until this lands.
Activity
aarontrowbridge commentedon Oct 25, 2026
Resolved on main: ffdeccd.