Repository navigation
Description
Slice of #1619. It's independent of #1658 / #1659 / #1706 and needs nothing else first.
Problem
When a Dev-Lead stage hits its budget, we can't tell what the session was doing.
_claude_chain_invoke (scripts/engine.sh, ~L1105–1160) runs:
timeout "$timeout_sec" claude --print --model "$model" --output-format json …
--output-format json emits a single envelope at the end of the session. GNU timeout kills the process at the budget, so on exit 124:
- no envelope is ever written, and stdout is empty;
parse_engine_usage(scripts/lib/token-metrics.sh) has nothing to parse, so thetoken-usage-<run_id>artifact has no row for the stage;- the chain hop log (
_claude_log_hop, fix(engine): unblock Opus 5.5 deep tier — valid Sonnet 5 id, fresh CLI cache, honest chain errors (#1957) #1958) records only the failed hop's header, plus at most 1000 bytes of stderr; - the only things kept are the timeout sidecars (
/tmp/dev-lead-timeout-{tier,budget,elapsed}), and Measure and tune the dev-lead action budget: #1609 died 1s over ACTION_TIMEOUT_SEC and lost all its work — near-miss alerting + checkpoint push before tuning the number #1619's correction showselapsedis alwaysbudget + 1.
So we can't answer: how many turns ran, which tool calls took the time, whether it was stuck on one finding or working through eight, how many tokens it burned, or how far along it was. Tuning the budget, splitting issues or slimming prompts (#1619) is guesswork without this.
Recent example
#1953: Dev-Lead run 36856300906, step "Run fix-reviews", ran 11:35:46Z → 12:21:05Z on 2026-10-01, covering cubic's 8 findings. It failed with no record of what happened in those ~46 minutes. It had posted 10 comment dispositions by 11:56Z, so roughly 25 minutes of activity before the kill are unaccounted for. (The orphaned dispositions are #1992.)
Fix
Switch the Claude writer and agentic calls to --output-format stream-json --verbose. That emits one JSON event per line as the session runs: assistant turns, tool calls and results, and a final result event with the same usage / result fields as today's envelope. Tee the stream to a per-stage file, so whatever happened before the kill is on disk when timeout fires.
Acceptance criteria
- Stream to disk. Claude stages that can hit
ACTION_TIMEOUT_SEC/DEEP_TIMEOUT_SECwrite a line-delimited event stream to$RUNNER_TEMP/engine-stream/<stage>-<hop>.jsonlas they run. At minimum that's the writer path; include deep/audit if the change is the same. The file must hold the events up to the moment of the kill, not only on success. - No behaviour change on success.
- The text the caller receives (today's extracted
.result) is identical. parse_engine_usagegets the sameusage/ model data, read from the finalresultevent.- Every existing bats test for the JSON-envelope path still passes, or is updated with an equivalent assertion.
ENGINE_USAGE_JSON=0(text mode) still works.
- The text the caller receives (today's extracted
- A per-stage timeline, even on timeout. Emit one
::notice::[engine-timeline] …line per stage. It carries: turn count, tool-call count by tool name, the slowest N tool calls with their duration, input/output tokens where known, the last assistant action before the kill, andoutcome(success/timeout/ …). Also write it to the step summary. On a timeout, this is the "what was it doing" answer. - Artifact on failure only. Upload the stream files as
engine-stream-<run_id>with short retention (≤ 14 days), only when the stage did not succeed (timeout,engine-error). Nothing extra is uploaded on success. - Redaction (CWE-532, same bar as fix(engine): unblock Opus 5.5 deep tier — valid Sonnet 5 id, fresh CLI cache, honest chain errors (#1957) #1958). The stream contains tool output and can contain secrets or untrusted PR text.
- Pass every line through
redact_secrets(scripts/lib/redact.sh) before it reaches the artifact or the logs. - Cap the size of each tool-result payload (e.g. keep the first and last ~2 KB and record the truncated length).
- If
redact_secretsis unavailable, upload nothing and only emit the timeline counts. - Never echo raw stream content into the Actions log.
- Pass every line through
- Gemini/Copilot: out of scope. Note in the PR whether each CLI has an equivalent streaming mode, so a follow-up can mirror this.
- Tests:
- a stub
claudethat emits N stream events and is then killed bytimeoutleaves a partial.jsonlwith those N events, and the timeline notice reports N turns and outcometimeout; - a successful stub run yields the same caller output and usage row as today;
- a token-shaped string in a tool result is redacted in the stored file. Build the token at runtime, per the fix(engine): unblock Opus 5.5 deep tier — valid Sonnet 5 id, fresh CLI cache, honest chain errors (#1957) #1958 gitleaks note.
- a stub
- Keep it inside the engine invocation and its artifact upload. Don't change budgets (Measure and tune the dev-lead action budget: #1609 died 1s over ACTION_TIMEOUT_SEC and lost all its work — near-miss alerting + checkpoint push before tuning the number #1619: tune only on evidence) or the run loop.
Why this is worth doing before tuning
#1658/#1659 will tell us how long runs take. This slice tells us why: which part of a timed-out pass ate the budget. That decides whether the fix is a bigger budget, smaller issues, a cheaper prompt (e.g. not re-researching already-dispositioned comments, #1992), or a tool that hangs.
Rollout
These are Dev-Lead engine changes. Promote through the normal channel rings; no manual tag moves.
Activity
github-actions commentedon Oct 1, 2026
Thanks for the detailed write-up and for tying it back to the concrete #1953 run, which makes the motivation easy to follow. The plan to stream events to a file that survives the
timeoutkill looks well scoped. One question before implementation: when a stage times out, would you rather the timeline's "slowest tool calls" and "last assistant action" be limited to short summaries (tool name, duration, truncated args), or do you want a bit of redacted tool-result context included too? That choice decides how much payload we keep per event under the redaction and size-cap requirements.