Skip to content

Thread the execution context through the interpreter - #452

Draft
tobert wants to merge 6 commits into
fix/live-job-streamsfrom
refactor/thread-exec-context
Draft

tobert wants to merge 6 commits into
fix/live-job-streamsfrom
refactor/thread-exec-context

Conversation

@tobert

@tobert tobert commented Sep 13, 2026

Copy link
Copy Markdown
Owner

The interpreter functions between the pipeline runner and the dispatch layer read per-invocation state from the shared Kernel::exec_ctx slot, which dispatch_command and dispatch_statement copy into and back out of (GH #369). Two bugs came from that shape, each pinned by a test that failed before its fix:

f() { bash script; }; timeout 1 f        # ran past the deadline
echo x | alias greet='echo hi'           # greet defined afterward
echo x | kaish-output-limit off          # limit still on afterward

This branch threads an ExecContext parameter through the interpreter in one mechanical commit, then moves one field family at a time from the slot to that context. A pipeline's last stage now returns every session change, the same as a statement run on its own. The cancel token and watchdog come from the context a command is called with, so timeout reaches an external command in a function body, in a piped function body, and inside $(...) in that body's arguments. The argument binder carries the caller's context through a KernelArgSource. Embedder tools still run on the kernel's context slot, so the arm that calls them copies the command's cancel token and watchdog onto it first; embedder_tool_sees_the_call_timeout pins that a request timeout stops such a tool.

Still to come on this branch: $(...) in a redirect target, heredoc, or here-string gets the caller's cancel token (it still builds a root context in CommandDispatcher::eval_expr); stdin and pipe ends move to the context, which also fixes echo "got [$(cat f)]" | cat printing only the file; then pipeline position and background flags, session fields, and deleting the dispatch layer's copy-in and copy-back. Draft until those land.

Stacked on #451.

🤖 Generated with Claude Code

tobert and others added 5 commits September 13, 2026 09:20
`timeout` installs a child cancel token on its context and dispatches
the command it wraps. When that command is a function, the body's
pipeline builds a fresh context from the kernel's own cancel token
(`execute_pipeline` reads `self.cancel_token`), not the token `timeout`
installed, so the timer never reaches an external command the body
runs. `f() { bash script; }; timeout 1 f` ran past 10s, while
`timeout 1 bash script` exits 124.

A pipeline's last stage kept its scope, cwd, and alias changes but lost
its ignore config and output limit. `echo x | alias greet='echo hi'`
left `greet` defined; `echo x | kaish-output-limit off` left the limit
on, and `echo x | kaish-ignore clear` left the ignore list in place. The
rule, decided for this change: a last stage returns every session change,
the same as a statement run on its own. Earlier stages stay isolated.

Both tests fail until the execution context is threaded through the
interpreter (GH #369), which is the rest of this branch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The pipeline join copied the last stage's scope, cwd, prev_cwd, and
aliases back to the statement, but not its ignore config or output
limit. `echo x | kaish-output-limit off` left the limit on while
`echo x | alias greet='echo hi'` left `greet` defined, and a statement
run on its own returns all six. The join now returns the same set.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The interpreter functions between the pipeline runner and the dispatch
layer read per-invocation state from the shared `exec_ctx` slot, which
`dispatch_command` and `dispatch_statement` copy into and out of. This
commit only adds the parameter those functions will read instead and
forwards it: the dispatch layer passes its ctx, and each run builds a
root context from a slot snapshot. No read or write of the slot changes,
so behavior is identical; later commits move one field family at a time
from the slot to the ctx.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
`timeout` installs a child cancel token on its context and dispatches
the command it wraps. `dispatch_command` copied that token into the
shared slot, but a function body's pipeline built each stage's context
from the kernel's own token, so `f() { bash script; }; timeout 1 f` ran
past the deadline, and so did the same body piped into `cat`.

Pipelines, builtins, external commands, and `for`/`while` checkpoints
now read the cancel token and watchdog from the context they are called
with. `dispatch_command` and `dispatch_statement` no longer copy either
into the slot. The argv entry point builds its root context once the
watchdog is installed. `$(...)` in a command's arguments still runs
under a root context of its own; the argument binder carries the
caller's context in a later change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Arguments are bound through `ArgValueSource`, whose kernel
implementation evaluated `$(...)` under a fresh root context cloned from
the kernel's own cancel token. `f() { echo $(bash script); }; timeout 1 f`
ran past the deadline because the substitution's external never saw
`timeout`'s child token.

`build_args_async` and `build_args_flat` now take the caller's context,
and the kernel's argument source carries it into each evaluation, so a
substitution in an argument runs with its command's cancel token. A
substitution will also take its stage's stdin from that context once
the slot stops carrying stdin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tobert
tobert added this pull request to stack #453 September 13, 2026 14:33
@tobert tobert changed the title refactor/thread exec context Thread the execution context through the interpreter Sep 13, 2026
@tobert

tobert commented Sep 13, 2026

Copy link
Copy Markdown
Owner Author

kaibo review, cast deepseek (explorer and synth deepseek-flash). It read the whole files on commit a8939b7, without a diff. Findings, and what happened to each:

Defect 2 (regression from 8873dfc): the embedder-tool arm of execute_command_depth hands backend.call_tool the shared slot, and C4 removed the only thing that refreshed the slot's cancel (dispatch_command's copy). At top level the slot holds a constructor-default token that nothing ever cancels, so an embedder tool that waits on cancellation ignores Kernel::cancel(), a request timeout, and timeout. The test-only BackendDispatcher passes the threaded ctx, which is why no test saw it. Fixed in the follow-up commit with a test that fails first.

Defect 1 (existing, not a regression): CommandDispatcher::eval_expr, which evaluates $(...) in redirect targets, heredoc bodies, and here-strings, builds a root context from the kernel's token. So timeout around a function body does not reach an external command inside > "$(slow)". Before this branch, execute_pipeline also read the kernel token on that path. Recorded as the next item on this branch, with its own test.

No defect found in:

  • the watchdog chain for patient holds;
  • KernelArgSource's stdin consumption and locking;
  • the last-stage join leaking session changes;
  • whether the four new tests can fail for the reason they name.

Existing, noted for later:

  • ExecuteOptions::interrupt cannot stop a loop running on a pipeline-stage or scatter fork (fork_inner clears interrupt).
  • A stage before scatter keeps a kaish-ignore change that an ordinary non-last stage drops.

🤖

The embedder-tool arm of `execute_command_depth` passes the kernel's
context slot to `backend.call_tool`. The previous change on this branch
stopped `dispatch_command` copying the cancel token into that slot, and
nothing else refreshes it, so at top level an embedder tool saw a token
that never fires: a request timeout, `Kernel::cancel()`, and `timeout`
all missed a tool waiting on cancellation. The test-only dispatcher
passes the threaded context, which is why no test noticed.

The arm now copies the calling command's cancel token and watchdog onto
the slot before the tool runs. `MockBackend` gains a mode that waits on
cancellation, and `embedder_tool_sees_the_call_timeout` pins the timeout.

Co-Authored-By: DeepSeek V4 Flash <noreply@deepseek.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant