Repository navigation
feat(voice-session-context): recent_context tool + writer convention - #665
Merged
Merged
Conversation
Chi 2026-05-13: "the voice agent lost context. Can it auto reload
context from recent tasks and results? ... do this only in active mode.
... hybrid."
Voice-agent's Gemini context window rolls off after ~10 minutes of
turns. After a long debug session this morning Chi had to remind voice
~5 times what "the post" or "Mini Draft A" referred to. The fix: a
small JSON state file that core writes to whenever a durable decision
lands, plus a voice-callable tool to read it.
This PR ships the read path (the tool) and the writer convention (a
CLAUDE.md note). It deliberately defers the per-turn auto-injection
into Gemini's system message — that's a deeper voice-agent change
and gets a follow-up PR. The tool is enough for the immediate gap:
voice can call `recent_context()` whenever it senses confusion.
Changes:
- New `recentContextTool` (inline) at src/inline-tools.ts:805. Returns
the parsed JSON content of `state/voice-session-context.json` or a
`{note: 'no context recorded yet'}` placeholder if the file is
missing. Registered in both `inlineTools` and `ownerOnlyTools`
arrays.
- CLAUDE.md task-bridge section gets a "Voice session context"
paragraph: schema (active_drafts / pending_action / last_results),
prune rule (~3 entries each), when to write (durable decisions).
- File at `state/voice-session-context.json` is gitignored via
existing `state/*` rule — runtime data, not source.
Schema (informal, documented in CLAUDE.md):
```json
{
"updated_at": "ISO ts",
"active_drafts": [{"name": "...", "summary": "...", "path": "..."}],
"pending_action": {"kind": "paste|...", "what": "...", "where": "..."} | null,
"last_results": [{"task_id": "...", "subject": "...", "ts": "..."}]
}
```
Active-mode gate (Chi's constraint): only relevant if/when we add
auto-injection. The `recent_context` tool itself is model-triggered,
which is already "active-mode" by definition — Gemini only calls
tools during live sessions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 21, 2026
liususan091219
pushed a commit
to liususan091219/sutando-fork
that referenced
this pull request
May 28, 2026
…onichi#665) Chi 2026-05-13: "the voice agent lost context. Can it auto reload context from recent tasks and results? ... do this only in active mode. ... hybrid." Voice-agent's Gemini context window rolls off after ~10 minutes of turns. After a long debug session this morning Chi had to remind voice ~5 times what "the post" or "Mini Draft A" referred to. The fix: a small JSON state file that core writes to whenever a durable decision lands, plus a voice-callable tool to read it. This PR ships the read path (the tool) and the writer convention (a CLAUDE.md note). It deliberately defers the per-turn auto-injection into Gemini's system message — that's a deeper voice-agent change and gets a follow-up PR. The tool is enough for the immediate gap: voice can call `recent_context()` whenever it senses confusion. Changes: - New `recentContextTool` (inline) at src/inline-tools.ts:805. Returns the parsed JSON content of `state/voice-session-context.json` or a `{note: 'no context recorded yet'}` placeholder if the file is missing. Registered in both `inlineTools` and `ownerOnlyTools` arrays. - CLAUDE.md task-bridge section gets a "Voice session context" paragraph: schema (active_drafts / pending_action / last_results), prune rule (~3 entries each), when to write (durable decisions). - File at `state/voice-session-context.json` is gitignored via existing `state/*` rule — runtime data, not source. Schema (informal, documented in CLAUDE.md): ```json { "updated_at": "ISO ts", "active_drafts": [{"name": "...", "summary": "...", "path": "..."}], "pending_action": {"kind": "paste|...", "what": "...", "where": "..."} | null, "last_results": [{"task_id": "...", "subject": "...", "ts": "..."}] } ``` Active-mode gate (Chi's constraint): only relevant if/when we add auto-injection. The `recent_context` tool itself is model-triggered, which is already "active-mode" by definition — Gemini only calls tools during live sessions. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Chi 2026-05-13: voice-agent's Gemini context window rolls off after ~10 min, voice forgets "the post" / "Mini Draft A" / "the draft we picked" — caught it ~5 times during this morning's debug session. Fix per Chi's "hybrid" pick: a small JSON file core writes when durable decisions land, plus a voice-callable tool to read it.
What this PR ships
Read path (Gemini-callable):
recentContextToolinsrc/inline-tools.ts. Returns parsedstate/voice-session-context.jsonor{note: 'no context recorded yet'}placeholder. Registered in bothinlineToolsandownerOnlyTools.Writer convention (CLAUDE.md):
{updated_at, active_drafts[], pending_action|null, last_results[]}.State file at
state/voice-session-context.jsonis gitignored via existingstate/*.What this PR defers
The per-turn auto-injection into Gemini's system message (proposed flavor #1). That's a deeper voice-agent change touching the bodhi
VoiceSessionlifecycle and gets a follow-up PR after we see therecent_contexttool perform in real sessions.For the immediate "voice forgets specifics" gap, the tool is enough: Gemini will call
recent_context()when it sees a deictic pronoun ("the post", "the draft") it can't resolve from its own transcript window.Active-mode gate
Chi asked for refresh-only-when-active. The
recent_contexttool is model-triggered, which by definition only fires during live VoiceSession turns. Zero token cost when voice is idle. If the per-turn injection follow-up lands, the active-mode gate goes there.Test plan
recent_contextand return Mini Draft A details[RecentContext] returned (updated_at=..., N drafts, N results)state/voice-session-context.jsonstays gitignored🤖 Generated with Claude Code