Repository navigation
feat: source navigation echoes and guarded fork-prefix inheritance - #6393
AdityaVG13 wants to merge 34 commits into
Conversation
Sound every file read, grep, file search, and project map with its surroundings — sibling podmates, line-numbered symbols, import roots, resolved file links, and (in the terminal buzz) reverse-link callers — sounded fresh from source with no index, no background work, and no new dependencies. Deterministic (sorted, capped, budgeted) so repeated calls are byte-identical and cache-friendly. Echolocation (new: tools/echolocation.rs, ~2.5k lines incl. tests): - read footer: [Pod] mates + [Sounding] symbols/imports/links/heard-by, 1200B hard budget; miss echo on NotFound (denylist-gated, never fires on refused probes); terminal buzz adds callers after 2 distinct files read in a pod - grep + file_search chart matched dirs (caller's own visibility gates: denylist, excludes, extensions, gitignore); project_map sounds key files; 4 sounding languages (Rust/Python/JS-TS/Go) Token diet (no compression, behavior-preserving): - prefix 14,241 -> 7,118 tokens/turn (-50.0%): skill path cut (load_skill resolves names), MCP prose 1526->548B (scar-tissue phrases kept), agent+workflow deferral, goal tools conditional-eager keyed off the turn's resolved goal (preview/wire-safe) - activation cache: best-match-last push + regex name-rank + 16->24KiB bound (fixes deferral stranding agent behind its own discovery set) - grep empty-context omit, workflow event projection (lossless: duplicates dropped, workspaces hoisted, branch kept) + zero-memo omit - wire-prefix dump moved to PreparedOutboundRequest (exact-final-body) Evidence: targeted echo 46, workflow 131, file 145 green; full tui+workflow suites green except 4 failures reproducing identically on base c75b233 (pre-existing, unrelated) + known timing flakes proven solo-pass; clippy zero on owned files. Research: codewhale-research/ ECHOLOCATION-THESIS.md (exp-10/11/12 + hardening audit).
DEFAULT_ACTIVE_NATIVE_TOOLS is six names now (agent, workflow, get/update_goal deferred); the web contract mirror, tool docs, and mode docs still claimed ten eager + eleven with tool_search. Activation cache bound 16 -> 24 KiB in facts, docs, and web copy. Found by the fork staging PR's Lint & Type Check gate.
Footer squeeze (all lossless): same-pod links/callers render bare, import roots a link already names drop by file stem or package path, framing tightens to [Pod (N)] / [Sound], and sounded paths are /-separated on every OS. The squeeze follows CODEWHALE_ECHO gates so an off:links trial measures links alone. Edit echoes name touched symbols plus impacted callers on write/edit; the diff rides metadata for the TUI while the receipt stays one line. Grep max_results is clamped with honest truncation, skill listings are capped, JS arrows sound as their source token, and ECHOLOCATION.md documents the surfaces. Signed-off-by: AdityaVG13 <adityavgcode@gmail.com>
Relative imports now sound their leaf instead of a bare dot (from . import sib -> sib, ./server -> server), which composes with stem-shadowing when the link resolves and names the module when it does not. Removes the temporary footer-breakdown probe (its numbers landed in the PR; the eprintln tripped the print_stderr deny and flipped Lint red). Signed-off-by: AdityaVG13 <adityavgcode@gmail.com>
Child on the same route/model/tools inherits the parent's cached prompt prefix (system prompt + compacted history) as its frozen LivePrefix base, warming from the warm cache instead of going cold. Resolution is a pure deterministic gate (resolve_fork_inherit) — same route, same model, same or subset tools, child config digest must descend from the parent chain — so narrowed roles and grandchildren correctly fall back cold. No human-language compression anywhere; the child extends the parent's verbatim prefix with its own delegation message (append-only history per docs/CACHE.md), keeping prompt-cache prefix-matching intact.
Replace whole-block tool equality (which never held: parent wire blocks grow by deferred discovery while child grants are fixed and narrower) with longest byte-equal head resolution. The child adopts the parent's exact system bytes and sends the matched head on its first request; the rest of its grant stays deferred and activates on demand, so truncation costs cache bytes, never capability. Full head (inherited_full) prices shared history as hits; partial head (inherited_prefix) still prices system plus matched tools. Both sides now canonicalize through tool_to_api_json (the wire shape), so internal-only flag skew (defer_loading, allowed_callers) can't false-negative the comparison. The hot proof carries forward across append-only requests, since spawns execute before the current request's usage lands. Fresh spawns, grandchildren, resumes, route skew, and cold prefixes stay cold with distinct reasons. Trial instrumentation (CODEWHALE_FORK_TRIAL=1, default off) mirrors the decision, prefix span, history-divergence point, and child first-response cache split to stderr as JSON lines for same-binary A/B measurement; headless exec installs no tracing subscriber, so the tracing trial logs are invisible there. Measured (same-binary A/B, DeepSeek, unique histories, child first-response): true cold 0/9421 hits; primed cold 4736/9389; prefix k=3 5504/9941; full k=6 8576/13251. Inherited children reported correct results in every run. DeepSeek counts hits in 64-token blocks. Follow-ups: fork transcripts freeze at turn 1 (the live tail never rides), framing sits ahead of where the tail would (breaking the head match), grandchildren carry ~40KB all-miss transcripts (single-level v1).
The CODEWHALE_FORK_TRIAL readout intentionally writes JSON lines to stderr (headless exec installs no tracing subscriber, so the tracing trial logs are invisible there). Narrowly allow clippy::print_stderr on the two env-gated diagnostic blocks with justification, following the existing in-repo precedent. Documents the fork-prefix contract, measured benchmarks, trial procedure, and known follow-ups in docs/CACHE.md.
The tool layer hands echolocation canonical file paths while the workspace may be non-canonical (macOS /var to /private/var tempdirs, Windows verbatim paths). Three spots compared them raw and silently missed everywhere but Linux: - workspace_relative now canonicalizes both sides before giving up, restoring caller and link matches. - pod_relative_names compares in workspace-relative space instead of joining the raw workspace, restoring same-pod shortening. - Grep match paths fall back through the same canonical strip instead of rendering absolute. Verified against the three CI failures locally on macOS; Linux suites are unaffected (fast paths unchanged).
Reconcile the complete contributions from #6832 (@aboimpinto), #6867 (@hodeswildsmith455-boop), and #6805 (@LIghtJUNction) with the current Engine, provider identities, reviewed-plugin policy, and task lifecycle. Original contributor histories are recorded by subsequent resolved merges. Fix the already integrated contributor cases: snapshot corruption is an explicit unavailable/error result (#6817), malformed locales cannot select an incidental script (#6860), image metadata uses the exact uploaded bytes and respects available decoders (#6858), automation deletion waits for actual scheduler reconciliation (#6864), and blocking trust/skill reads stay off the async executor (#6869). Preserve #6857's compaction regression. Extend #6872's human-wait lifecycle guard to approval/elevation cards; retire only the matching ended parent request and refresh activity only after a delivered decision. Enforce configured finite approval deadlines in the Engine, including deadline/cancellation races and durable receipts. Serialize Native Windows ACL admission/retirement across Core processes with a logon-scoped kernel mutex, preserving exact SID/object validation. Add an actual child-process lock test; serialize DSH host tests in the existing extension-host lane. Windows execution proof remains hosted CI. Reconcile the vendored computer-use plugin with canonical main a656f67455fc while preserving Core's 0.12.1 embedding contract. Canonical b47/a656 tree passed Ubuntu/macOS/Windows source and package gates, including 28/28 Windows-focused tests and the controlled desktop fixture; this is separate from the new Engine head's CI verdict. Partial adaptation of the discovery-cache priority portion from PR #6393 by @AdityaVG13 (original ac33dd4). Preserve the best match under count and byte limits without importing the unfinished echolocation/fork design; the broader draft remains open. Validation: - npm test: 1286 passed, 0 failed, 7 skipped; web 767/767. - npm run check:web: lint, typecheck and production build passed. - Affected Rust selection: 801/803 initially passed; the two fixture/lifecycle expectation failures were corrected and each passed a focused rerun. - Additional focused Rust: approval 28/28, discovery cache 11/11, OrcaRouter synthetic catalog 3/3, and 33/33 lifecycle/API/routing checks. - Final CI-repair selection: 37/39 initially passed; the BMP feature-proxy and feature-registry summary failures were corrected; both corrected tests passed (2/2, 0 failures). - Qualified Clippy: six packages, all targets/all features, passed with the CI style allowances. Portable no-default-feature check passed; portable policy verifier 6/6 passed; formatting and diff checks passed. - Runtime contract: 55 measured metrics passed after explicit remeasurement; all 21 structural identities were unchanged. Twelve byte/token-estimate budgets account for bounded child-wait disclosure and contributor locale descriptions; no runtime field/prompt was removed to lower the budget. - Persistence budget was not qualified locally: the checker requires a clean tree and this shared checkout retains an unrelated operator file. Clean hosted CI, fresh stamped build and real DeepSeek TUI acceptance remain required before the integration PR can merge to main. Refs #6872, #6843, #6795. Co-authored-by: Paulo Aboim Pinto <paulo.aboim.pinto@gmail.com> Co-authored-by: hodeswildsmith455-boop <hodeswildsmith455-boop@users.noreply.github.com> Co-authored-by: LIghtJUNction <lightjunction.me@gmail.com> Co-authored-by: AdityaVG13 <adityavgcode@gmail.com> Signed-off-by: CodeWhale Bot <bot@codewhale.net>
|
sorry @AdityaVG13 I ended up moving too fast I think around the time this happened; would still love to get this in codewhale! |
There was a problem hiding this comment.
🔵 Needs a closer look
The draft has unresolved capability loss, blocking filesystem work, lossy workflow projection, and provider-specific cache behavior.
12 open findings
Partial inheritance drops the child's remaining tools · New Anthropic partial inheritance breaks wire-level cache matching · New Deletion edits cannot identify deleted symbols · New Unbounded sibling scanning performs excessive synchronous I/O · New Synchronous edit sounding blocks the Tokio worker · New NotFound sounding performs blocking filesystem I/O · New Synchronous read sounding blocks the async handler · New Search sounding runs blocking I/O after the async walk · New Project-map sounding blocks the async handler · New Capped skill listing makes later skills undiscoverable · New Workspace projection loses task-to-directory ownership · New Documentation misstates eager loading of active goal tools · New
What changed in this PR
This draft reduces model context costs through deterministic tool-result “echolocation,” deferred tool schemas, and forked-subagent prefix-cache inheritance.
Changes:
- Adds bounded navigation echoes to file, search, project-map, and mutation tools.
- Shrinks eager tool/skill/workflow payloads and expands the deferred-schema cache.
- Adds same-route fork-prefix inheritance and cache diagnostics.
| File | Description |
|---|---|
web/lib/public-surface-contract.test.ts |
Updates public tool-cache assertions. |
web/lib/content/tools.ts |
Revises public tool activation copy. |
docs/zh_hans/MODES.md |
Documents deferred tools in Chinese. |
docs/TOOL_SURFACE.md |
Updates eager-tool and cache contracts. |
docs/public-surface-facts.json |
Synchronizes public surface facts. |
docs/MODES.md |
Documents deferred Work-mode tools. |
docs/ECHOLOCATION.md |
Defines the echolocation feature. |
docs/CACHE.md |
Documents fork-prefix inheritance. |
crates/workflow/src/lib.rs |
Omits empty memo usage from serialization. |
crates/tui/src/tools/workflow/mod.rs |
Projects workflow events into smaller payloads. |
crates/tui/src/tools/subagent/tests.rs |
Expands deferral and inheritance tests. |
crates/tui/src/tools/subagent/mod.rs |
Implements child prefix inheritance. |
crates/tui/src/tools/spec.rs |
Tracks per-directory read activity. |
crates/tui/src/tools/skill.rs |
Bounds skill listings and descriptions. |
crates/tui/src/tools/search/tests.rs |
Tests grep limits and pod soundings. |
crates/tui/src/tools/search.rs |
Adds bounded grep results and pod charts. |
crates/tui/src/tools/project.rs |
Adds key-file symbol soundings. |
crates/tui/src/tools/mod.rs |
Registers the echolocation module. |
crates/tui/src/tools/file/tests.rs |
Tests read, miss, buzz, and edit echoes. |
crates/tui/src/tools/file.rs |
Adds file echoes and correct byte receipts. |
crates/tui/src/tools/file_search.rs |
Adds matched-directory pod data. |
crates/tui/src/tools/diff_format.rs |
Updates diff metadata documentation. |
crates/tui/src/skills/tests.rs |
Updates pathless skill-index tests. |
crates/tui/src/skills/mod.rs |
Removes native skill paths from prompts. |
crates/tui/src/session_manager.rs |
Applies formatting-only cleanup. |
crates/tui/src/prompt_zones.rs |
Adds live-header inheritance gates. |
crates/tui/src/hooks/executor.rs |
Updates deferred-tool classification tests. |
crates/tui/src/core/session.rs |
Expands and reprioritizes activation caching. |
crates/tui/src/core/engine/turn_loop.rs |
Captures and updates live request headers. |
crates/tui/src/core/engine/tool_catalog/tests.rs |
Updates eager/deferred catalog tests. |
crates/tui/src/core/engine/tool_catalog.rs |
Defers delegation and goal tools. |
crates/tui/src/core/engine/tests.rs |
Synchronizes engine surface tests. |
crates/tui/src/core/engine/preview/tests.rs |
Updates preview call signatures. |
crates/tui/src/core/engine/preview.rs |
Supplies goal state to preview catalogs. |
crates/tui/src/core/engine.rs |
Adds live headers and goal-aware activation. |
crates/tui/src/client/prepared.rs |
Dumps final wire prefixes when requested. |
crates/tui/src/client/chat.rs |
Expands prompt dumps across wire dialects. |
crates/tui/src/client.rs |
Updates child goal-surface expectations. |
crates/core/src/prefix_cache.rs |
Exposes canonical tool serialization. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
@Hmbown no problem, i'll get my agents working on it right now |
Preserve all eight original @AdityaVG13 commits and reconcile their source-navigation, bounded tool-output and fork-prefix work with current main88b4c58d. Use the existing Engine, Runtime header ownership, Core serializer and opaque installed route receipt; retain policy-filtered child tools, durable protected header observations and cold fallback on mismatches. Derived source reads remain protected/contained and current async consumers run in existing blocking workers. Retain current eager core tools and the eight-name/16KiB deferred cache. The explicit seven synchronous helper sites are recorded as future-call debt under the existing checker policy, not an intrinsic worker guarantee. Add the three feature rows and derive their facts plus the own-main agent docs from the existing owners. No unrelated release/catalog/pricing/OAuth source or WIP ancestry is included. Actual local Mac evidence is on the separate combined ECD+102-path source, with exact feature selections/source custody independently bound: Summary20 Runtime tests run:20passed/0failed/176outsidefilter; Summary30 Core tests run:30passed/0failed/67outsidefilter. Original TUI gate ran636:631passed/5failed; the next coherent repair ran58:57passed/1failed. These failures remain preserved. Search cap/context/trust/path-warning cases then passed; the same eager fixture was renamed/migrated and the two-case final selection passed2/0/15417outsidefilter. Feature registry2/0/0skipped. Actual CI-flags workspace/all-target/all-feature Clippy12m19s/0diagnostics; npm540Node+772Vitest=1312passed/0failed/7skipped; generated-docs repair and web/fmt/diff passed. No new agent test function/code comment. All79 original contributor declarations remain with the explicit eager-name migration; the conditional Unix fixture may early-return. The intended42-path own-PR tree has not yet been compiled or tested; fresh exact-head hosted Ubuntu/macOS/Windows/Lint/reviews remain a required HOLD before merge. No Windows distribution/provider cache-savings/live grant or final release claim. Source branch and original authors/history remain. Refs codewhale-hq#6393
|
Thank you, @AdityaVG13, for the index-free source navigation, bounded tool responses and fork-prefix work—and for being willing to keep working with us after main moved. Your eight original commits are preserved. I have pushed a history-preserving adaptation to your branch using the current Engine, route identity and protected-read boundaries; the PR body explains the12 review dispositions and the limits of the local evidence. The repaired canonical feature checks, registry, Clippy, npm and web gates passed. Fresh CI on your actual PR head is now running, so I am keeping the merge and review-thread resolutions gated on those results. The feature remains experimental for0.10.2, which has not shipped; no provider cache-saving or final Windows-package claim is being made. Thank you for giving us the source and tests to carry this forward as your own PR. |
Refs codewhale-hq#6393 The first own-head Version drift job failed its contributor-credit gate because the website list omitted @AdityaVG13, whose original source-navigation and fork-prefix work is preserved in this branch. Add their literal handle beside the existing unreleased contributor without changing product source or the credit gate. Frozen combined canonical verification: npm 540 Node plus 772 Vitest passed; 0 failed; 7 skipped. npm run check:web, fmt and diff checks pass. This exact file matches the verified source. The original failed job log is retained; fresh exact-head hosted CI, reviews and Windows owner-recovery proof remain required. No upload, release, provider call or Windows distribution acceptance occurred.
… not canonical grep_files walks the canonicalized search root from ToolContext::resolve_path but stripped matches against the raw context.workspace. Whenever the workspace path runs through a symlink (macOS /var and /tmp, symlinked project dirs) strip_prefix failed and every match came back as an absolute /private/var/... path. #6393's grep pods expose this as its macOS-only grep_pods_chart_matched_dirs_with_marks failure ("//private/var/.../net"). Strip against the canonical workspace as a fallback. Evidence (macOS, this tree): a temporary, uncommitted grep check on a tempdir workspace returned "net/server.rs" with the fix and "/private/var/folders/.../net/server.rs" without it (1 failed). Existing tools::search + tools::file_search: 41 passed, 0 failed. cargo fmt --check clean. Full workspace suite left to hosted CI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NBKbhx19BRbZtvWK8rVKjz
Preserve @AdityaVG13's source-echo work. codewhale-hq#6393 intentionally appends the pod footer after the paging footer (see contract_read_appends_a_pod_footer and the "page plus paging footer first, pod footer after" case); three assertions in two older main tests still encoded the pre-echo output and failed on every OS. The macOS-only grep_pods failure was a grep path bug on main, fixed there in c6bcf78 and merged in here. Evidence (macOS, merged tree): nextest tools::file + tools::search + echolocation + budget_handback: 273 passed, 0 failed; cargo fmt --check clean. Hosted CI on this head is the full-suite gate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NBKbhx19BRbZtvWK8rVKjz
Brings in codewhale-hq#6973; the only conflict was web/lib/release-credits.ts, where both PRs add a contributor. Kept both (@aboimpinto, @AdityaVG13). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NBKbhx19BRbZtvWK8rVKjz
Echolocation: measured baseline and next implementation sliceThanks @Hmbown for the history-preserving integration in Completed-task baseline: main, not this PRBefore extending the feature, we measured unmodified main at Protocol: first-party DeepSeek
Tool policy, prompts, and budgets were identical across repetitions. No answer hints, source modifications, shell commands, or internal-docs evidence were supplied. Success required all four requested points to be materially correct and source-cited, assessed by manual review against the pinned source, not by exit code.
Result: 5/6 correct completions. Median successful-task time: 50.299 s for T (2/3 correct) and 68.111 s for R (3/3 correct). The failed T1 answer incorrectly treated step-budget exhaustion as unconditionally ending in failure; the report-only/no-tool path can complete. That attempt remains in the table and cost totals. Across all six attempts: 111 model requests, 192 tool calls, 6,097,638 input tokens (5,457,142 cached; 640,496 uncached), 40,641 output tokens including reasoning, and $0.136830426 at the applicable provider usage rates. Limits: this is a small navigation baseline, not a coding/editing benchmark or evidence of echolocation improvement. Provider cache was shared and uncontrolled, and the model is a mutable provider alias. R1 received the engine's soft-landing notice before its final answer but still passed the source check. Wall time includes tools, thinking, and the spend-forwarding transport; it is not isolated provider latency. The earlier eight-step/non-thinking pilot produced six incomplete answers. It is preserved separately and is not used for successful-task timings. Including that pilot, aggregate usage-priced spend was $0.165399630. The September fork-prefix cache measurements also remain a separate historical experiment, not a speedup claim for the current PR. Exact prompts, commands, source/binary identities, actual answers, grading notes, and raw provider/native receipts are retained locally; the receipt bundle has not yet been published. Current integration and qualification
These are source-level checks of the pre-commit merge tree, with an injected model client for the Engine trajectories—not live-provider, packaged-platform, full-web or completed-task performance qualification. Earlier CI/budget receipts are not current-head receipts. The PR remains draft for the contemporaneous completion/control/ablation experiment below. Next: improve the navigation, then measure the completed workThe intended benefit is fewer searches to find relevant code, narrower useful reads, and safer use of source references, not merely a higher cache-hit percentage.
New source observations stay in append-only tool/history evidence, not the session-pinned prefix. This work stays in the existing Engine and tool paths; no daemon or parallel execution system is proposed. Wider composition and impact-navigation work follows only after this slice is integrated and qualified. This comment will be maintained as verified slices land. No current-PR speedup, cost reduction, or quality improvement is claimed yet. |
PR codewhale-hq#6393 own-head Ubuntu CI 38684332452 attempt 2 reported 19173 passed, 1 failed and 34 skipped: the existing failure/timeout fixture observed one request instead of two. Its 100ms step timeout also bounded the ordinary request and pre-report durable prompt persistence. Use the fixture's existing two-second timeout case and require the report-start notification before asserting the terminal result. Preserve the exact two-request, no-retry, usage, missing-coverage and deliverable assertions; include the mode and result in the request-count diagnostic. No production deadline, receipt persistence, or provider behavior changes. No new test functions or code comments. Exact source review and rustfmt passed. This source-only follow-up preserves the original contributor's history and draft status; actual new-head native CI and npm/web execution remain pending before merge. The canonical checkout and peer build inputs are unchanged. Refs codewhale-hq#6393 Signed-off-by: CodeWhale Bot <bot@codewhale.net>
…enerated docs Preserve AdityaVG13 and printf033 in release credits and regenerate the two TOOL_SURFACE entries in agent docs; all Rust and blocking-budget changes auto-merge, including the hand-back fixture repair. Use the existing docs derivation owner on the exact automatic-merge sources:26 topics,52 entries and39 paths. Only the documented conflict paths are replaced. Static full-tree and replacement-blob review, generation consistency and diff whitespace checks pass. No new production edits or canonical checkout changes. Both original contributor ancestry and main68f4dd3406 are retained. Fresh resulting-head native and web CI remain pending; this source reconciliation is not a merge-readiness or release claim. Refs codewhale-hq#6393 Signed-off-by: CodeWhale Bot <bot@codewhale.net>
codewhale-hq#6393 intentionally removes native skill file paths from the ambient index, but its unchanged runtime budget rejects the first new prompt fingerprint. Keep the complete provider-free measurement in the failing checker output so all changed identities and numeric limits can be reviewed from the exact CI source. Existing validation, budget contents and exit status remain unchanged. Validation: all 22 existing runtime-budget checker tests passed against the exact proposed script; syntax and four-line diff reviewed. Ubuntu 19,174/0/34 skipped and Windows 18,429/0/27 skipped belong to parent d991cff, not this new head. macOS and fresh hosted qualification remain pending. No local native build, provider call or budget relaxation. Canonical peer files untouched. Refs codewhale-hq#6393 Signed-off-by: CodeWhale Bot <bot@codewhale.net>
Merge current main 96ba4f8 into contributor PR codewhale-hq#6393 without rewriting its ancestry. Preserve worker ownership and late completion routing alongside the contributor's guarded header inheritance. The intentional pathless skill index shrinks the representative fixture by 56 bytes. Record the actual hosted values and four cumulative identities: seven numeric leaves decrease, four hashes change, and all other values and seven history notes remain unchanged. No ceiling is increased. Give fallback base measurements their own scratch Cargo target. Both recent runs compiled the current tree, built the base into its target, then reported 26 missing prompt_zones exports on the next current-tree compile. The exports exist in the PR. Isolating outputs prevents a base build from replacing PR dependency artifacts; the precise consumed metadata was not traced. Keep ratchet exit rules and the persistence budget unchanged. Correct only the two existing comments. Cold base rebuilds occur only after a current-tree failure. Verification: - CI 38771336339, Lint 116548295102, actual checkout 0f70b22 has the same complete tree as diagnostic head 5687729. Receipt line 2652 precedes base comparison. Root checked the raw log and checkout binding. - All 55 numeric metrics and 21 identities audited; only the seven decreases and four intended hashes differ. Old budget: exit 2. Replacement: exit 0, all 55 metrics exactly at budget. Estimated tokens 2451 -> 2437 are fixture estimates, not measured provider savings. - bash -n and ShellCheck passed; 48 existing Python harness tests passed (3 + 22 + 3 + 20). No new tests, code comments, Rust test changes or assertion weakening. - Prior fork Lint success was advisory: both actual ratchets exited 1. Fresh resulting-head native, web and budget checks remain required. No local Cargo or npm suite was run for this two-file follow-up. Raw measurement log SHA256: f60ff90e0b46345eb744106aa4c70372e0ff9553fb8e73a8f4a4d22bca4871b0 Preserve @AdityaVG13's original commits and branch. Keep the PR draft pending qualification. This is not release publication. Refs CW-6439, PR codewhale-hq#6393 Signed-off-by: Hunter Bown <hunter@shannonlabs.dev>
Preserve the contributor's original history and integrate main 248634f. This carries authoritative, isolated Runtime fixtures and settlement of stale approval/input requests, plus the earlier macOS LLVM setup and Windows startup diagnostics. The clean merge retains this PR's measured budgets and gate policies. The sole overlap adds owner_session_id to the duplicate-name fixture; the source-navigation and guarded inheritance implementation is unchanged. Main repair validation: 387 existing focused Rust tests passed, 0 failed; npm 1312 passed, 0 failed, 7 skipped; check:web passed. Exact merged-tree review preserved each PR delta and the main repair. This resulting PR head still needs fresh hosted qualification; previous runs remain historical. No provider traffic, deployment or publication. Signed-off-by: Hunter Bown <hunter@shannonlabs.dev>
|
@AdityaVG13 thanks for preserving the failed baseline attempt and separating source-navigation correctness from cache-hit claims. I also see that you returned this PR to draft and listed the revision-qualified navigation slice and contemporaneous completion/ablation measurements as unfinished. For the 0.10.2 intake, should the currently published slice be considered for merge once its exact-head checks pass, with that additional work following separately, or should this PR stay draft until the full slice and measurements are ready? We will retain draft status while that scope is unresolved. The latest history-preserving integration is — GPT-6 (AI agent), on behalf of the maintainer |
Retain the original candidate and contributor histories while qualifying the combined source in parallel with original PRs #6985 and #6989. Both contributor PRs must still land as themselves before this candidate's conditional squash. The source-navigation work in draft PR #6393 is excluded pending its author's scope decision and successful qualification. Include current main 248634f: worker ownership and late completion receipts, stale approval/input settlement, shared mock authority, isolated fixture workspaces, macOS LLVM setup and Windows clipboard launch diagnostics. Carry @henrikdukefoss's Standard Compute descriptor with Unknown billing and @shenjackyuanjie's OHOS Ninja selection and ungated CLI hashing unchanged. Resolve CI/credit/generated-document intersections, synchronize approval documentation with Engine execution IDs and captured input, and narrow the Windows 10 1607 statement to verified dynamic lookup behavior. Verification: object-only merge and authored/generated-document review; 72 feature rows and 26 documentation topics preserved; git diff --check passed. No build or native/npm test was run on this combined source. Prior source receipts remain separate; hosted exact-head qualification is required. Canonical main and its index remain unchanged. No publication is authorized. Signed-off-by: Hunter Bown <hunter@shannonlabs.dev>
|
@Hmbown in my opinion, I think we should leave it draft ready for the next major release depending on your timeline. I want to test and make sure that the idea is sound with some empirical data before merging. |
Sound every file read, grep, file search, and project map with its surroundings — sibling podmates, line-numbered symbols, import roots, resolved file links, and (in the terminal buzz) reverse-link callers — sounded fresh from source with no index, no background work, and no new dependencies. Deterministic (sorted, capped, budgeted) so repeated calls are byte-identical and cache-friendly. Echolocation (new: tools/echolocation.rs, ~2.5k lines incl. tests): - read footer: [Pod] mates + [Sounding] symbols/imports/links/heard-by, 1200B hard budget; miss echo on NotFound (denylist-gated, never fires on refused probes); terminal buzz adds callers after 2 distinct files read in a pod - grep + file_search chart matched dirs (caller's own visibility gates: denylist, excludes, extensions, gitignore); project_map sounds key files; 4 sounding languages (Rust/Python/JS-TS/Go) Token diet (no compression, behavior-preserving): - prefix 14,241 -> 7,118 tokens/turn (-50.0%): skill path cut (load_skill resolves names), MCP prose 1526->548B (scar-tissue phrases kept), agent+workflow deferral, goal tools conditional-eager keyed off the turn's resolved goal (preview/wire-safe) - activation cache: best-match-last push + regex name-rank + 16->24KiB bound (fixes deferral stranding agent behind its own discovery set) - grep empty-context omit, workflow event projection (lossless: duplicates dropped, workspaces hoisted, branch kept) + zero-memo omit - wire-prefix dump moved to PreparedOutboundRequest (exact-final-body) Evidence: targeted echo 46, workflow 131, file 145 green; full tui+workflow suites green except 4 failures reproducing identically on base c75b233 (pre-existing, unrelated) + known timing flakes proven solo-pass; clippy zero on owned files. Research: codewhale-research/ ECHOLOCATION-THESIS.md (exp-10/11/12 + hardening audit).
DEFAULT_ACTIVE_NATIVE_TOOLS is six names now (agent, workflow, get/update_goal deferred); the web contract mirror, tool docs, and mode docs still claimed ten eager + eleven with tool_search. Activation cache bound 16 -> 24 KiB in facts, docs, and web copy. Found by the fork staging PR's Lint & Type Check gate.
Footer squeeze (all lossless): same-pod links/callers render bare, import roots a link already names drop by file stem or package path, framing tightens to [Pod (N)] / [Sound], and sounded paths are /-separated on every OS. The squeeze follows CODEWHALE_ECHO gates so an off:links trial measures links alone. Edit echoes name touched symbols plus impacted callers on write/edit; the diff rides metadata for the TUI while the receipt stays one line. Grep max_results is clamped with honest truncation, skill listings are capped, JS arrows sound as their source token, and ECHOLOCATION.md documents the surfaces. Signed-off-by: AdityaVG13 <adityavgcode@gmail.com>
Relative imports now sound their leaf instead of a bare dot (from . import sib -> sib, ./server -> server), which composes with stem-shadowing when the link resolves and names the module when it does not. Removes the temporary footer-breakdown probe (its numbers landed in the PR; the eprintln tripped the print_stderr deny and flipped Lint red). Signed-off-by: AdityaVG13 <adityavgcode@gmail.com>
Child on the same route/model/tools inherits the parent's cached prompt prefix (system prompt + compacted history) as its frozen LivePrefix base, warming from the warm cache instead of going cold. Resolution is a pure deterministic gate (resolve_fork_inherit) — same route, same model, same or subset tools, child config digest must descend from the parent chain — so narrowed roles and grandchildren correctly fall back cold. No human-language compression anywhere; the child extends the parent's verbatim prefix with its own delegation message (append-only history per docs/CACHE.md), keeping prompt-cache prefix-matching intact.
Replace whole-block tool equality (which never held: parent wire blocks grow by deferred discovery while child grants are fixed and narrower) with longest byte-equal head resolution. The child adopts the parent's exact system bytes and sends the matched head on its first request; the rest of its grant stays deferred and activates on demand, so truncation costs cache bytes, never capability. Full head (inherited_full) prices shared history as hits; partial head (inherited_prefix) still prices system plus matched tools. Both sides now canonicalize through tool_to_api_json (the wire shape), so internal-only flag skew (defer_loading, allowed_callers) can't false-negative the comparison. The hot proof carries forward across append-only requests, since spawns execute before the current request's usage lands. Fresh spawns, grandchildren, resumes, route skew, and cold prefixes stay cold with distinct reasons. Trial instrumentation (CODEWHALE_FORK_TRIAL=1, default off) mirrors the decision, prefix span, history-divergence point, and child first-response cache split to stderr as JSON lines for same-binary A/B measurement; headless exec installs no tracing subscriber, so the tracing trial logs are invisible there. Measured (same-binary A/B, DeepSeek, unique histories, child first-response): true cold 0/9421 hits; primed cold 4736/9389; prefix k=3 5504/9941; full k=6 8576/13251. Inherited children reported correct results in every run. DeepSeek counts hits in 64-token blocks. Follow-ups: fork transcripts freeze at turn 1 (the live tail never rides), framing sits ahead of where the tail would (breaking the head match), grandchildren carry ~40KB all-miss transcripts (single-level v1).
The CODEWHALE_FORK_TRIAL readout intentionally writes JSON lines to stderr (headless exec installs no tracing subscriber, so the tracing trial logs are invisible there). Narrowly allow clippy::print_stderr on the two env-gated diagnostic blocks with justification, following the existing in-repo precedent. Documents the fork-prefix contract, measured benchmarks, trial procedure, and known follow-ups in docs/CACHE.md.
The tool layer hands echolocation canonical file paths while the workspace may be non-canonical (macOS /var to /private/var tempdirs, Windows verbatim paths). Three spots compared them raw and silently missed everywhere but Linux: - workspace_relative now canonicalizes both sides before giving up, restoring caller and link matches. - pod_relative_names compares in workspace-relative space instead of joining the raw workspace, restoring same-pod shortening. - Grep match paths fall back through the same canonical strip instead of rendering absolute. Verified against the three CI failures locally on macOS; Linux suites are unaffected (fast paths unchanged).
Refs codewhale-hq#6393 The first own-head Version drift job failed its contributor-credit gate because the website list omitted @AdityaVG13, whose original source-navigation and fork-prefix work is preserved in this branch. Add their literal handle beside the existing unreleased contributor without changing product source or the credit gate. Frozen combined canonical verification: npm 540 Node plus 772 Vitest passed; 0 failed; 7 skipped. npm run check:web, fmt and diff checks pass. This exact file matches the verified source. The original failed job log is retained; fresh exact-head hosted CI, reviews and Windows owner-recovery proof remain required. No upload, release, provider call or Windows distribution acceptance occurred.
Preserve @AdityaVG13's source-echo work. codewhale-hq#6393 intentionally appends the pod footer after the paging footer (see contract_read_appends_a_pod_footer and the "page plus paging footer first, pod footer after" case); three assertions in two older main tests still encoded the pre-echo output and failed on every OS. The macOS-only grep_pods failure was a grep path bug on main, fixed there in c6bcf78 and merged in here. Evidence (macOS, merged tree): nextest tools::file + tools::search + echolocation + budget_handback: 273 passed, 0 failed; cargo fmt --check clean. Hosted CI on this head is the full-suite gate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NBKbhx19BRbZtvWK8rVKjz
An echo naming a symbol still makes an agent search for its body. Make Rust reads produce actionable source windows, and preserve the distinction between host computation, access authority and model-visible delivery. Implementation - Parse the admitted byte snapshot with the existing syn dependency; publish bounded function/impl-method observations and omission counts. - Return path, offset, limit and SHA-256 expected_revision together so direct reads and execute_tools programs can consume the same reference. - Refuse changed source before applying stale offsets; normal unguarded reads retain their existing contract. - Reuse HandleStore in a separate bounded host-owned runtime store. Cloned runtime services share observations, not authority or an agent's delivery ledger. Cache hits require admission and current-byte equality. - Record references from final projected tool-result text only when the model client accepts the request. Hidden nested reads do not count as delivery; resume rebuilds use the existing logged conversation. - Keep rendered navigation within the existing 1200-byte echo footer budget and structured metadata within its separate bounded view. Architecture and compatibility No new daemon, executor, persistent index, dependency or prompt contributor. Navigation rides append-only tool-result history. The optional read schema field is static and pinned with the session tool catalog. Ordinary reads remain unguarded unless the caller supplies expected_revision. Limits This is Rust syntax evidence, not macro expansion, cfg evaluation, semantic name/type resolution, an LSP result or a repository-wide call graph. Shared facts do not prove body readership. No live-provider cache-hit, latency, token-saving or packaged-release claim is made for this slice. Integration Selectively replay the navigation checkpoint onto published PR head 5912dc2, without replaying its superseded reconciliation parent. Preserve the protected-read, ignore-walker, 32-candidate and blocking-pool repairs. Keep navigation cache-key canonicalization on the blocking pool. Update the existing schema-property assertion for expected_revision; the existing catalog byte-budget assertion passes unchanged. Verification Strict-remote RCH on spark-1672, cargo test -p codewhale-tui --lib --locked --offline, with one test thread and a 64 MiB test-thread stack. Selected navigation, echolocation, codemode, read/source, working-set, handle, file-tool/schema and grep/file-search pod tests: 159 passed; 0 failed; 0 ignored. No full-suite execution. The Engine regression uses an injected model client to inspect the actual projected request; tool composition runs the real JavaScript execution path. Both navigation regressions fail with the corresponding production guard removed (stale-source refusal and delivered-reference recording), then pass after restoration. Their assertions were unchanged through mutation. Changed-file rustfmt and git diff --check pass. UBS scanned changed Rust files but returned file-wide heuristic findings, not a clean gate. The worker lacks nextest. GNU ld overflowed AArch64 branch relocations; the passing command uses installed LLD via a command-local RUSTFLAGS, without changing repository build profiles or configuration. No native Windows/macOS test execution, workspace-wide Cargo run, live-provider benchmark or release build is claimed. No-Issue: local navigation port for fork PR #3 and upstream draft codewhale-hq#6393; publication remains a separate step.
sound_echolocation takes Option<&SourceNavigation> so ready navigation supplies symbols from the same admitted source snapshot, bypassing the legacy symbol scan. SSE line framing moves to a borrowing slice walker (take_sse_line_from_slice) over a single cursor. Subagent fork seeding hands the resolved fork straight through, and the read-guard, file, project and search surfaces follow the admitted-snapshot contract; budget handback tests track the new seams. Signed-off-by: AdityaVG13 <adityavgcode@gmail.com>
Six kept levers across the settled-cell caches, keyed steady-frame skip (details-epoch invalidation), and render-path shaping, gated one lever at a time by the ignored full-frame bench (bench_full_frame_scroll_cost_by_history_length) under a paired A/B release protocol: FRAME_PARITY buffer hashes must match byte-for-byte in empty and populated modes, AA noise bands <= ~1%, every frame config within the no-material-regression band. Cumulative populated p50 5711 -> 1543 ms over the campaign. The composed-cell row-cache direction (pass32-34) was explored and fully reverted after two consecutive gate rejects; the reverted tree re-proved FRAME_PARITY 6/6 byte-identical to the retained baseline binary. Evidence: 90/90 focused dev lane green (settled/files 13, transcript 70, mounted 5, keyed 2); ECHOLOCATION.md documents the measured budgets. Signed-off-by: AdityaVG13 <adityavgcode@gmail.com>
Reuse the secure file-open boundary, remove the duplicate navigation lookup and stray merge marker, and update fixtures to the current child catalog and workflow event shapes. Keep root goal mutation out of children and align discovery/cache checks with the 16 KiB schema budget. Verification on Spark: 24 child-surface/cache tests and 8 credential/read/workflow tests passed against byte-matched sources. The wider selected run had 872 passes and one independent Native-persona failure: AppArmor denies bwrap setpcap/net_admin. Touched-file rustfmt and diff checks pass; unrelated repository-wide formatting remains unchanged. UBS static scan reports legacy/test heuristics, not a clean gate. Signed-off-by: AdityaVG13 <adityavgcode@gmail.com>
Preserve the prior maintainer integration alongside the rebased navigation and render work without rewriting either history. Reconcile Engine constructors, request/fork state, protected file helpers, workflow evidence, and skill discovery with the current contracts. Isolate the invocation-gate fixture from real HOME skill libraries; keep model/user invocation assertions, child goal authority, and deferred-schema budgets unchanged. Verification: 181 selected codewhale-tui library tests passed on spark-1672 via RCH (zero failures); formatting passed for the 19 staged Rust files; generated website facts and git diff checks passed. Full suites, native packages, and live-provider qualification were not run. Refs codewhale-hq#6393.
…egistries Integrate upstream main at 4f1d7ab without rewriting contributor history. Resolve the two generated website registry conflicts from the merged source. Verification: 187 focused codewhale-tui library tests passed on the RCH worker, including real Engine navigation and outbound-reference projection. Repository and cloud fact checks pass. Refs codewhale-hq#6393
Progress update: implementation, optimization deltas and findingsThe latest integration is Measured optimization checkpointsThese are results from the individual optimization checkpoints, not a fresh performance qualification of the latest merged head. Timing rows report p50 on their named Linux ARM64 optimized-test workloads; process-inclusive probes are not isolated tool latency or completed-task timings. Deltas below are calculated from the unrounded before/after medians.
The two full-frame rows are benchmark-invocation totals, not milliseconds per frame, and the activity-cache result is included in the cumulative result; the percentages must not be added. That campaign used paired A/B and A/A controls plus byte-identical terminal-buffer checks. The earlier prefix and workflow rows are separate wire/synthetic-record experiments, not measurements of today's catalog or complete session cost. Other timing rows retain their own workload and assertion/parity limits; the streaming run did not collect full event-payload hashes. Additional findings and changes
Together, these changes support our larger goal: native, index-free echolocation that uses each observation to make the next navigation action more precise. Revision-qualified windows make symbol hints actionable, while freshness checks and delivery tracking keep source reuse honest. The scan and streaming changes reduce host overhead, while the rendering changes address responsiveness during long sessions. The intended progression is broad orientation, focused source inspection, useful relationship traversal, then a stop once the task has enough evidence. Success means correct, source-grounded results with fewer searches, rereads and model round trips, not merely smaller outputs or higher cache-hit rates. The completed-task comparisons still need to establish whether these component improvements deliver that outcome. Rejected or inconclusive workThe composed-cell/dirty-row cache direction failed the retention gate and was fully reverted; a roughly 0.5% populated result did not clear the 1% bar. Its revert preserved terminal-buffer parity. Crate-root memoization reduced metadata work without a separated whole-read latency gain and was also reverted. Ordinary whole-read and small-edit controls remained noise-limited, so the boundary/probe wins above are not generalized to those paths. Current verification and what remains187 focused native TUI tests passed against the integrated merge source: zero failed or ignored. These include the real Engine navigation trajectory, outbound-reference projection through tool composition, source/echo contracts, child discovery/cache behavior, skill invocation gates and workflow projection. Generated repository/cloud fact checks and diff checks also pass. The Engine trajectories use an injected model client; this is not live-provider, packaged-platform or completed-task performance qualification. The existing source-checked task baseline remains preserved, including its failed completion and incomplete pilot. The PR stays draft pending the contemporaneous completed-task comparison, control runs and echo ablations: correctness first, then completion time, calls, full token cost and echo overhead, including regressions. The component results above do not establish a current-head agent-task speedup, cost reduction, quality improvement or provider cache saving. PR still not done yet |



Index-free source-navigation echoes, bounded source reads and guarded fork-prefix inheritance on the existing Engine. Original contributor commits and credit are preserved; there is no replacement turn loop or indexing daemon.
Current head:
78985be7d, history-preserving integration of upstream main4f1d7ab56. The two website-registry conflicts were resolved by regenerating from merged source, not by discarding either side's feature declarations or docs.Implemented slice
Current source-level verification
The integrated merge source passed 187 focused
codewhale-tuilibrary tests: 0 failed, 0 ignored, executed remotely through RCH before the merge commit. This includes the real Engine navigation trajectory, outbound-reference projection through tool composition, child-surface/catalog discovery, forked message/session projection, source-read/edit contracts, echoes, searches, skill invocation gates and workflow projection.The Engine tests use an injected model client: they establish host execution and projected request contents, not live-provider acceptance or completed-task improvement. The initial skill-listing failure reproduced with unrelated global skills present and passed with an isolated home; the existing fixture now isolates
HOMEandCODEWHALE_HOMEunder the shared environment lock, without weakening its assertions.web/scripts/check-facts.mjs,web/scripts/check-cloud-facts.mjs, andgit diff --checkpass. No full workspace suite, full web build, new actual runtime-budget receipt, or packaged Windows/macOS acceptance is claimed from this run. Earlier native, npm, web, budget and CI receipts remain historical evidence, not qualification of this head.Still draft: empirical qualification remains
This stays draft in accordance with the contributor's scope decision. The source-checked baseline, including its failed completion and incomplete pilot, is preserved in the maintained progress record.
The revision-qualified navigation slice is integrated and has focused source-level evidence. The remaining task is the contemporaneous completed-task/control/echo-ablation comparison: correctness first, then completion time, calls, complete token cost and echo overhead, including regressions. No current-PR speedup, cost reduction, quality improvement or live-provider cache saving is claimed. Wider composition/impact-navigation work and release acceptance are not implied.
Refs #6393, CW-6439