Repository navigation
fix: preserve observed engine session initialization - #76815
Conversation
|
❌ PR Specification Reviewer failed to deliver outputs. Please review the logs for details.
|
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅
|
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Ponytail Reviewer completed successfully! Ponytail ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ PR Code Quality Reviewer completed the code quality review. PR Code Quality ReviewerOutcome: The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Design Decision Gate 🏗️Outcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
There was a problem hiding this comment.
Comments that could not be inline-anchored
.github/workflows/shared/crush.md:322
This early return drops `nativeInit` (and its diagnostic) whenever the harness-framed preamble is followed by a valid canonical JSON array/JSONL session, because only `records` and `terminal` are returned here. That violates T-UAS-009 and the new Section 3.5 initialization obligation to preserve observed initialization fields in canonical/full unified sessions. Include the startup events before returning the recognized records, and add a regression fixture for preamble + canonical session output...
Generated by PR Specification Reviewer for #76815 · copilot · gpt50 · ~3 AIC · ⌖ ~2 AIC · ⊞ 1.6K
Comment /review-specs to run again
Comment MemoryPeek at saved memory (pr-code-quality-reviewer)Note This comment is managed by comment memory.Expand the saved memory block to view or edit the persistent context for this thread.
|
There was a problem hiding this comment.
Found two correctness risks in the Crush startup parser. The exact-wording allowlist can silently drop all metadata after a CLI wording change, and the all-or-nothing model/session gate discards valid partial observations. Please make parsing resilient and emit independently observed fields. Static review only; repository code was not executed per review instructions.
🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~4 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review to run again
| const nativeLine = /^(?:INFO {2}(?:Running in non-interactive mode|Overriding large model for non-interactive run provider=\S+ model=\S+|Created session for non-interactive run session_id=[a-f0-9-]+|ModelProvider called provider=\S+ model=\S+)|WARN {2}(?:Failed to parse skill file path=.+ error="no YAML frontmatter found"|Ripgrep \(rg\) not found in \$PATH\. Some grep features might be limited or slower\.|Using large model as small model for unknown provider provider=\S+ model=\S+))\r?$/; | ||
| const firstAnswer = nativeLines.findIndex(line => !nativeLine.test(line)); | ||
| const nativePreamble = nativeLines.slice(0, firstAnswer < 0 ? nativeLines.length : firstAnswer).join("\n"); | ||
| const model = nativePreamble.match(/^INFO {2}Overriding large model for non-interactive run provider=(\S+) model=(\S+)\r?$/m); |
There was a problem hiding this comment.
The all-or-nothing gate here discards valid partial evidence: if the model/provider line is observed but the session-id line is absent or changes format, nativeInit is omitted entirely. Since all fields in SessionInitData are optional, emit session.init with whichever independently validated fields were observed and keep the remaining startup lines in the diagnostic event.
There was a problem hiding this comment.
Fixed in 96d94dd: provider/model and session ID are extracted independently, preserving whichever metadata is observed; framed startup lines remain diagnostics. Both Crush locks were regenerated, and final validation passed.
| const executionStart = logContent.match(/^\[crush-harness\] starting Crush CLI\r?\n/m); | ||
| const preamble = executionStart ? logContent.slice(executionStart.index).match(/^\[crush-harness\] starting Crush CLI\r?\n(INFO {2}Running in non-interactive mode(?:\r?\n(?:INFO|WARN|ERROR) {2}[^\r\n]*)*)/) : undefined; | ||
| const nativeLines = preamble?.[1].split("\n") ?? []; | ||
| const nativeLine = /^(?:INFO {2}(?:Running in non-interactive mode|Overriding large model for non-interactive run provider=\S+ model=\S+|Created session for non-interactive run session_id=[a-f0-9-]+|ModelProvider called provider=\S+ model=\S+)|WARN {2}(?:Failed to parse skill file path=.+ error="no YAML frontmatter found"|Ripgrep \(rg\) not found in \$PATH\. Some grep features might be limited or slower\.|Using large model as small model for unknown provider provider=\S+ model=\S+))\r?$/; |
There was a problem hiding this comment.
This parser only promotes metadata when the startup lines match an exact wording allowlist. A minor Crush wording/version change therefore makes nativeInit undefined and silently drops all observed model/provider/session metadata—the regression this change is intended to prevent. Please retain the recognized harness preamble as diagnostics and either parse fields independently or emit a warning when the framed preamble is present but does not match.
There was a problem hiding this comment.
Fixed in 96d94dd: metadata extraction no longer depends on the exact startup-message allowlist, and changed startup logger lines are retained as diagnostics. Both Crush locks were regenerated, and final validation passed.
There was a problem hiding this comment.
🟡 Changes recommended
After startup-prefix removal, the quiet-mode decoder can misclassify JSON answers as saved-session tool activity, while [] drops all events, including initialization and diagnostics.
1 open issue
What changed in this PR
This PR preserves observed engine initialization metadata in gh-aw’s unified session artifacts and separates recognized Crush startup diagnostics from assistant output.
Changes:
- Retain CLI version and provider fields in unified initialization events, types, and schemas.
- Extract Crush startup model, provider, and session identifiers.
- Add regressions, update documentation, and regenerate Crush and Kiro workflow locks.
| File | Description |
|---|---|
docs/src/content/docs/specs/unified-agent-session-specification.md |
Document retained initialization fields. |
docs/public/schemas/unified-session.schema.json |
Add version and provider to unified schemas. |
docs/public/schemas/agent-session.schema.json |
Add version and provider to canonical schema. |
actions/setup/js/unified_session_payload.test.cjs |
Test initialization field retention and aliases. |
actions/setup/js/unified_session_payload.cjs |
Preserve version and provider during projection. |
actions/setup/js/types/unified_session.d.ts |
Extend unified initialization types. |
actions/setup/js/types/agent_session.d.ts |
Extend canonical initialization types. |
actions/setup/js/parse_deepseek_log.test.cjs |
Verify DeepSeek metadata survives projection. |
actions/setup/js/kiro_session_pipeline.test.cjs |
Verify Kiro initialization survives the pipeline. |
actions/setup/js/crush_session.test.cjs |
Cover startup extraction and answer preservation. |
.github/workflows/smoke-kiro.lock.yml |
Refresh generated workflow hashes. |
.github/workflows/smoke-crush.lock.yml |
Include the updated Crush parser. |
.github/workflows/shared/kiro.md |
Update comments describing version retention. |
.github/workflows/shared/crush.md |
Extract startup metadata and separate diagnostics. |
.github/workflows/engine-conformance-kiro.lock.yml |
Refresh generated workflow hashes. |
.github/workflows/engine-conformance-crush.lock.yml |
Include the updated Crush parser. |
.changeset/patch-observed-session-initialization.md |
Record the patch release change. |
Review effort: Balanced (auto)
| if (nativeInit) { | ||
| logEntries.push({ type: "session.init", data: nativeInit }); | ||
| logEntries.push({ type: "crush.diagnostic", data: { content: nativePreamble } }); |
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /diagnosing-bugs, /tdd and /codebase-design. No blocking issues. Two suggestions are posted inline.
📋 Key Themes & Highlights
Key Themes
- Partial preamble handling (
shared/crush.md):session.initneeds both the model and session lines. If either is missing, the preamble leaks into answer text. This predates the PR, so it is not a regression, but it is the gap this PR targets. - Untyped new fields (
unified_session_payload.cjs):providerandagentVersionskip the type guards used by neighbouring fields. - Undocumented event:
crush.diagnosticis not in the unified types,EVENT_FIELDS, or the spec. - Test gap: no test covers a partial preamble (only the session line or only the model line missing).
Positive Highlights
- ✅ The preamble is framed and matched against an allowlist, so quoted or unframed text is not promoted to
session.init. - ✅ The Kiro test now compares against the raw parse, which is a cleaner assertion.
- ✅ A patch changeset is included.
Scope
Reviewed against HEAD ae44735. The local working tree is dirty and differs from the PR head, so I did not review those edits. Vitest was not run in this environment.
@copilot please address the review comments above.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~7 AIC · ⌖ ~2 AIC · ⊞ 6.6K
Comment /matt to run again
| const nativePreamble = nativeLines.slice(0, firstAnswer < 0 ? nativeLines.length : firstAnswer).join("\n"); | ||
| const model = nativePreamble.match(/^INFO {2}Overriding large model for non-interactive run provider=(\S+) model=(\S+)\r?$/m); | ||
| const session = nativePreamble.match(/^INFO {2}Created session for non-interactive run session_id=([a-f0-9-]+)\r?$/m); | ||
| const nativeInit = model && session ? { sourceEngine: "crush", provider: model[1], model: model[2], sessionId: session[1] } : undefined; |
There was a problem hiding this comment.
[/diagnosing-bugs] nativeInit requires both the Overriding large model and the Created session lines. If either is missing, session.init is dropped and the harness preamble lines stay in assistant.message text. That is the leak this PR is meant to prevent. Consider emitting session.init from whichever fields are present, and always stripping the framed preamble once [crush-harness] starting Crush CLI matches.
@copilot please address this.
There was a problem hiding this comment.
Fixed in 96d94dd: initialization retains partial observed fields, and framed startup output is separated from assistant text. Both Crush locks were regenerated, and final validation passed.
| "session.init": { | ||
| sourceEngine: ["sourceEngine"], | ||
| agentVersion: ["agentVersion", "agent_version"], | ||
| provider: ["provider"], |
There was a problem hiding this comment.
[/codebase-design] provider and agentVersion are copied by selectFields without a type check. The neighbouring fields are guarded (for example isNonBlank). Add a string guard so a non-string value from a parser cannot reach the unified session.
@copilot please address this.
There was a problem hiding this comment.
Fixed in 96d94dd: non-string agentVersion and provider values are omitted from both session.init and session.start projections, with regressions added.
|
🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:
Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering. Structured data{
"completed_task": 0,
"head": "681a2632f8f3e84827817bbc8b6a330c282dd7e0",
"kind": "requested",
"schema_version": "1",
"state": "8270951a1e6e6414f5b45397c79c7bc23e4f284f845f350e706c37d533a2b1c1",
"work": [
"06e3a7ad1b2a8d03f715ee2c2da9a424737a14053eab55e24fb90dfc96ea4010",
"847a92e67259c6e7507cd70a9a23253dd618e85dbcf4d03e31a3f0e46a35ef41",
"d7de3a9f32e6e2e597af85948b5537e83ae94726abfbb4fed9c2d9f22e7fa511",
"ec7788b22e1517bbb2739e43b81b8e1c11a841f5e69b1ac79241eb3c2fb615b0"
],
"workflow": "pr-sous-chef"
}
|
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Merged latest |
|
🎉 This pull request is included in a new release. Release: |

Recent engine runs retained initialization metadata in raw logs or canonical traces but lost it from the unified session artifact. This preserves that observed evidence without inventing tool activity, accounting, or execution outcomes.
Changes
agentVersionandproviderin unifiedsession.init/session.start, with matching types and generated schemas.Observed evidence
2.31.1survived canonical parsing but disappeared from the unified initialization.Validation
make agent-report-progresspassed, including type checking, lint, schema freshness, focused tests, and workflow-lock drift checks.make recompilecompiled all 365 workflows. No workflows were dispatched or rerun.Review status
Draft pending final review. The earlier automated security review found no confirmed vulnerabilities, but could not review the final revised Crush parser and generated locks because its synchronous reviewer could not accept follow-up. This PR does not claim that review gate passed.