Skip to content

feat(cli): full conversation preview in the trajectory browser - #510

Merged
0xKT merged 14 commits into
mainfrom
feat/trajectory_full_preview
Sep 19, 2026
Merged

0xKT merged 14 commits into
mainfrom
feat/trajectory_full_preview

Conversation

@forrestjgq

Copy link
Copy Markdown
Contributor

Summary

Space on an attempt row in the raven trajectory browser previously
showed two truncated per-turn preview lines. This PR replaces it with an
interactive full-conversation viewer. It re-lands the content of PR #382,
which was auto-closed by the 2026-09-12 history linearization without
being merged; the branch is rebased onto the linearized main, with one
extra commit expressing the CJK display-width test fixtures as unicode
escapes to satisfy the source-language gate that main gained since.

  • Data layer raven/trajectory/conversation.py: rebuilds an attempt's
    span snapshot into a causally ordered stream of labeled records (User
    input, LLM input/output, Tool input/output, skill, memory, and
    subagent events). Event-time semantics (inputs at span start, outputs
    at span end) plus nesting depth from parentSpanId keep the causal
    order. Full content comes from artifact files (paths restricted to
    the trace store, streamed 512 KiB cap per file); span previews are
    only a visible fallback. ERROR spans and expected-but-unreadable
    payloads always yield a placeholder record. LLM inputs omit only the
    item-by-item verified common message prefix against the previous call
    (omissions and rewrites are announced); tool_calls of unexecuted
    tools keep their full arguments.
  • Rendering: kind-colored labels aligned to one column, hanging-indent
    bodies with word-aware CJK-safe wrapping, degraded/error/meta note
    lines, single-pass turn separators (interleaved traces are never
    regrouped), and a stacked layout on narrow terminals that never drops
    characters.
  • Interactive viewer (prompt_toolkit full-screen, entered for every
    preview on a terminal): scrolling, a muted bottom bar with an
    'h for help' hint, filter/collapsed state and the scroll range (laid
    out by display width, never overflowing any terminal), a scrollable
    help page reachable in full on small screens, a global collapse (s:
    bodies over five display lines fold to five plus an ellipsis; error
    and degradation notes never fold), and a label filter (%: real input
    line, case-insensitive, empty clears; filtered-out turn groups lose
    their separators). Resizes re-wrap and re-judge folding live; the
    content caret is hidden (only the filter input shows one). q/Esc
    returns to the list; Ctrl+C cancels the whole browser from any viewer
    state. A rebuild failure falls back to the legacy per-turn preview;
    without a terminal the preview degrades to direct printing.
  • Every dynamic field passes the sanitization gates (control
    characters, rich markup, and ANSI never reach the terminal). Adds the
    Conversation Record term to CONTEXT.md.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

  • uv run pytest tests/test_trajectory_conversation.py tests/test_cli_preview_viewer.py tests/test_cli_trajectory_browse.py tests/test_trajectory_store.py -q -> 298 passed (viewer assertions
    run against really rendered screens)

  • uv run pytest -q (full, on the rebased branch) -> 23463 passed,
    125 skipped; 33 failed + 20 errors reproduce with identical node ids
    on a clean origin/main worktree - environment baseline, not
    introduced here (passed-count delta of 96 equals the tests this
    branch adds)

  • make check-source-language, make check-large-files,
    make lint-types, make lint-python, make lint-imports -> all pass

  • Manual: rendered 6 real local attempts at widths 72/120/30 with zero
    over-width lines (largest attempt 7060 lines in 0.35s); drove the
    real TUI in a pty at normal and 44x12 sizes (viewer, help paging,
    collapse, label filtering, Ctrl+C cancel)

  • Relevant tests pass locally

  • Relevant lint / type checks pass locally

  • User-facing docs or screenshots are updated when needed

Risk

  • Read/display side only; no persisted format changes. The untrusted
    input surface (artifact content, typed filter input) is contained by
    the store-prefix path check, the 512 KiB streamed cap, and per-field
    sanitization.

  • User-visible change: the Space preview is now a full-screen viewer;
    the legacy rendering remains as the automatic fallback when the
    rebuild fails, and non-terminal environments print directly.

  • Rollback: revert this PR, or restore the previous _preview_screen
    call site to get the old preview back.

  • Security impact considered

  • Backward compatibility considered

  • Rollback path is clear for risky changes

Related Issues

N/A

forrestjgq and others added 12 commits September 19, 2026 01:38
Add raven/trajectory/conversation.py: attempt_conversation rebuilds an
attempt's span snapshot into a flat, causally ordered stream of labeled
ConversationRecord entries (User input, LLM input/output, Tool
input/output, skill/memory/subagent events). Inputs sort at span start,
outputs at span end; ties break by phase, then by nesting depth derived
from parentSpanId. Full content is read from artifact files (paths
restricted to the trace store, 512 KiB cap) with span previews as a
visible fallback; ERROR spans and unreadable expected payloads always
yield a record instead of disappearing. LLM inputs omit only the
verified common message prefix within one (traceId, parentSpanId) chain
and announce omissions and rewrites. Document the Conversation Record
term in CONTEXT.md.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
Bound artifact reads to the 512 KiB cap plus one byte instead of reading
whole files first, and reject non-regular files before opening them.
Emit a Turn marker record for body-less session.turn roots so a turn's
identity and start time survive an empty body. Render thinking_blocks as
reasoning evidence when reasoning_content is absent, without duplicating
identical text. Always emit a separate malformed-record notice so a
readable payload cannot make a span with corrupt status or attributes
read as fully OK.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
A session.turn whose root carries only an output (old or damaged logs)
produced no input-phase record, so the turn's own start time was
unaddressable for rendering. Emit the Turn marker whenever the root
yields no User input record, not only when the body is fully empty.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
An ERROR is a completion event: attach it to the span's last
output-phase record instead of whatever was emitted last, so the Turn
start marker never carries an end-of-turn failure. A span without any
output-phase record gets a completion placeholder at endTime.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
Space on an attempt row now renders the causally ordered record stream
from raven.trajectory.conversation instead of the two truncated per-turn
preview lines: kind-colored labels aligned to one column, hanging-indent
bodies wrapped by display width (CJK-aware), degraded/error/meta notes,
and turn separators inserted in a single pass over the ordered stream so
interleaved traces are never regrouped. Narrow terminals switch to a
stacked layout that never drops characters. Streams taller than the
screen page through less when an interactive pager is confirmed (both
stdio ends a tty, capable TERM, less on PATH); the subprocess is judged
by its exit code and only a clean pager run skips the press-any-key
waiter. Every dynamic field passes the sanitization gates before layout.
A rebuild failure falls back to the legacy per-turn preview.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
Ctrl+C while paging now closes the pipe, terminates and reaps the pager
(killing it after a bounded wait), then raises the browser-wide cancel
instead of unwinding as a raw interrupt; the early-quit BrokenPipe path
also releases the pipe before the exit code decides. Turn separators are
pre-wrapped like every other line, so the produced line count is the
true screen usage that drives the pager threshold, and the minimum
layout width is an explicit 2-cell floor (a wide character cannot be
split) instead of a silent widening to 4.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
Real-trace self-testing showed two readability gaps: bodies were
hard-wrapped mid-word (display-width only), and an LLM call repeated the
same model/token note on its input, thinking, and output records. Wrap
now breaks after the last space when one exists (hard wrap remains for
unbroken runs; the width contract is unchanged), and consecutive records
of one span sharing a meta show it once, on the last of them. Adds the
pager-threshold boundary test (exactly height minus two lines prints
directly, one more pages) and a three-trace interleave render snapshot.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
Span ids are only unique within one trace, so a merged attempt can
interleave records whose span ids and meta match across traces; the
consecutive-meta suppression now also requires the same trace id, so
each trace keeps its own note.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
Replace the external less pager with a prompt_toolkit full-screen
viewer: the new bottom bar shows a muted 'h for help' hint plus filter
and collapse state, h opens an aligned key-reference page, s globally
folds any body longer than five display lines (judged after wrapping)
down to five plus a dim ellipsis while never hiding degraded/error/meta
notes, and % opens a real input line that filters records by label,
case-insensitively (empty input clears; groups left empty lose their
turn separators). The viewer passes its actual content width to the
line builder on every render, so a resize re-wraps and re-judges
folding, and Ctrl+C is the global cancel in all three states (viewing,
help, filter input), surfacing as the browser-wide cancel. Rich lines
convert to prompt_toolkit fragments with whole-line styles merged
explicitly. Without a terminal the preview degrades to direct printing
with the press-any-key waiter; the less subprocess management and its
tests are removed.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
The bottom bar now budgets by display width: the scroll range and the
'h for help' hint always survive, variable fields (title, filter,
collapsed) shrink first with the first non-fitting one clipped, the
hint compacts to 'h' before dropping, and a final clip keeps the bar
inside any terminal (CJK filters measured by cells, not len). The help
page became a scrollable line flow: scroll keys page it, any other key
returns to the preview at its kept position, long entries wrap at
narrow widths, and its own offset feeds the bar's range. Tests assert
against really rendered screens (renderer frames and a synchronous
layout render), covering collapse+filter on 40 columns, long and CJK
filters, and small-screen help reachability.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
The focused content control painted its caret on the first character of
the view and the help page (covering the P of 'Preview keys'). The
control keeps focus for a valid layout but no longer shows a cursor;
the filter input line still does, where typing happens.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
The preview tests exercise CJK display-width handling; the linearized
main gates PR additions on English-only source, so the fixture values
move to named constants built from unicode escapes. Runtime values and
assertions are unchanged.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: current LLM input artifacts must be resolved before the full preview can merge.

I found one blocker in the live trace compatibility path. I covered the full diff, AGENTS.md and the canonical trajectory terms in CONTEXT-MAP.md/CONTEXT.md, the preview callers, the current trace writer and artifact resolver, relevant history, backward compatibility, and whether the tests weakened or bypassed behavior; I found no separate architecture issue.

Verification: uv run pytest tests/test_trajectory_conversation.py tests/test_cli_preview_viewer.py tests/test_cli_trajectory_browse.py -q passed (221 tests); uv run pytest tests/test_tracing_api.py tests/test_trajectory_bundle.py -q passed (89 tests); the source-language and large-file check scripts passed. A direct reproduction using TraceStore.address_items and audit.artifact.v2 showed contains full message: False, with the record displaying a $msg hash instead.

Comment thread raven/trajectory/conversation.py Resolved
The live trace writer stores an llm.input as a v2 shell whose messages
are $msg references into the content-addressed store; the preview
parsed the shell as inline JSON and rendered reference dicts, hiding
every newly recorded LLM input. The shell now resolves through
raven.tracing.artifact_v2 before prefix comparison and rendering; a
missing message blob renders as its deterministic placeholder and is
announced in the record's degradation note without breaking the
verified-prefix omission of later calls. The new tests build shells
through the real TraceStore writer instead of hand-built inline
payloads.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: v2 message-blob reads must honor the preview size bound.

The original $msg rendering defect is fixed and its thread is resolved. I found one new blocker in the resolver path. I rechecked the full PR diff and the 80fe873 delta, AGENTS.md, CONTEXT-MAP.md/CONTEXT.md terminology, the preview caller, the live v2 writer/resolver and image-bearing request path, relevant history, backward compatibility, and whether tests were weakened; no other actionable issue survived refutation.

Verification: uv run pytest tests/test_trajectory_conversation.py tests/test_cli_preview_viewer.py tests/test_cli_trajectory_browse.py tests/test_tracing_api.py tests/test_trajectory_bundle.py -q passed (313 tests). The source-language and large-file scripts passed. A real TraceStore reproduction with one 700 KiB image-bearing message produced a 716,900-byte blob, a 716,878-character preview record, degraded=None, and the complete payload in memory despite _ARTIFACT_LIMIT being 524,288 bytes.

Comment thread raven/trajectory/conversation.py Resolved
resolve_payload reads each referenced blob with an unbounded read_text,
so an inline-image message bypassed the preview's 512 KiB limit. Blob
resolution now goes through the preview's own bounded reader (reference
validation and addressing stay in raven.tracing.artifact_v2): a blob
over the cap renders as a deterministic placeholder instead of its
content, a missing or unparseable one keeps the existing placeholder,
and both are announced per payload in the degradation note. Blobs are
cached per attempt, so repeated LLM histories read each blob once while
every payload still reports its own unavailable blobs.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

The bounded v2 blob fix resolves the remaining blocker, and both threads I opened are now resolved. I rechecked the full PR and the f49c291 delta, AGENTS.md and trajectory terminology, the preview caller, current v1/v2 writers and resolver behavior, image-bearing and damaged-store compatibility, relevant history, architecture boundaries, and whether tests were weakened; I found no new actionable issue.

Verification: uv run pytest tests/test_trajectory_conversation.py tests/test_cli_preview_viewer.py tests/test_cli_trajectory_browse.py tests/test_tracing_api.py tests/test_trajectory_bundle.py -q passed (316 tests). The source-language and large-file scripts passed. The original 700 KiB reproduction now performs one bounded 524,289-byte read, emits an 87-character placeholder with the degradation note, and does not materialize the payload.

@0xKT

0xKT commented Sep 19, 2026

Copy link
Copy Markdown
Member

Not a blocker -- two things, neither of which changes a rendered pixel today.

1. _resolve_v2 is a second decider of the v2 per-field rule, and it already disagrees with the
owner
(raven/trajectory/conversation.py:415)

raven/tracing/artifact_v2.py:6-7 says of that module: "The canonical serialization, the envelope
key, the reference validation and the per-field resolution rules all live here and nowhere else" --
and gives the reason, that a JavaScript mirror exists and a cross-language round-trip test compares
the two.

_resolve_v2 delegates the parts its own docstring names -- is_v2, ref_sha1 -- but re-decides
the rest: it strips artifactFormat itself and resolves messages itself, and it does not resolve
TEXT_FIELDS (systemPrompt, prompt), which the owner does at artifact_v2.py:145-148.

Measured on one shell, both resolvers pointed at the same blobs:

OWNER artifact_v2.resolve_payload   systemPrompt = 'You are Raven.'
NEW   conversation._resolve_v2      systemPrompt = {'$msg': 'e7b04557...'}
messages equal                      True      <- control: both read the same blobs
whole payload equal                 False
fields that differ                  ['prompt', 'systemPrompt']

The messages line is the control: it is identical, so the divergence is the TEXT_FIELDS policy
and not my harness. (My first attempt put the blobs under the owner's default layout rather than
state/logs/audit-artifacts, which made every message read as missing -- worth saying because that
version would have looked like a much larger defect and was my setup, not your code.)

Nothing renders differently today: this module never reads systemPrompt or prompt -- grep over
conversation.py finds neither outside the resolver -- so I am filing this as what it is, a second
live decider of one rule rather than a visible bug.

The reason for the fork is legitimate and worth keeping: resolve_payload's inner load() reads
the blob with an unbounded read_text (artifact_v2.py:133), which is exactly the bound this PR set
out to impose. That argues for giving the owner a bounded-read hook -- a read= callable, say --
and calling it from here, rather than keeping a partial copy that starts out already differing on
two fields. As it stands the tree has three implementations of one wire rule (this one, the owner,
and server.js:1088) and a round-trip test that compares only two of them.

Smaller, same family: _oversize_placeholder (conversation.py:386) is a second placeholder shape
beside artifact_v2.placeholder, which this module uses correctly two lines away at :405.

2. Every preview open renders the whole conversation twice (raven/cli/trajectory_browse.py:535)

if not _conversation_lines(records, width):      # rendered, tested for emptiness, discarded
    ...
if _viewer_available():
    def _make_lines(...): return _conversation_lines(records, view_width, ...)
    if pviewer.view_lines(_make_lines, ...)      # rendered again

The first call exists to answer "is there anything to show", and pays for a full render to do it.
For a long attempt that is the whole cost twice on every open. Either keep the first result and
hand it to the viewer as the initial width's lines, or answer the emptiness question from
records.

For the record, what I checked hardest and found sound: the 512 KiB bound is applied where the
docstring says it is -- _load_message_blob routes every blob through _read_artifact
(conversation.py:402-403) rather than reading it directly, so a content-addressed inline image gets
the same bound as any other artifact; and the four test files this PR touches, plus the owner
module's own suite, are 248 passed on this head.

@0xKT
0xKT merged commit 32e612d into main Sep 19, 2026
23 checks passed
@0xKT
0xKT deleted the feat/trajectory_full_preview branch September 19, 2026 12:24
LivXue pushed a commit that referenced this pull request Oct 12, 2026
## Summary

Space on an attempt row in the `raven trajectory` browser previously
showed two truncated per-turn preview lines. This PR replaces it with an
interactive full-conversation viewer. It re-lands the content of PR
#382,
which was auto-closed by the 2026-09-12 history linearization without
being merged; the branch is rebased onto the linearized main, with one
extra commit expressing the CJK display-width test fixtures as unicode
escapes to satisfy the source-language gate that main gained since.

- Data layer `raven/trajectory/conversation.py`: rebuilds an attempt's
  span snapshot into a causally ordered stream of labeled records (User
  input, LLM input/output, Tool input/output, skill, memory, and
  subagent events). Event-time semantics (inputs at span start, outputs
  at span end) plus nesting depth from parentSpanId keep the causal
  order. Full content comes from artifact files (paths restricted to
  the trace store, streamed 512 KiB cap per file); span previews are
  only a visible fallback. ERROR spans and expected-but-unreadable
  payloads always yield a placeholder record. LLM inputs omit only the
  item-by-item verified common message prefix against the previous call
  (omissions and rewrites are announced); tool_calls of unexecuted
  tools keep their full arguments.
- Rendering: kind-colored labels aligned to one column, hanging-indent
  bodies with word-aware CJK-safe wrapping, degraded/error/meta note
  lines, single-pass turn separators (interleaved traces are never
  regrouped), and a stacked layout on narrow terminals that never drops
  characters.
- Interactive viewer (prompt_toolkit full-screen, entered for every
  preview on a terminal): scrolling, a muted bottom bar with an
  'h for help' hint, filter/collapsed state and the scroll range (laid
  out by display width, never overflowing any terminal), a scrollable
  help page reachable in full on small screens, a global collapse (s:
  bodies over five display lines fold to five plus an ellipsis; error
  and degradation notes never fold), and a label filter (%: real input
  line, case-insensitive, empty clears; filtered-out turn groups lose
  their separators). Resizes re-wrap and re-judge folding live; the
  content caret is hidden (only the filter input shows one). q/Esc
  returns to the list; Ctrl+C cancels the whole browser from any viewer
  state. A rebuild failure falls back to the legacy per-turn preview;
  without a terminal the preview degrades to direct printing.
- Every dynamic field passes the sanitization gates (control
  characters, rich markup, and ANSI never reach the terminal). Adds the
  Conversation Record term to CONTEXT.md.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_trajectory_conversation.py
  tests/test_cli_preview_viewer.py tests/test_cli_trajectory_browse.py
  tests/test_trajectory_store.py -q` -> 298 passed (viewer assertions
  run against really rendered screens)
- `uv run pytest -q` (full, on the rebased branch) -> 23463 passed,
  125 skipped; 33 failed + 20 errors reproduce with identical node ids
  on a clean origin/main worktree - environment baseline, not
  introduced here (passed-count delta of 96 equals the tests this
  branch adds)
- `make check-source-language`, `make check-large-files`,
  `make lint-types`, `make lint-python`, `make lint-imports` -> all pass
- Manual: rendered 6 real local attempts at widths 72/120/30 with zero
  over-width lines (largest attempt 7060 lines in 0.35s); drove the
  real TUI in a pty at normal and 44x12 sizes (viewer, help paging,
  collapse, label filtering, Ctrl+C cancel)

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- Read/display side only; no persisted format changes. The untrusted
  input surface (artifact content, typed filter input) is contained by
  the store-prefix path check, the 512 KiB streamed cap, and per-field
  sanitization.
- User-visible change: the Space preview is now a full-screen viewer;
  the legacy rendering remains as the automatic fallback when the
  rebuild fails, and non-terminal environments print directly.
- Rollback: revert this PR, or restore the previous `_preview_screen`
  call site to get the old preview back.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants