Repository navigation
perf(chat): render a streaming answer block by block instead of re-parsing the whole message per token - #4422
Conversation
…rsing the whole message per token Every streamed token re-lexed, re-parsed, and re-sanitized the entire assistant message and then replaced its whole DOM subtree, so each frame cost as much as the text so far (about 9 ms per frame at 12 KB and 90 ms at 37 KB in the parser and sanitizer alone) and long answers dropped frames while they streamed. While a message streams, MarkdownBlock now keeps the previous frame's top-level tokens, re-lexes only the appended tail, and renders one dangerouslySetInnerHTML payload per block. A settled block keeps its payload object, so React leaves its DOM alone and only the growing block repaints; per-frame cost stays flat (about 0.4 ms) regardless of length. Once the run settles the message renders as one document, exactly as history does, and a source with reference-style link definitions always takes the full render because those resolve across blocks. Journey: evals/specs/streamed-markdown-answer.e2e.test.ts streams a multi-block answer through the app and asserts that finished blocks render as markdown while later blocks are still arriving, that every block is on screen exactly once when the answer settles, and that a reload renders the same document.
…nt mock The streamed-markdown world served its answer from a loopback HTTP server on the test host, which the engine cannot reach once the desktop runs on Daytona. Boot the shared agent mock through the placement instead, and let a workload deliver its final reply as fixed-size content deltas with no tool steps, so the answer streams block by block on either lane.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 4 Skipped Deployments
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
![diff-warden[bot]](https://avatars.githubusercontent.com/u/118544361?s=60&v=4){kind=small}
There was a problem hiding this comment.
Warden security clearance: clear. No blocking findings (security or desktop↔den sync) in this diff (2f1a7f156a9127c7917a3f9496f626f3a5847b36). Automated clearance satisfies the required-review gate only — a human still reviews and merges. Analysis run
Deploying openwork with
|
| Latest commit: |
fdb0133
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://6a3b1eb0.openwork.pages.dev |
| Branch Preview URL: | https://perf-streaming-markdown-bloc.openwork.pages.dev |
There was a problem hiding this comment.
Warden security clearance: clear. No blocking findings (security or desktop↔den sync) in this diff (4ed6af240f55181475bb4a63aebb3076319c883a). Automated clearance satisfies the required-review gate only — a human still reviews and merges. Analysis run
Test evidence — a-streaming-answer-renders-as-markdown-block-by-block-and-settles-to-the-same-do — ✅ passed✅ passed · 0 expectations passed · 0 failed · 0 pending [world] mock(mcp) · den(daytona) · desktop(as admin) · workspace(/tmp/openwork-streamed-markdown-answer-1788555542549) · [seed:raw] evalIn() ×3 · session("New task") Test run created 2026-09-04T20:55:22.042Z · Source: |
benjaminshafii
left a comment
benjaminshafii
left a comment
There was a problem hiding this comment.
Reviewed incremental parsing/sanitization and the conflict resolution retaining both quiet-completion and chunked-answer mock behavior. The requested mid-stream assertions now use user.see/user.notSee. The final-commit Daytona journey passed (1 test, 0 failures, 0 skips), and matching evidence is published. Required CI checks pass.
2 participants
Summary
MarkdownBlocknow renders a streaming answer block by block: it keeps the previous frame's top-level marked tokens, re-lexes only the appended tail, and renders onedangerouslySetInnerHTMLpayload per block. A settled block keeps the same payload object, so React leaves its DOM alone and only the growing block repaints.Why
Measured on a 24-section streamed answer (12 KB markdown → 98 KB HTML) fed in 40-character deltas, parser + sanitizer only (no DOM work):
Per-frame cost is now flat regardless of answer length. On top of that, the old path also re-parsed the full HTML string into a fresh DOM subtree every frame (images reloaded, code blocks lost scroll position, layout invalidated for the whole message); the new path only touches the DOM of the block that changed.
Issue
Scope
apps/app/src/components/markdown/markdown-primitive.ts:createStreamingMarkdownRenderer()— incremental lexing of an appended source (re-lexes the last two blocks so lazy list continuation, setext underlines, and table delimiter rows still reshape their predecessor), per-block parse + sanitize, stable payload identity for unchanged blocks, full re-lex fallback for replaced text and for any source containing reference-style link definitions (those resolve across blocks).apps/app/src/components/markdown/markdown.tsx: use the block renderer whilestreaming, keep the whole-document render for settled text and history;useSelectionStableValueand the text-highlight / code-wrap / mermaid effects key off the committed render instead of an HTML string.apps/app/src/components/markdown/mermaid.ts:useMermaidEnhancertakes the rendered content identity as its dependency (was an HTML string; the only use was as a dependency key).evals/specs/streamed-markdown-answer.e2e.test.ts+evals/worlds/chat.tsstreamedMarkdown: the journey.scripts/mock-oauth-mcp-server.mjs+evals/packages/labs/src/mock-mcp.ts: an agent workload may deliver its final reply as fixed-size content deltas (finalReplyChunkSize) and may have no tool steps; both additive.Out of scope
MarkdownBlock(react-app/domains/session/surface/markdown.tsx) does not stream and is unchanged.Testing
Ran
OPENWORK_EVAL_REF=2f1a7f156a9127c7917a3f9496f626f3a5847b36 pnpm evals:e2e streamed-markdown-answer—placement: daytona (daytona CLI authenticated),checkout resolved 2f1a7f156a91pnpm --dir apps/app exec bun test --isolate tests/markdown-streaming-blocks.test.ts tests/markdown-selection-stability.test.ts tests/markdown-code-block.test.ts tests/markdown-mermaid.test.ts tests/markdown-math.test.tspnpm --dir apps/app typechecknode evals/scripts/spec-boundary-ratchet.mjs,node evals/scripts/spec-channel-ratchet.mjsnode --test scripts/mock-oauth-mcp-server.test.mjsResult
✓ a streaming answer renders as markdown block by block and settles to the same document— 1 passed, 0 failed, 0 skipped, exit 0, verdictpassed. Steps: finished blocks render as markdown while later blocks are still arriving (1.4 s); the settled answer shows every block exactly once and no markdown syntax (7.7 s); history renders the same document after a reload (2.0 s). Evidence is published in the test-evidence comment.markdown-streaming-blocks.test.tsproves every streamed frame (1-, 7-, and 40-character chunks) renders byte-identical HTML to the whole-document renderer over a document with setext headings, lazy and loose lists, nested lists, a table, fences and HTML blocks containing blank lines, math, and a blockquote; settled blocks keep payload identity; a growing tail can reshape its predecessor; reference definitions resolve on every frame; replaced text and CRLF input fall back to a full render that still reuses unchanged blocks;reset()drops retained state.pnpm --dir apps/app typecheckexit 0. Both spec ratchets green (75 specs / 50 e2e specs). Mock server tests pass.evalstsc -p .has pre-existing errors in unrelated files and none in the files touched here.CI status
Manual verification
Evidence
2f1a7f156).Risk
<div>s exist only while streaming. No stylesheet or spec uses a child combinator on.markdown-content; the DOM helpers (applyTextHighlights, code-wrap sync, mermaid) query from the root.Rollback