Skip to content

feat(ui): render Mermaid diagrams locally - #898

Draft
dajiaohuang wants to merge 14 commits into
alphaXiv:mainfrom
dajiaohuang:feat/430-mermaid-previews
Draft

dajiaohuang wants to merge 14 commits into
alphaXiv:mainfrom
dajiaohuang:feat/430-mermaid-previews

Conversation

@dajiaohuang

@dajiaohuang dajiaohuang commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Closes #430

Mermaid fences and standalone diagram files can be rendered locally with Pele, including local images and math. Invalid or oversized diagrams fall back to source. Rendering runs in a worker, and diagram links use the existing file navigation where available.

This draft addresses the mechanical review feedback: standalone artifact diagrams, a loading placeholder instead of the timed source flash, a compact source control, chart theme tokens, shared native-path and anchor handling, and normalized Windows paths. The separate XML parser optimization has been removed. The app-wide media changes were split out; Markdown network images now follow the shared behavior merged in #909. Pele is needed to render the supported Mermaid families locally without contacting a network service.

Human visual review and screenshots for the revised output are still outstanding; this remains a draft. The loading placeholder has not been proven to eliminate all layout shift. Diagram-family coverage, accessibility, real application flows, performance, and non-Windows behavior remain incomplete. The current automated browser captures are synthetic fixtures rather than visual acceptance.

At a00b8b0 on Windows, direct TypeScript checking passed; 35 focused Mermaid and Markdown target tests passed; a Chrome fixture passed for chat and artifact Markdown links without remote image requests. git diff --check passed. These checks do not cover the full UI suite, all app surfaces, or other platforms. Earlier performance and full-suite observations apply to previous heads and do not validate this revision.

Assisted-by: gpt-6.1-terra

Assisted-by: gpt-6.1-terra
@dajiaohuang
dajiaohuang marked this pull request as ready for review November 5, 2026 09:11
@greptile-apps

greptile-apps Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The PR changes how images and diagrams appear across several UI reading views.

The PR appears safe to merge, with three non-blocking issues in diagram image resolution.

Findings

  1. P2 Local destinations lose fragments ▶
  2. P2 Available images do not load ▶
  3. P2 Changed charts stay stale ▶
Fix with agent prompt
### Issue 1
ui/src/mermaidImages.ts:18-19
`resolveMermaidImages` turns `file:///tmp/chart.svg#plot` into the raw file URL without `#plot`. SVG images can therefore show the wrong view, and local links lose their selected section. Ordinary Markdown images preserve this through `chatImageTarget`. Preserve the fragment when building the diagram URL too.

### Issue 2
ui/src/mermaidImages.ts:19
`resolveMermaidImages` uses only the first image candidate. In chat, `artifacts/chart.png` can exist in the session checkout rather than the artifact store; `chatImageTargets` supplies fallback locations for that case. The diagram requests only the artifact-store URL, leaving a broken image when a later candidate exists. Try the remaining candidates when the first image cannot load.

### Issue 3
ui/src/components/ArtifactsTab.tsx:154-156
Artifact images inside diagrams do not refresh when the image file changes. `mermaidImages` returns an unversioned URL and stays unchanged when `entries` changes. With unchanged Markdown source, `MermaidBlock` keeps the same SVG and image URL. Ordinary artifact images already include the image entry's version. Include that version here and update the rendered image URLs when it changes.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds local Mermaid rendering for Markdown fences and standalone diagram files.

  • Mermaid diagrams render in Markdown and standalone diagram files.
  • Markdown shows local images and leaves network images out.
  • HTML previews limit automatic image loads to embedded assets.

dajiaohuang explicitly leaves broader preview coverage, full diagram coverage, performance, and cross-platform behavior for later review. Those acknowledged limits are not reported as findings.

No runtime tests or browser captures were performed during this review.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Markdown fence or diagram file] --> B[Measure math locally]
  B --> C[Pele worker]
  C --> D[Rewrite image and link URLs]
  D --> E[Render SVG]
  C -->|Error or timeout| F[Show source]
Loading

Reviews (1) · Last reviewed commit: "Resolve local images in unscoped Markdow..." · Reviewed by Greptile

Comment thread ui/src/mermaidImages.ts Outdated
Comment on lines +18 to +19
const native = nativeMarkdownPath(src);
const url = native ? absoluteFileUrl(native) : resolve?.(src)[0]?.url;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Local destinations lose fragments

resolveMermaidImages turns file:///tmp/chart.svg#plot into the raw file URL without #plot. SVG images can therefore show the wrong view, and local links lose their selected section. Ordinary Markdown images preserve this through chatImageTarget. Preserve the fragment when building the diagram URL too.

Prompt To Fix With AI
This is a comment left during a code review.
Path: ui/src/mermaidImages.ts
Line: 18-19

Comment:
**Local destinations lose fragments**

`resolveMermaidImages` turns `file:///tmp/chart.svg#plot` into the raw file URL without `#plot`. SVG images can therefore show the wrong view, and local links lose their selected section. Ordinary Markdown images preserve this through `chatImageTarget`. Preserve the fragment when building the diagram URL too.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment thread ui/src/mermaidImages.ts Outdated
}
if (isExternalMarkdownTarget(src) && !isWindowsDrivePath(src) && !nativeMarkdownPath(src)) continue;
const native = nativeMarkdownPath(src);
const url = native ? absoluteFileUrl(native) : resolve?.(src)[0]?.url;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Available images do not load

resolveMermaidImages uses only the first image candidate. In chat, artifacts/chart.png can exist in the session checkout rather than the artifact store; chatImageTargets supplies fallback locations for that case. The diagram requests only the artifact-store URL, leaving a broken image when a later candidate exists. Try the remaining candidates when the first image cannot load.

Prompt To Fix With AI
This is a comment left during a code review.
Path: ui/src/mermaidImages.ts
Line: 19

Comment:
**Available images do not load**

`resolveMermaidImages` uses only the first image candidate. In chat, `artifacts/chart.png` can exist in the session checkout rather than the artifact store; `chatImageTargets` supplies fallback locations for that case. The diagram requests only the artifact-store URL, leaving a broken image when a later candidate exists. Try the remaining candidates when the first image cannot load.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment thread ui/src/components/ArtifactsTab.tsx Outdated
Comment on lines +154 to +156
const target = artifactTarget(projectId, folder, src, artifactUrl);
return target ? [{ url: target.url, resolveAsset: () => null }] : [];
}, [absoluteFileUrl, artifactUrl, projectId, folder]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Changed charts stay stale

Artifact images inside diagrams do not refresh when the image file changes. mermaidImages returns an unversioned URL and stays unchanged when entries changes. With unchanged Markdown source, MermaidBlock keeps the same SVG and image URL. Ordinary artifact images already include the image entry's version. Include that version here and update the rendered image URLs when it changes.

Prompt To Fix With AI
This is a comment left during a code review.
Path: ui/src/components/ArtifactsTab.tsx
Line: 154-156

Comment:
**Changed charts stay stale**

Artifact images inside diagrams do not refresh when the image file changes. `mermaidImages` returns an unversioned URL and stays unchanged when `entries` changes. With unchanged Markdown source, `MermaidBlock` keeps the same SVG and image URL. Ordinary artifact images already include the image entry's version. Include that version here and update the rendered image URLs when it changes.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

@dajiaohuang
dajiaohuang force-pushed the feat/430-mermaid-previews branch from af88a58 to 1a12218 Compare November 5, 2026 09:28
@small-angel

Copy link
Copy Markdown
Contributor

comment written by Claude Opus 5.5

thanks for sticking with this one. pele was the right pick and the core works: we ran the patch locally and every diagram family rendered in light and dark, math and local images worked, external images stayed blocked, file links opened in a tab, and the checks pass.

it doesn't look good enough to merge though. we need the diagrams to look and feel good in the app, and that part needs a human to actually look at the output and make the taste calls. another agent pass won't get there. Clover doesn't have the time to make those calls for you, so until someone has gone over it by eye, this stays a draft. share screenshots on the next revision.

blocking, all of these:

  • a standalone .mmd renders in Files but shows only source in Artifacts. ArtifactsTab.tsx picks markdown or CodeView and has no mermaid case.
  • every diagram shows its source for at least 250ms, then jumps to the drawing. the layout shift is very visible in a long doc or a chat.
  • every diagram gets a full-size View source button above it. ten diagrams means ten buttons. find something quieter.
  • pie and xy charts use pele's default orange and blue instead of the app's theme tokens, so they don't match anything around them.
  • the file: / drive-letter check (/^file:/i.test(src) || /^[a-z]:[\\/]/i.test(src)) is copied four times across Md.tsx, ArtifactsTab.tsx and FileViewer.tsx. make it one helper.
  • the anchor lookup in MermaidBlock is pasted into four handlers (onClick, onDoubleClick, onAuxClick, onKeyDown).
  • drive letters get through by spreading ..."abcdefghijklmnopqrstuvwxyz" into imageSchemes and linkSchemes. can paths be normalized before render instead?
  • MermaidBlock landed between mdCodeComponents and its doc comment, so that comment now describes the wrong function.
  • @xmldom/xmldom adds a second XML parser just to trim a ~160ms stall on a 5,000-edge diagram. drop it unless it matters for diagrams people actually write.
  • the LibraryPicker cover removal and the HtmlPreview image CSP are app-wide media changes. they follow the issue thread, but split them into their own PR so they get reviewed on their own.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support mermaid diagrams

2 participants