Summary
The Visual Regression workflow has failed consistently since run #2252, even on unrelated pull requests. Only the extension-host-visual job fails; webview-visual and theme-fixtures continue to pass.
The failure is not a container or rendering regression. The chat-dark Electron snapshot includes the live context-token counter. PR #1505 changed the generated system prompt and reduced the deterministic test request from 4.2k / 1.0m to 3.1k / 1.0m. Those changed digits account for the 10-pixel snapshot difference reported on all three Playwright retries.
Evidence
The workflow container is pinned to an immutable Playwright 1.62.1 digest, and successful runs immediately before the cutoff used the same image.
Expected behavior
Prompt-only changes that alter token usage should not cause the visual-regression workflow to fail when the rendered layout and styling remain correct.
Suggested fix
Short term: inspect the extension-host-visual-regression artifact and accept the intentional electron-chat-dark-sidebar.png baseline change.
Long term: normalize or mask the context-token value before capturing the sidebar screenshot, while continuing to snapshot the surrounding progress UI. This prevents future system-prompt changes from creating unrelated visual-regression failures.
Summary
The
Visual Regressionworkflow has failed consistently since run #2252, even on unrelated pull requests. Only theextension-host-visualjob fails;webview-visualandtheme-fixturescontinue to pass.The failure is not a container or rendering regression. The
chat-darkElectron snapshot includes the live context-token counter. PR #1505 changed the generated system prompt and reduced the deterministic test request from4.2k / 1.0mto3.1k / 1.0m. Those changed digits account for the 10-pixel snapshot difference reported on all three Playwright retries.Evidence
apps/vscode-e2e/src/visual/electron.visual.ts:241apps/vscode-e2e/src/visual/__screenshots__/electron-chat-dark-sidebar.pngThe workflow container is pinned to an immutable Playwright 1.62.1 digest, and successful runs immediately before the cutoff used the same image.
Expected behavior
Prompt-only changes that alter token usage should not cause the visual-regression workflow to fail when the rendered layout and styling remain correct.
Suggested fix
Short term: inspect the
extension-host-visual-regressionartifact and accept the intentionalelectron-chat-dark-sidebar.pngbaseline change.Long term: normalize or mask the context-token value before capturing the sidebar screenshot, while continuing to snapshot the surrounding progress UI. This prevents future system-prompt changes from creating unrelated visual-regression failures.