Skip to content

fix: preserve non-ASCII bytes when stripping compiler ANSI - #2879

Open
r3wretrhy wants to merge 1 commit into
mozilla:mainfrom
r3wretrhy:fix/preserve-non-ascii-compiler-output-2864
Open

r3wretrhy wants to merge 1 commit into
mozilla:mainfrom
r3wretrhy:fix/preserve-non-ascii-compiler-output-2864

Conversation

@r3wretrhy

Copy link
Copy Markdown

When stdout/stderr is not a terminal (or ColorMode::Off), sccache strips ANSI from cached compiler output before writing it back. That path used strip_ansi_escapes::Writer, which parses through VTE and drops any byte that is not valid UTF-8 ground-state input — including Latin-1 octets in #warning text, C1 controls, and bare \r.

So a plain gcc diagnostic like caf\xe9 survives, but the same bytes through sccache become caf with the 0xe9 gone. Repro from the issue (non-TTY stderr):

printf '#warning caf\xe9 na\xefve\nint f(void){ return 1; }\n' > lat1.c
gcc -c -Wall lat1.c -o /dev/null 2> bare.txt
sccache gcc -c -Wall lat1.c -o /dev/null 2> sccache.txt
cmp bare.txt sccache.txt

This replaces the VTE-based stripper with a small byte-wise CSI/OSC walker that only removes escape sequences and copies every other octet unchanged. ColorMode::On / TTY paths still write the original bytes.

Fixes #2864.

strip-ansi-escapes routes output through VTE, which drops Latin-1 and
other non-UTF-8 octets in diagnostics when stdout is not a terminal.
Replace it with a byte-wise CSI/OSC stripper so ColorMode::Off still
keeps the compiler's raw bytes.

Fixes mozilla#2864.
@codecov-commenter

codecov-commenter commented Oct 4, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 59.74026% with 31 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.71%. Comparing base (50b3296) to head (7dd9487).

Files with missing lines Patch % Lines
src/commands.rs 59.74% 31 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2879      +/-   ##
==========================================
+ Coverage   73.34%   76.71%   +3.36%     
==========================================
  Files          72       72              
  Lines       40776    40884     +108     
==========================================
+ Hits        29909    31363    +1454     
+ Misses      10867     9521    -1346     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compiler output loses every non-ASCII byte when stdout is not a terminal

2 participants