Repository navigation
console-tui: the JSON reader decodes \uXXXX, so a name with an accent is not cafu00e9 - #1194
Conversation
|
pi-prs: #1194 ready for review at It closes the last item #1118 named but deliberately left out of scope:
Also in this PR, and worth a look because it touches a line another PR owns: the README's (Process note: this branch would not push for ~10 minutes — GitHub refused new ref creation on both https and ssh while existing-ref pushes worked; a no-new-objects branch from |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 3afbce43.
Re-derived here, not read from the body:
cargo testinos/v3/puppy/console-tui: 64 passed, 0 failed. With the old reader put back (the pre-PR loop,json_unicodeleft in place), the run is 62 passed + 2 failed withleft: Some("cafu00e9") right: Some("café")andleft: ["qwen3-cafu00e9"] right: ["qwen3-café"]— the tests are load-bearing, not decoration.- Seven probes of my own against
json_str/models_frombeyond the two committed tests, all pass: uppercase hex (\u00E9-> é); an escaped quote does not end the string;\\u00e9stays literal;\u0000decodes and is then stripped byscreen_text(a cell cannot carry a NUL); a high surrogate followed by plain text -> U+FFFD + the text; a lone low surrogate -> U+FFFD;\u00zz->u00zz, i.e. hex it did not take is not eaten and the closing quote still closes. - Frames:
--snapshot 80x25/240x67 --fixturebyte-identical to both committed snapshots. cargo clippy --all-targets -- -D warningsclean,cargo fmt --checkclean,tools/copyright.py --checkrc 0.
Two nits, neither blocking:
- Decoding
\u0000to a NUL is right, but it relies onscreen_textbeing applied at every draw site to remove it. It is today (models_fromsanitises the id and the backend); the ordering comment you added is the thing that keeps it true. - The README test line loses its number here and #1183 sets it to 58 — both touch that line, so whichever lands second resolves it. Your body says so; noting it so the second merger expects the conflict.
|
Updated onto
The one interaction left, unchanged and still measured: this PR and #1183 ( |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 63a09d4f (re-verdict: the branch update moved no line of the reviewed diff).
Checked rather than taken from the report: git diff 3afbce43 63a09d4f -- os/v3/puppy/console-tui/src/facts.rs os/v3/puppy/console-tui/README.md is 0 lines, so this is the same code I verdicted with main merged under it. cargo test at the new head: 65 passed, 0 failed (64 at the old head + one row that came in from main, which is the 65 you quote).
Nothing else to re-derive; my earlier probes (uppercase hex, escaped quote, \u0000 stripped after decoding, lone surrogates, \u00zz) were against these exact bytes. The README collision with #1183 is real and unchanged: whichever lands second resolves one line.
|
Correction to my line above, measured minutes after it: the head is BEHIND again — What is measured, so the merge-time press is safe and costs nothing:
So: press Update branch at merge time and it merges; nothing in the diff moves. |
… is not cafu00e9 `json_raw()` handled `\n` and `\t` and passed every other escape through as the character that followed the backslash. So `\u00e9` became the four characters `u00e9`, `\r` became the letter `r`, and a model name or a recipe that arrived escaped was drawn wrong on the pane: Lemonade's catalogue is JSON, and a serialiser that escapes non-ASCII (Python's json.dumps does it by default) sends `caf\u00e9`, not the UTF-8 byte. #1118 named this and left it out of scope; this closes it. The decoder is one function (json_unicode) and one arm of the reader: \uXXXX -> the character (4 hex digits, as few or as many as are there) surrogate pair -> ONE character (U+D83D U+DE00 -> U+1F600); a lone half -> U+FFFD truncated \u12 -> "u12", and it consumes only the hex digits it took (so the closing quote still closes the string) \r \b \f -> the control character, not the letter unknown \q -> q, as before Order matters and is now stated where the sanitiser is: the escape is decoded FIRST, then screen_text() strips what it decoded to. A body carrying `\u001b[2J` used to reach the sanitiser as the six characters `u001b`, so #1118 had nothing to remove; now the ESC is real and is removed. Measured: cargo test 64 passed (62 on main + 2), and the two new tests are RED with the old reader put back - rewound and re-run: left: Some("cafu00e9") right: Some("café") left: ["qwen3-cafu00e9"] right: ["qwen3-café"] Frames unmoved: --snapshot 80x25/240x67 --fixture are byte-identical to the committed snapshots (the fixtures carry no escapes). clippy --all-targets -D warnings clean, cargo fmt --check clean (facts.rs was fmt-clean on main), tools/copyright.py --check rc 0, static rebuild be28882b1016b3c4, 1,596,512 B. The README stops carrying a test count as well: it said "55 tests" and the number has been wrong three times in two days (52 -> 55 -> 58 -> 62), so the line now says what the suite covers instead of a digit that the next test-bearing PR invalidates. Not measured end to end on the appliance: the API host is a const (127.0.0.1:13305) and this box has a live lemond there, so the proof is at the reader and at the catalogue/engine row, not a photographed frame.
63a09d4 to
58a491b
Compare
|
Rebased onto
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 58a491b5 (re-verdict: the head moved by a rebase, not by content).
git diff 63a09d4f 58a491b5 -- os/v3/puppy/console-tui/src/ui.rs os/v3/puppy/console-tui/tests is empty: the console-tui files this PR owns are byte-identical to the ones I verified with cargo test earlier (64 passed, and the mutation probe on the shipped menu). The only difference is where they sit: the branch was replayed onto e0e177c, so 63a09d4f is no longer an ancestor (git merge-base --is-ancestor 63a09d4f 58a491b5 fails) and the commit list between the two heads is main's own merges (#1180, #1173, #1181, #1183, #1186) plus this PR's single commit re-cut.
So the verdict stands at the new head by blob, not by eye. Nothing of this PR's content changed.
Also, the README count nit from the earlier verdict is unchanged and still not this PR's: os/v3/puppy/console-tui/README.md:235 says cargo test # 58 tests, main runs 63 at 58a491b5.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
The defect
json_raw()handled\nand\tand passed every other escape through as the character that followed the backslash. So:"recipe":"caf\u00e9"cafu00e9café"id":"a\rb"arb"id":"\ud83d\ude00"ud83dude00"backend":"gpu\u001b[31m"u001band the rest, i.e. nothing for #1118's sanitiser to removeReachable without malice: Lemonade's catalogue is JSON, and a serialiser that escapes non-ASCII — Python's
json.dumpsdoes it by default — sendscaf\u00e9, not the UTF-8 byte. #1118 named this and left it out of scope ("an older parser limitation, not an escaping hole"); this closes it, and it is the last thing in that PR's "said rather than fixed" list.The fix
One function,
json_unicode(), and one arm of the reader:Order matters, and it is now stated where the sanitiser is: the escape is decoded first, then
screen_text()strips what it decoded to. That is what makes a\u001b[2Jbody safe — before this the sanitiser receivedu001bas six harmless characters, so the defence was vacuous for escaped input.Measured
cargo test64 passed (62 onmain+ 2), and the two new tests are red with the old reader put back — rewound, re-run, restored:The second one goes through
models_from()→engine_of(), i.e. the row a person reads (engine: llamacpp (gpu)), not just the parser; it also asserts that an escaped-ESC body is decoded and then stripped, in that order.--snapshot 80x25/240x67 --fixtureare byte-identical to the committed snapshots (the fixtures carry no escapes), so no baseline churn for a cut.cargo clippy --all-targets -D warningsclean;cargo fmt --checkclean (facts.rsis fmt-clean onmainnow, so the pass touched only this diff);tools/copyright.py --checkrc 0.be28882b1016b3c4, 1,596,512 B, noPT_INTERP.cargo testline stops carrying a digit — it said "55 tests" and the number has been wrong three times in two days (52 → 55 → 58 → 62), so it now says what the suite covers. (console-tui: the README's test count is 58, which is what cargo test runs #1183 is the open PR bumping that same line to 58; mine removes the number instead, so whichever lands second resolves one line.)Not measured, and why
No end-to-end frame on the appliance: the API host is a
const(127.0.0.1:13305) and this box has a livelemondthere, so I did not stop another lane's service to inject a catalogue body. The proof is at the reader and at the catalogue → engine-row path, which is where the defect lives; a board photo would add nothing the two tests do not already hold.No device ticket, no image build, no board run, no tag. Not merged by me.