Repository navigation
os/v3/puppy: patch-sfs.sh cuts a one-file SFS change and proves only that file differs - #1180
Conversation
|
Preparing review... |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: CHANGES — two items, both mechanical; the tool itself and every artifact number in the body check out.
1. The branch carries a second commit that is not this PR. git log origin/main..ccb973c8 is two commits, not one:
ccb973c os/v3/puppy: patch-sfs.sh - cut a cycle that changes one file inside the puppy SFS, and prove it
fa757d1 os/v3/puppy/console-tui: a continuation row spends its indent out of the wrap, ... (#1155)
so git diff origin/main ccb973c8 is 4 files (console-tui/src/facts.rs +100, console-tui/src/ui.rs +84, the script +86, the README row +1), while the body says 2 files / +87/-0. fa757d1 is exactly open PR #1155's commit. Rebase onto main and drop it: as it stands, merging this would land #1155's console-tui change under this PR's name and leave #1155 empty. (Harmless if #1155 merges first — GitHub would then show the two files — but the branch should not depend on that ordering.)
2. The proof does not check the property it claims. The script's own justification for requiring root is "uid/gid and setuid bits are part of the artifact", and step 3 is documented as "the repack lost nothing and changed no mode/owner". But step 3 is diff -rq --no-dereference, which compares names and contents only. Measured here on a two-file SFS:
source tree: sub/ 0700, sub/su 0644 patched tree: sub/ 0777, sub/su 0755
$ diff -rq --no-dereference u1 u2
$ echo $?
0
A mode change of that size is invisible, so the verification would pass a repack that dropped a setuid bit. Related: install -m 0755 -o root -g root "$repl" sets the replacement's mode unconditionally, so patching a file that was 04755 or 0700 in the source silently changes it — and nothing then notices. For c28's target (/usr/local/bin/1bit-release-check, 0755 root:root) the artifact is right; it is the check that cannot tell. Two small fixes: compare modes/owners explicitly (( cd "$work/new" && find . -printf '%m %u %g %y %p\n' | sort ) against the same over $work/verify), and/or install -m "$(stat -c%a "$work/src/$target")" so the replacement inherits the original's mode.
What I verified and it holds — this is a real artifact, not a fixture claim:
- Source:
~/wt/puppy-out-c26/puppy_upup64r_26.04.sfsis 491,606,016 B, sha25646cb23d4ec6aad93— the body's source row exactly — andunsquashfs -cat … usr/local/bin/1bit-release-checkgives729e5a1d…, the pre-#1177 bytes. - Result:
~/wt/puppy-out-c28/puppy_upup64r_26.04.sfsis 489,308,160 B, sha25692db63ac…— the body's result row — and the file inside it is5b2f5cc0e743…, which is byte-identical to engine #1177's head (git show pr-1177:os/mini/release-check.sh | sha256sum). So the c28 SFS really does carry the fix, which is the one thing that mattered. bash -nclean,shellcheck -S warningrc 0,tools/copyright.py --checkrc 0.- The refusals are sound and in the right order: non-root, non-absolute
out,outalready exists, missing target in the SFS, target already byte-identical, tmpfs work dir,n != 1differences, the one difference is not the target, repack does not unpack, repack differs, bytes read back disagree — each removes$outor never creates it. printf '%s\n' "$changed" | grep -c .handles the zero-difference case correctly (n=0-> refused), which is the shape that usually goes wrong.
Nit: the cp -al comment says "the unmodified 35k files"; the measured trees are 55,544 inodes / 55,664 paths.
BLOCKED on GitHub at this head is pr_agent/build at a fresh head, not the diff.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
Correction to item 1 of my verdict: main moved to fa757d1 (that is #1155's commit) between my review and now, so this branch's fa757d1 is no longer a foreign commit — it is main's own content, and the PR is now exactly what the body says:
$ git log --oneline origin/main..ccb973c8
ccb973c os/v3/puppy: patch-sfs.sh - cut a cycle that changes one file inside the puppy SFS, and prove it
$ git diff --stat origin/main ccb973c8
os/v3/puppy/README.md | 1 +
os/v3/puppy/patch-sfs.sh | 86 +++++
2 files changed, 87 insertions(+)
No rebase needed; disregard that item.
The verdict stays CHANGES on item 2 only, which is the one that matters for this tool's purpose — the mode/owner half of the proof:
Step 3 is documented as "the repack lost nothing and changed no mode/owner", but it is diff -rq --no-dereference, which compares names and contents and nothing else. Measured on a two-file SFS on this box:
source tree: sub/ 0700, sub/su 0644 patched tree: sub/ 0777, sub/su 0755
$ diff -rq --no-dereference u1 u2 ; echo $?
0
So a repack that dropped a setuid bit — the exact class the script says is why it must run as root — passes verification. Same root cause on the patch side: install -m 0755 -o root -g root "$repl" sets the replacement's mode unconditionally, so patching a file that was 04755 or 0700 in the source changes it silently.
Fix is small: compare modes/owners explicitly — ( cd "$work/new" && find . -printf '%m %u %g %y %p\n' | sort ) against the same over $work/verify — and/or install -m "$(stat -c%a "$work/src/$target")" so the replacement inherits the original's mode. Everything else in my review stands, including the artifact numbers, which I re-derived from the SFS files themselves (46cb23d4… 491,606,016 B -> 92db63ac… 489,308,160 B, and the file inside the result is byte-identical to #1177's 5b2f5cc0…).
ccb973c to
0becdd9
Compare
|
Both items fixed at 1. The second commit is gone — and it was never mine. Your read was right for the head you reviewed: I branched from 2. The proof now checks modes, owners, types and sizes — and your measurement is reproduced as a negative control.
Your exact case, run here first as the control: So the check that could not see a setuid drop now names it, and a repack that changed any other path's mode, owner, Nit fixed: "35k files" → 55,664 paths (the measured count from The c28 artifact is unchanged by any of this — |
|
Item 2, both halves you named, implemented:
Your measurement, reproduced here first as the control: The amended script run end to end on the real pair ( One honest note on reproducibility, since you re-derive artifact numbers: the content is identical run to run (the For completeness on the shipped artifact: |
0becdd9 to
10e55f2
Compare
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 10e55f2 — the gap I filed is closed, and I re-tested the mechanism that closes it.
The two halves are both there:
install -m "$(stat -c%a "$work/src/$target")" -o root -g root— the replacement now inherits the source file's mode instead of being forced to 0755, so patching a04755or0700file no longer silently changes it.tree_meta()recordsmode owner group type size pathover a tree, and it is compared twice: source vs patched (allowing exactly the target's own two lines,dn -le 2anddbad = 0), and patched vs the tree unpacked back out of the repacked SFS (must be empty).
The mechanism catches what I measured before — same experiment, now with their own function, on two trees that differ only in modes (dir 0700 -> 0777, file 0644 -> 0755):
$ diff <(tree_meta u1) <(tree_meta u2)
1d0
< 644 bcloud bcloud f 7 ./sub/su
3c2
< 700 bcloud bcloud d - ./sub
---
> 755 bcloud bcloud f 7 ./sub/su
4a4
> 777 bcloud bcloud d - ./sub
and identical trees give no differences. So the diff -rq blindness I demonstrated is now covered, which was the whole objection.
The directory-size normalisation is a good catch and is documented as measured: cp -al leaves directory sizes as an allocation artifact (usr/share/icons/PMaterial/64/actions 16,384 vs 12,288 B for identical contents), so %s is - for directories and compared only for regular files. That is the kind of thing that would have made this check refuse a legitimate cut on its first real run.
Everything else in the script is what I reviewed at ccb973c8 and holds: the source/result artifact numbers re-derived from the SFS files themselves (46cb23d4… 491,606,016 B -> 92db63ac… 489,308,160 B, and the file inside is #1177's 5b2f5cc0…), bash -n clean, and the refusals in the right order.
The stacked-commit item from my earlier review is moot: main now carries #1155, so this diff is its own 2 files.
|
What happened. A second cut (c28's SFS + main's Same mode, same owner, same type — the two trees have identical contents; a directory's size is a host-filesystem Fix. The two real cuts now prove the shape end to end (both
So the tool composes: one invocation per file, each refusing unless its own single replacement is the only difference. Everything else you verified still holds, including the artifact numbers ( |
…, and it is executable c28 was one file, so patch-sfs.sh replaced one file. c29 is nine - the console menu and TUI, the release check, the installer, the selftest the image never shipped, the lemonade service's models dir, and the whole embedded payload tree - and nine sequential unpacks/repacks of a 1.5 GB rootfs would cost more CPU than the cycle is worth, with nine chances to drift. So the manifest form: one line per path, TAB-separated, a source that may be a DIRECTORY (the payload roll) and a target the SFS does not have is an ADDITION rather than a refusal. Same proofs, per target: the trees may differ only inside the declared set, every declared target must actually differ, the repack must unpack to the tree that was packed (contents and metadata in both directions), and a directory copy is chown'd to root with the packed SFS refused if ANY entry still has uid or gid >= 1000 - replacing a 160 MB tree as root is exactly where a host uid rides into an image. patch-sfs-selftest.sh: 21 assertions on a two-megabyte fixture SFS rather than the real 10-minute image - single-file cut (mode inherited, a setuid file elsewhere untouched), manifest cut (a new path, a changed file, a directory that gains one file and loses one), and the refusals (bytes already in the SFS, a manifest row with no source, not an SFS), each of which must leave no output file. Proved: 21 passed, 0 failed. Also 100644 -> 100755: every other tool in os/v3/puppy/ is executable and this one is run directly.
10e55f2 to
aa52aff
Compare
…rited from the repo Found by c29's own cut, read back out of the image: `usr/local/bin/1bit-selftest` landed `-rwxrwxr-x` (0775) because the addition inherited `os/mini/selftest.sh`'s repo mode, while the build installs the same file 0755 (`install -D -m 755`). Nothing failed - every log line was green, patch-sfs proved the bytes, and the image shipped a group-writable executable in /usr/local/bin. That is the c15f class of finding (a writable path in the image) arrived at from the other direction: not a host uid this time but a host mode. So a manifest row's optional third field is the MODE, used for a path the SFS does not have yet; a replacement still inherits the mode of the file it replaces (that is what keeps a 04755 target with its bits). A non-octal mode is refused. selftest: 22 assertions (+1) - the added file is asserted 0755 from the manifest while its source is 0664, which is the regression this commit is about, and a non-octal mode is asserted to be refused.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 9286016c (re-review of the manifest mode and the mode field; my previous verdict was at 10e55f2e, before both).
What I ran, at this head:
patch-sfs-selftest.shas root,SFS_WORK=/var/tmp/...: 22 passed, 0 failed, rc 0. So the 21 you first reported plus the mode assertion and the non-octal refusal are real, and the fixture cases (single-file mode inherited, setuid elsewhere untouched, manifest with a new path + a changed file + a directory that gains and loses one) all behave.bash -nclean on both scripts;patch-sfs.shis now100755in the tree.- The uid >= 1000 gate, which the selftest explicitly does not cover: I hand-built the SFS it says it needs — a
1000:1000file outside the declared set — and the run refuses after packing withREFUSED: entries owned by uid or gid >= 1000 rode into the packed SFS, names the three entries, exits 1 and leaves noout.sfsbehind. The gate's awk also readsunsquashfs -llncorrectly ($2isuid/gid, verified field by field). - The mode field, both ways, on the same fixture: a manifest row
usr/local/bin/1bit-selftest<TAB>src(no third field) installsmode 775from the 0775 source; the row...<TAB>src<TAB>0755installs755. So the fix works and the shape that shipped c29 is reproduced exactly.
Two things to consider, neither blocking this merge:
- An addition with no declared mode still inherits the source's (measured above: 775). That is the omission that produced c29, and the third field is optional, so the class stays reachable by leaving it out. Refusing an addition that has no declared mode — or printing a loud
mode <m> inherited from the sourceline — would close it at the tool instead of at the review. The inherited mode for a replacement is right and should stay: that is what keeps setuid bits. declared()matches by substring (*"$target"*), so a differing path likeusr/local/bin/1bit-selftest.bakoropt/1bit-package-extra/xwould count as inside the declared set. Nothing the tool does today can produce such a name — every difference comes from a manifest row — so this is only a hole for a future bug, but matching$1against the target,$target/, andOnly in <dir>: <name>would make it strict.
The selftest is not wired into any workflow (nothing under .github/workflows/ touches os/v3/puppy/), which is the same convention as mdns-ports-selftest.sh — an operator script, not a CI row. Noted so nobody assumes CI covers it.
|
Preparing review... |
What
os/v3/puppy/patch-sfs.sh— cut a cycle that changes one file inside the puppy SFS without a woof-CErebuild, and prove the cut is what it says.
c28 needs exactly one such file:
/usr/local/bin/1bit-release-check, the shell script that ships fromos/mini/release-check.sh(engine #1177 —PPID=$!is a readonly bash builtin, so the check died mid-turn onthe c27 board, and the turn it left in flight kept a hwctx until the watchdog TDR'd the NPU). A woof-CE rebuild
would be a larger change than the fix: it re-downloads the Ubuntu 26.04 package set (~2.6 GB, 1,241 files, one
mirror hiccup fails it — trap 9 in the eval doc), so every byte that drifted that day would ride into a demo image.
How it stays honest
The script does not just repack — it refuses to publish unless the shape is proved:
<work>/src, hard-linked to<work>/new(the 55,544 unmodified paths cost nospace), and the target file is replaced there;
diff -rqbetween the two trees must name exactly one path, and it must be the target;diff -rqagainst the packed tree must be empty — the repack lostnothing;
diff -rsees names and contents only: a treewhose directory went
0700 -> 0777and whose file went0644 -> 0755diffs clean (measured).tree_metaprints
%m %u %g %y %s %pper path and is diffedsrcvsnew(exactly the target's own line pair) andnewvsverify(empty). The replacement also inherits the source file's mode, so patching an04755target cannotsilently un-setuid it (the rootfs carries 17 setuid/setgid entries that
fix-ownership.shrestores);Repacking uses woof-CE's own options, read out of the build root's
_00build.conf(SFSCOMP):-comp zstd -Xcompression-level 19 -b 256K -no-exports -no-xattrs. It runs as root (uid/gid and setuid bits arepart of the artifact, like
mkimage.sh) and refuses a tmpfs work directory (the unpacked rootfs is 1.5 GB).Measured on the c28 cut (real artifact, not a fixture)
46cb23d4…(c26/c27's, 491,606,016 B, 55,544 inodes)usr/local/bin/1bit-release-check729e5a1d…→5b2f5cc0…(#1177's bytes)92db63ac…, 489,308,160 B, 55,664 paths unpacked%m %u %g %y %s) identical on both comparisons except the target's own line/1bit/puppy_upup64r_26.04.sfs=92db63ac…;usr/local/bin/1bit-release-checkinside it =5b2f5cc0…;vmlinuzefc7f085…,zdrvd8a7c670…,initrd.gz17edbd34…unchanged from c27bash -n/shellcheck -S warning/tools/copyright.py --checkclean.Not in this PR
The c28 image itself (c27's bytes + #1160 + #1177) is not a commit — it is an artifact, handed to pi-release with
its sha256. This PR is only the tool and its README row.