[miguel-heygen]

What

The project history no longer files a version of a file that a save had only half finished:

• A file an outside save has just emptied (opened for writing, nothing written yet) is left for the next sweep instead of being filed as an empty version. Once it has stayed empty for SETTLE_MS (100 ms), the next sweep records it, and a follow-up sweep is scheduled so that happens without another write.
• A file that changes while the history reads or copies it is read again, up to three times, and only bytes read while the file stood still are filed. A file still changing after the third read is left for the next sweep.
• A file still changing when the history first opens (or when it first takes in the media ledger) stays pending, the same way large media does while it is copied in the background. Its first whole copy becomes where its history starts, and it is never filed as an outside add. Retries back off from 100 ms to at most 30 s, and a close leaves the file named for the next open.

Why

A save that truncates a file and then writes it leaves the file empty for a moment. A sweep that lands in that moment, for example a flush or a claim of another file, filed the empty file as the file's new version. After the save finished and was claimed, undoing it brought back an empty file instead of the last whole version.

Repro on main, in package terms: open index.html with open(path, "w"), call history.flush() and history.claim(who, "x", []), write the content, close, claim index.html, then step back. The file reads '', and history.list() shows an outside entry whose new version is the empty file's hash (e3b0c442…).

Related work

Refs #5627, which lets the history copy a changing file without stalling its sweep. This PR uses the same approach: it reads again at once and never waits for the writer.

How

• "Gone" and "still changing" are different results: storeIfPresent returns the hash, "gone" or "changing". Every caller reads both:
  • observe: either one is skipped, and the next sweep looks again.
  • firstOpen, the media ledger, and background adoption: "gone" is left out as before, and "changing" stays pending in adopting until a whole copy lands.
  • storeBytes copies a private staged file and ignores the result.
• One owner of "is this file still being written": blobStore.ts exports SETTLE_MS and isSettling(stat, now). put() compares size, mtime and ctime before and after the read or the copy.
• The sweep decides: sweep() skips a file that is empty, changed within SETTLE_MS of the sweep, and not written inside an open window, and it leaves the tracked state alone. Three callers take every file as it is, so a deliberate empty save is still recorded at once:
  • a claim that names the path (the claimer is telling us its write is done)
  • a write an open window takes (that window's writer ends it, and the sweep that ends it records the write)
  • the sweep before an undo, redo, restore or new window (settle()), so an undo still sees the file as it is and refuses to write over it
• The bound: SETTLE_MS = 100. It has to be longer than the 20 ms burst a watcher gathers before it starts a sweep. Otherwise the sweep that a save's own first event triggers would always find the file older than the bound and file it. 100 ms adds at most that much delay before an emptied file is recorded. RACY_MS (2 s) is a separate rule: it decides when a stat can be trusted, not when a file can be filed.
• Known limits:
  • A save paused longer than 100 ms between truncating and writing is still filed empty.
  • A save paused with only its first bytes written (non-empty) is filed as it stands. A test pins this behaviour.
  • A same-size rewrite within one disk timestamp tick keeps the same stat, so the before/after read check cannot see it. Linux's coarse file clock ticks every few ms.
• Why empty only (measured): I applied the same wait to every just-changed outside file, not only empty ones. On the package's history suite, 25 tests fail (24 existing, plus the new half-written test). They all write a file and then immediately flush, claim, close or reopen, and expect that write recorded: outside entries logged before a claim, a held drag ending at an outside write, changes made while closed, entry order by write time. Under the general rule these writes are recorded late, out of order, or not at all, which breaks attribution and ordering. With the empty-only rule, 0 existing tests fail.
• Why windows are exempt (measured): without the exemption, an agent window that writes an empty file and closes, or is flushed, right away loses that write to a later outside entry. The test files a window's empty write in that window, ended at once by close or flush fails.

Test plan

• Unit tests added: blobStore.test.ts (torn small-file read, torn large-file copy, a file that never stops changing), and a new projectHistory.saving.test.ts (paused-writer repro, half-written save, emptied-for-good file recorded by the next sweep, claimed empty save, window's empty write, undo after a fresh empty).
  fails without the fix: projectHistory.saving.test.ts > files no empty version for a save paused between emptying the file and writing it

• A file that is still changing during every read at first open is not filed as an outside add, and undo leaves it in place (a file still being written when the history first opens). This test fails at the previous head of this PR, where such a file later showed up as an outside add whose undo deleted it.

• On main, 5 of the new tests fail: the 3 read tests, the paused-writer repro, and the emptied-for-good test. All pass on the branch.

• Breaking each piece in turn turns a test red:

  • read check off: 3 read tests fail
  • empty wait off: paused-writer and emptied-for-good fail
  • claim names nothing: claimed empty save fails
  • window exemption off: window test fails
  • undo sweep waits: undo test fails
  • no follow-up sweep: emptied-for-good fails

• Full @hyperframes/studio-server suite: 1649 passed, 2 skipped. tsc --noEmit and oxfmt --check are clean.

• Manual testing performed

• Comments follow CONTRIBUTING.md "Comments"

[github-actions]

Edit accuracy: accurate 267 (base branch 267), smooth 107 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

• seqplainkeys-none-center-r0-nested-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): pass, passes
• seqplainkeys-none-center-r0-root-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): pass, passes
• seqplainkeys-none-center-r30-root-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): not run
• seqplainkeys-none-pct-r0-root-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): fail / pass / pass, passes
• seqplainkeys-none-pct-r30-root-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): not run
• seqplainkeys-none-px-r0-nested-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): pass, passes
• seqplainkeys-none-px-r30-nested-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): not run
• seqplainkeys-none-px-r30-root-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): not run

Unstable (2)

• seqrepeat-none-px-r0-root-z100: tracking 0.05, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true
• seqplainkeys-none-pct-r0-nested-z100: tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true

[heygengenesis]

Changes requested at af0e7006. There are two blockers.

1. The body has no "fails without the fix:" line. This lane requires a fix PR to name a test that fails without the fix, on a line that starts fails without the fix:. The test plan says that "5 of the new tests fail" on main, but it never uses that line. The claim itself holds. I ran the new tests against main's blobStore.ts and projectHistory.ts, and exactly these 5 fail:

• files no empty version for a save paused between emptying the file and writing it
• records a file emptied for good by the next sweep, once it has been quiet a moment
• the three a file written to while the history reads it cases

Add a line such as fails without the fix: projectHistory.saving.test.ts › files no empty version for a save paused between emptying the file and writing it.

2. A file still changing when the history first opens is later filed as an outside add, and undoing that entry deletes the file. put() now returns null for a file that keeps changing. storeIfPresent passes that null on, and three callers read null as "the file is not there":

• firstOpen skips the file, so it is not in tracked or in the baseline.
• adopt → recordAdopted returns early and drops the file from adopting, so a large media file being written at open never joins the baseline.
• takeInUnnamedLedger leaves the ledger out of the baseline.

On the next sweep, observe finds no known hash and records before: null. The file therefore appears as an entry under "Changed outside the app" that adds a file which existed before the history opened.

I reproduced this with the real engine, stubbing only put() to return null for one file three times during openProjectHistory. That is the same result the new retry loop gives for a file that keeps changing. After flush(), list() shows [["Changed outside the app", [["media.txt", null, "<hash>"]]]]. undo of that entry returns ok: true, and media.txt is deleted. On main, put() never returned null for a file that was there, so the file went into the baseline, possibly with torn bytes but never as an add.

Possible fixes: keep "gone" separate from "still changing" in what storeIfPresent returns, and on the baseline paths (firstOpen, adopt, the ledger) either take the file as it stands or keep it pending, as adopting does, until it settles. Please add a test for it.

The rest looks right to me:

• The empty-only wait is used only by sweep(). Claims for named paths, writes taken by an open window, and settle() ("every") take the file as it stands.
• sweepIn(SETTLE_MS) reschedules after a waiting sweep, and it is a no-op when closed.
• observe treats a null from a moving file as "skip, try next sweep", and tracked stays as it was.
• The known limits are stated and pinned.
• At this head the history suite passes: 13 files, 268 passed, 2 skipped.

[heygengenesis]

Approved at 2abfcfc4. Both blockers from my review of af0e7006 are resolved.

• The "fails without the fix:" line is now in the body. It names projectHistory.saving.test.ts > files no empty version for a save paused between emptying the file and writing it, and that test fails on main, as I checked last round.
• A file still changing at first open is no longer filed as an outside add. storeIfPresent now returns {hash}, "gone" or "changing". firstOpen, the media ledger and recordAdopted send "changing" to adopting, which sweep() skips and which a START restore refuses while it is non-empty. adoptLater retries with backoff capped at 30 s, and adopting.json carries the path to the next open.
• The new test, is not filed as added outside: its first whole copy is where its history starts, fails red-first. It fails with this head's tests run against af0e7006's projectHistory.ts and passes at this head.
• The history suite passes at this head: 13 files, 269 passed, 2 skipped. CI, regression, Windows render verification, Comments and CodeQL are green on this head.

Not a blocker, and found by reading the code only: once a retry is scheduled, the promise for that path in adopting has already resolved. A claim() naming the path therefore no longer waits for the adoption. Its sweep skips the path, and the claimed bytes can end up in the baseline instead of in the claim. This only happens when a file is changing at open and is then claimed before it settles. Worth a follow-up, for example keeping the path's promise pending until the retry lands.