Skip to content

fix(sync-workspace): migrate pre-wsId flat branches instead of silently failing - #1463

Merged
qingyun-wu merged 1 commit into
staging-workspace-revampfrom
fix/sync-workspace-flat-to-wsid-migration
Jun 5, 2026
Merged

qingyun-wu merged 1 commit into
staging-workspace-revampfrom
fix/sync-workspace-flat-to-wsid-migration

Conversation

@qingyun-wu

Copy link
Copy Markdown
Collaborator

Problem

#1459 switched vault branches from flat host/<host> to nested host/<host>/<wsId>. A leftover flat branch — local or on the vault — is a leaf ref that DIRECTORY/FILE-conflicts with the nested ref: git cannot create refs/{heads,remotes/origin}/host/<host>/<wsId> while a ref named .../host/<host> exists (cannot lock ref ... exists).

Two coupled faults made this fail silently:

  1. Swallowed error — the pull-side git checkout -B ... 2>&1 | tee -a "$LOG" >/dev/null reports tee's exit status, not git's. When the D/F conflict aborted the checkout, the script kept running on the wrong branch, reported success, and pushed nothing.
  2. No migration path — nothing retired the pre-wsId flat branch, so every host that synced before feat(sync-workspace): per-workspace branch identity via wsId (host/<host>/<wsId>) #1459 hits the collision on its first post-feat(sync-workspace): per-workspace branch identity via wsId (host/<host>/<wsId>) #1459 sync (reproduced live on this host; the Pro host will hit it next).

Fix

  • _migrate_flat_branch helper:
    • Local: carries the flat branch's history into the wsId branch (delete-flat-first to clear the local D/F conflict, then seed from the saved SHA), leaving HEAD on the wsId branch.
    • Vault: retires the remote flat branch (delete-then-push — git refuses to create the nested ref while the flat leaf exists, even with --atomic, verified). Gated on an ancestor check so unmerged history is never dropped, and pushes the wsId branch explicitly since the bidirectional push step skips a clean tree.
  • Pull-side checkout -B now captures the real exit status the set -e-safe way (|| rc=$?) and dies loudly on genuine failure instead of swallowing it.
  • Test 26 covers the full flat→wsId migration (local + vault).

Idempotent: a no-op once no flat ref remains.

Test

bash tests/sync-workspace.test.sh → 75/75 pass (69 prior + 6 new assertions). Verified idempotent no-op on an already-migrated workspace.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cold review (full diff + own-repo-grep for sibling patterns).

Direction: 👍

The bug-class identification — pull-side checkout -B ... 2>&1 | tee | tee >/dev/null masking a git failure — is real and dangerous. The fix shape (|| _co_rc=$? + die) is the right pattern: explicit exit capture is robust independent of set -o pipefail/set -e subtleties, which famously can fail to abort in contexts that look unconditional. The migration helper is well-considered: idempotent, delete-then-push to navigate the D/F constraint, ancestor-check before remote retire, HEAD-detached before delete.

Suggestions (non-blocking)

1. Sibling swallow-patterns: lines 586 (fetch) and 686 (reset)

The same git X 2>&1 | tee -a "$LOG" >/dev/null shape exists at:

  • L586: git fetch --all --quiet 2>&1 | tee -a "$LOG" >/dev/null — fetch failure → migration runs against stale peer view + checkout falls through to local-only.
  • L686: git reset --hard "$pre_pull_sha" 2>&1 | tee -a "$LOG" >/dev/null — this one is the rollback action in the mass-deletion tripwire path. A failed reset there leaves the workspace in a half-rolled-back state while the script continues.

Both deserve the same || rc=$? + diagnose treatment. (Lines 554 and 739 are if-guarded so OK.) Per feedback_per_file_pattern_audit: same file, same class — worth one pass.

2. Local-side ancestor check would mirror the remote-side caution

_migrate_flat_branch already gates the remote delete on git merge-base --is-ancestor "$remote_flat_sha" "refs/heads/${wsid_branch}" to never drop unmerged history. The local half just deletes the flat branch and conditionally seeds — if a local wsId branch already exists AND is BEHIND the local flat branch's tip (partial prior migration, divergent local edits), _need_seed=0 means flat content is dropped from local without check.

Realistic-only-in-edge-cases, but the cheap defense is symmetric:

if [ "$_need_seed" -eq 1 ]; then
    git branch "$wsid_branch" "$flat_sha" ...
elif ! git merge-base --is-ancestor "$flat_sha" "refs/heads/${wsid_branch}"; then
    die "wsId migration: local flat $flat_branch ($flat_sha) not ancestor of $wsid_branch; resolve manually"
fi

3. Test 26 strength + 2 cheap additions

The happy path is well-covered. Two cheap additions would harden:

  • Idempotency: run --push-only (or full sync) a second time; assert no migrating local flat branch message + HEAD stays put. The PR description claims idempotent — worth proving in CI.
  • Safety branch (non-ancestor remote): setup where the vault flat branch carries commits NOT in the local wsId branch. Assert the script warns + leaves the flat alone (the NOT deleting remote flat origin/<flat> log path).

Approve

Overall a high-quality fix for a high-impact bug (silent push-nothing is the kind of thing that erodes trust in sync). The 3 above are tightening, not blocking. Setting state to APPROVE.

— qingyun-sutando

…ly failing

`host/<host>/<wsId>`. A leftover flat branch (local or on the vault) is a leaf
ref that DIRECTORY/FILE-conflicts with the nested ref, so `git checkout -B`
(pull side) and the push both fail with "cannot lock ref ... exists". The
pull-side checkout error was swallowed by `... 2>&1 | tee >/dev/null` (pipe exit
status reflects tee, not git), so the script ran on the WRONG branch, reported
success, and pushed nothing — exactly how this host's first post-#1459 sync
silently no-op'd.

Fix:
- Add `_migrate_flat_branch`: carries the flat branch's history into the wsId
  branch (delete-flat-first to clear the local D/F conflict, then seed from the
  saved SHA), and retires the flat branch on the vault (delete-then-push, since
  git refuses the nested-ref creation while the flat leaf ref exists — even with
  `--atomic`). Remote retirement is gated on an ancestor check so unmerged
  history is never dropped, and the wsId branch is pushed explicitly because the
  bidirectional push step skips a clean tree.
- Capture the pull-side `checkout -B` exit status the set-e-safe way
  (`|| rc=$?`) and `die` loudly on genuine failure instead of swallowing it.
- Test 26 covers the full flat->wsId migration (local + vault).

Idempotent: a no-op once no flat ref remains. All 75 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@qingyun-wu
qingyun-wu force-pushed the fix/sync-workspace-flat-to-wsid-migration branch from 720dfc7 to c877ee1 Compare June 5, 2026 02:08
@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

@cla-assistant check

@sonichi

sonichi commented Jun 5, 2026

Copy link
Copy Markdown
Owner

Cold review (Pro): LGTM. The double-fault diagnosis is correct — tee masking git exit status is a classic silent-failure trap, and the D/F conflict on upgrade is the natural consequence of the nested branch rename. The migration path (detect flat host/host → rename to host/host/wsId → prune remote flat ref) is correct. 4/4 CI green.

@qingyun-wu
qingyun-wu merged commit fbb2f75 into staging-workspace-revamp Jun 5, 2026
5 checks passed
@qingyun-wu
qingyun-wu deleted the fix/sync-workspace-flat-to-wsid-migration branch June 5, 2026 02:14
qingyun-wu added a commit that referenced this pull request Jun 6, 2026
…ase-process.md; delete proposals

Part 2 of the v0.3.0 docs sweep (PR #1472):

- **RENAME** `docs/release-process-consolidated.md` → `docs/release-process.md`. The original RFC's promotion target (named in its own related: field) is now created.

- **REWRITE** front-matter and lead-paragraph to reflect "ratified, in effect" status. The release infrastructure (CHANGELOG-PENDING, tagged releases v0.1.0+, per-PR gates) shipped via various PRs on the way to v0.1.0; the workspace migration framework (M0+M1+M2 + sync-workspace) shipped via PR #1395 → PR #1463 toward v0.3.0. Drops the "plan-only; nothing implemented" framing.

- **DELETE** `docs/release-process-proposal-mini.md` and `docs/release-process-proposal-qingyun-sutando.md`. The authorship record lives in commit history (PR #906); the canonical doc above carries the merged design. Owner-directed cleanup.

Net diff: −611 lines, +6 lines (the new header).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
qingyun-wu added a commit that referenced this pull request Jun 6, 2026
…1467 thread + Lucy's Maddy report)

Part 3 of the v0.3.0 docs sweep (PR #1472).

Adds two substantive sections to `docs/workspace-sync.md`:

**1. Migration risks + pre-flight checklist** — distilled from:
- sonichi's "Migration" thread on 2026-06-05 (Mini + Pro + Sutando risk analyses)
- Lucy's Maddy-host migration run report 2026-06-06 #design

Covers:
- Pre-migration checks per host (vault URL set, $SUTANDO_WORKSPACE
  removed from shell rc AND .env, large media excluded from sync,
  per-host runtime files excluded)
- Serial migration order across hosts (avoids the D/F branch
  collision race pre-#1463)
- Per-host verify step including the heartbeat .alive cleanliness
  check Mini surfaced
- Known issues with workarounds (Claude memory not auto-moved,
  sync-workspace plain run silent-commits, $SUTANDO_WORKSPACE
  re-migrate loop)

**2. Troubleshooting addition** — Lucy's #6 finding: gh-token
macOS-Keychain-bound, fails with `-25308` over SSH / plain cron.
Documents the launchd-plist-scoped-to-GUI-session workaround.

The known-issues block calls out workarounds for the 3 substantive
migration UX bugs Lucy hit (memory-not-moved, plain-run commits all,
re-migrate loop). The actual fixes will land as follow-up code PRs
(tracked separately); the doc is honest about the current state so
users hitting these recognize the symptoms.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
qingyun-wu added a commit that referenced this pull request Jun 6, 2026
…resh [DRAFT]

Release-prep PR for v0.3.0 — the staging-workspace-revamp rollup.

**CHANGELOG.md:**
- Add v0.3.0 entry with thematic subsections under Added/Fixed/Changed
- Dedicated "Workspace contract rollup (M0 + M1 + M2 + sync-workspace)"
  subsection covering the 20 staging-revamp PRs (#1395 → #1463)
- Other sections cover non-revamp work since v0.1.0: voice/multimodal,
  bridges, skills, reliability, developer experience / CI
- Skips pure-doc PRs per Keep-a-Changelog convention

**KNOWN_ISSUES.md:**
- Rewrite entry #1 (workspace contract migration): M1 migration script
  is shipped, not "upcoming" — point users at `bash scripts/sutando-migrate.sh`
  + describe the 30-day grace window for `--delete-source` cleanup
- Entries #2-4 (task-status flicker, Gemini hallucination, Gemini Live
  15min idle timeout) unchanged — still apply, marked "by design"

**docs/memory-sync.md:**
- Setup step 2: prefer `vault.remote_url` in config / `--vault-url` flag
  over legacy `SUTANDO_MEMORY_REPO=` in `.env`
- "What gets synced" split into "New path (sync-workspace.sh)" and
  "Legacy path (sync-memory.sh)" tables — reflects PR #1445's
  workspace-as-git-repo architecture
- "What does NOT get synced" adds `vault.sync.include`/`vault.sync.exclude`
  customization callout citing PR #1447 + PR #1460

Wording is DRAFT — owner reviews before any of this lands. Once promoted,
this entry covers everything tagged as v0.3.0; CHANGELOG-PENDING.md
on this branch remains clean for v0.3.1 work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
qingyun-wu added a commit that referenced this pull request Jun 6, 2026
…ase-process.md; delete proposals

Part 2 of the v0.3.0 docs sweep (PR #1472):

- **RENAME** `docs/release-process-consolidated.md` → `docs/release-process.md`. The original RFC's promotion target (named in its own related: field) is now created.

- **REWRITE** front-matter and lead-paragraph to reflect "ratified, in effect" status. The release infrastructure (CHANGELOG-PENDING, tagged releases v0.1.0+, per-PR gates) shipped via various PRs on the way to v0.1.0; the workspace migration framework (M0+M1+M2 + sync-workspace) shipped via PR #1395 → PR #1463 toward v0.3.0. Drops the "plan-only; nothing implemented" framing.

- **DELETE** `docs/release-process-proposal-mini.md` and `docs/release-process-proposal-qingyun-sutando.md`. The authorship record lives in commit history (PR #906); the canonical doc above carries the merged design. Owner-directed cleanup.

Net diff: −611 lines, +6 lines (the new header).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
qingyun-wu added a commit that referenced this pull request Jun 6, 2026
…1467 thread + Lucy's Maddy report)

Part 3 of the v0.3.0 docs sweep (PR #1472).

Adds two substantive sections to `docs/workspace-sync.md`:

**1. Migration risks + pre-flight checklist** — distilled from:
- sonichi's "Migration" thread on 2026-06-05 (Mini + Pro + Sutando risk analyses)
- Lucy's Maddy-host migration run report 2026-06-06 #design

Covers:
- Pre-migration checks per host (vault URL set, $SUTANDO_WORKSPACE
  removed from shell rc AND .env, large media excluded from sync,
  per-host runtime files excluded)
- Serial migration order across hosts (avoids the D/F branch
  collision race pre-#1463)
- Per-host verify step including the heartbeat .alive cleanliness
  check Mini surfaced
- Known issues with workarounds (Claude memory not auto-moved,
  sync-workspace plain run silent-commits, $SUTANDO_WORKSPACE
  re-migrate loop)

**2. Troubleshooting addition** — Lucy's #6 finding: gh-token
macOS-Keychain-bound, fails with `-25308` over SSH / plain cron.
Documents the launchd-plist-scoped-to-GUI-session workaround.

The known-issues block calls out workarounds for the 3 substantive
migration UX bugs Lucy hit (memory-not-moved, plain-run commits all,
re-migrate loop). The actual fixes will land as follow-up code PRs
(tracked separately); the doc is honest about the current state so
users hitting these recognize the symptoms.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@sonichi sonichi left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The two coupled bugs (tee masking git exit code + no migration path) are both fixed correctly. The D/F conflict root cause is well understood and the fix sequence is right: capture flat_sha, detach HEAD, delete flat ref, seed wsId branch from flat_sha. Ancestor check before remote retire prevents dropping unmerged history. The 'brief window where vault has neither ref' comment accurately describes the safety argument. 4/4 CI green. Stand: Echo Act IV Pro

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants