Skip to content

laptop-backups: durable run record for pm-dashboard-backup + a freshness watchdog that names a stopped backup (design #1024, phase 3) #1841

Closed

Description

@dezren39

Relocated from dezren39/darwin-nix#1026 on 2026-10-28 (placement audit): laptop-backups run-record/watchdog — the implementation is the code-side port (backup modules relocated to code via code PR #1740 / nix PR #1043; implementation branch queue/1914-1026-run-record-watchdog). Original body below; issue references inside it are historical (dezren39/nix numbering).

Problem

The two jobs declared in modules/laptop-backups.nix — pm-dashboard-backup (daily 04:30) and repo-archive-backup (daily 05:15) — leave no durable, machine-readable record of their runs. A run's only traces are its journal lines and pm-dashboard-backup's .last-run-epoch, which records the journal window, not run health. #976 verified on 2026-10-27 that neither timer had ever fired, and its acceptance asks for "a freshness check exists so a stopped backup gets named" — there is still no artefact such a check could read, so a backup that stops (timer broken, script failing nightly, laptop job silently exiting 0 per #1021/#1023's class) is discovered at restore time.

Part of design #1024 ("a backup that can prove itself"), phase 3. Parent coverage issue: #976.

Scope

  • modules/pm-dashboard-backup.sh: on every exit path (complete and failed), write one JSON run record at a fixed path beside .last-run-epoch (e.g. $ROOT/.last-run.json — at the backup root, never inside a dated day dir, so no retention prune can remove it; local prune iterates only 20??-??-?? dirs). Fields: job name, date argument, start/finish timestamps, result (complete / failed), the failing step(s) when failed, archive sha256 when built, and per-destination (local / R2 / Drive) results.
  • modules/laptop-backups.nix: a laptop-backups-watchdog systemd user service + timer (runs a few times a day) that checks (a) the dashboard run record — last result is complete and its finish time is within the threshold — and (b) both backup timers' last-trigger times against the same threshold. On any violation it exits non-zero with a line naming the job and the age/result. The threshold is a module option (services.laptop-backups.watchdog.maxAgeHours or similar), default a little over one daily period (26h). The watchdog changes nothing and attempts no repair.
  • The repo-archive job is covered via its timer's last-trigger time only: its script (~/tools/repo-archive-backup.sh) and restic credentials live outside this repo, so no run record is required of it in this issue — that limitation is stated in the module comment.

Acceptance criteria

  • A complete pm-dashboard-backup run leaves .last-run.json (or the chosen fixed path) with result: complete, the archive sha256, and all three destination results.
  • A failed run (e.g. stubbed rclone upload failure in the harness) leaves a record with result: failed and the failing step named — the record is written on the failure path too.
  • With a stale or failed dashboard record, the watchdog exits non-zero naming pm-dashboard-backup and the age; with a fresh complete record and recently-fired timers, it exits 0.
  • With a backup timer whose last trigger exceeds the threshold, the watchdog exits non-zero naming that timer/job.
  • Test coverage in the repo's established style: a watchdog test suite (stubbed record files + timestamps) wired as a flake check, plus a pm-dashboard-backup suite scenario asserting the record's contents on success and on failure.
  • Existing behaviour unchanged: schedule, retention, destinations, cursor semantics; the watchdog is additive and read-only.

🤖 Agent note · state: filed

Activity

  1. developing-today commented on Oct 28, 2026

    @developing-today
    Contributor
  2. dezren39 commented on Oct 28, 2026

    @dezren39
    MemberAuthor

    History carried over from dezren39/darwin-nix#1026 (verbatim, oldest first):


    @dezren39 · 2026-10-28T14:50:55Z

    🤖 Agent note · triage confirmed


    @dezren39 · 2026-10-28T14:52:16Z

    🤖 Agent note · parked on sequencing


    @dezren39 · 2026-10-28T16:40:44Z

    🤖 Agent note · state: parked (CI pending)


    @dezren39 · 2026-10-28T17:37:15Z

    🤖 Agent note · blocked — target moved out of this repo

  3. added
    task/small-fixQueue packet kind: bounded fix with reproduction and targeted tests
    priority/normalOrdinary priority (the default)
    area/nixNix expressions, flakes, and packaging
    area/infraInfrastructure, deployment, and operations
    effort/mMedium: expected under 1 day of work
    queue/queuedHas a queued task in the Supabase agent queue
    on Oct 28, 2026
  4. added
    work/assignedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Oct 29, 2026
  5. 64 remaining items

  6. added
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    queue/claimedQueue worker lifecycle state
    and removed
    queue/claimedQueue worker lifecycle state
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Oct 31, 2026
  7. dezren39 commented on Oct 31, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  8. added a commit that references this issue on Oct 31, 2026
  9. dezren39 commented on Oct 31, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

    • Agent: queue worker (repo-worker-code), lane B
    • Session / ID: queue task 3057, lane B shift 2026-10-31 06:44 CDT
    • Found via: queue claim (resume from park)
    • Task: PR 2989 merged — laptop-backups run record (.last-run.json on every exit path) + freshness watchdog (service/timer, maxAgeHours default 26), ported from dezren39/nix branch queue/1914-1026-run-record-watchdog @ 55c636f, superseding nix task 1914.
    • Expectation: met — CI green (mcpx-docs build pass on head 0fd4f55), Codex clean on current head with 0 unresolved threads (all prior Codex/CodeRabbit findings dispositioned in-thread), merge commit 661582d.
    • Links: PR nixos/home: laptop-backups run record + freshness watchdog (#1841) #2989 (merged)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/infraInfrastructure, deployment, and operationsarea/nixNix expressions, flakes, and packagingeffort/mMedium: expected under 1 day of workkind/enhancementpriority/normalOrdinary priority (the default)source/design-architectFiled by the repo design architect jobtask/small-fixQueue packet kind: bounded fix with reproduction and targeted tests

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions