Repository navigation
epic(cluster): observability, backup & recovery #1090
Description
Observability, backup & recovery
Purpose
The cluster can be seen, backed up, and proven restorable - including after a power cut.
Scope
Metrics/logging (Cozystack ships VictoriaMetrics+Grafana); etcd + TALM config recovery vault (encrypted snapshots offsite, e.g. R2); restore drills that actually restore Postgres and etcd on a schedule; power-return boot order (dependency-sequenced bring-up) documented and tested; SMART drive-health watch feeding the storage records from epic 5.
Execution mode
Online (drills may schedule a physical power test with Drew).
Dependencies
Depends on epics 7 and 8.
Acceptance
Dashboards live; at least one full restore drill passed and recorded; power-return procedure written and exercised once.
Structure
Part of the basement cluster program. Project home: planning/roadmaps/basement-cluster.md in this repo. Full program register: goal workspace files/idea-register-2026-10-20.md (Muse goal goal_ba17a5777c21). Detailed task issues are filed as sub-issues of this epic; dependencies use blocks / blocked-by links.
Hard rules for the whole program: no HashiCorp anything; everything bootstrappable from config, no hand-built state.
Activity
developing-today commentedon Oct 27, 2026
CODE-230
dezren39 commentedon Oct 27, 2026
Epic map — basement cluster program
This epic: #1090 — Observability, backup & recovery
Dependencies: Blocked by #1087, #1088
Filed in the 2026-10-26 cluster seed. Project home:
planning/roadmaps/basement-cluster.md.docs(basement-cluster): link filed epic issues in roadmap
7 remaining items
dezren39 commentedon Nov 3, 2026
🤖 Agent note · handed off
ses_e5e5df8a4b7ffeumc74xfO2tt6repo-worker-code(bound worker uses this id for all queue helper verbs)opencode-route/workhorse, agentbuilddezren39 commentedon Nov 3, 2026
agent-note:v1 — task 7634 handed off (Lane A worker, 2026-11-03): execution lane is opencode with a bound session (ses_e5e5df8a4b7ffeumc74xfO2tt6) that already completed the triage work — verdict still_live_dependency_blocked (blocked by #1087 and #1088), with its triage note prepared at ~/git/fleet/notes/7634-1090-triage-note.md on the laptop after earlier posting attempts hit the GitHub secondary rate limit. The bound session posts that note and completes task 7634 as triaged; this lane does not work it directly.
dezren39 commentedon Nov 3, 2026
agent-note:v1 — triage (queue task 7634)
state: triaged — still live, correctly dependency-blocked, no children filed yet
Verdict at current main (
0467ae0f9, 2026-11-03): this epic is still live and its scope has not changed. The roadmap still carries it under "Later" as proposed (planning/roadmaps/basement-cluster.md:32), and no cluster observability/backup implementation exists in the tree yet (no VictoriaMetrics/Grafana config, no etcd/TALM snapshot job, no restore-drill assets). Nothing here is stale, superseded, or already handled, so the issue stays open.Dependencies (revalidated against fresh GitHub state):
blocked_byedges are present and correct: epic(cluster): Cozystack platform #1087 (Cozystack platform, open) and epic(cluster): platform services #1088 (platform services, open, itselfstatus/blocked).Children: none filed — the sub-issue list is empty and I found no closed children from earlier passes. That is correct for now: the child tasks (dashboards, vault, drills, boot order, SMART watch) only become concrete once Cozystack's bundled VictoriaMetrics/Grafana and the CNPG/RustFS choices from #1087/#1088 exist.
Fix path / ordering once unblocked:
mode/online).Unblock condition: #1087 and #1088 reach done / Cozystack + CNPG are running. At that point file the five child tasks above as sub-issues of this epic and schedule the restore drill and the physical power test with Drew.
Coverage: leave open; re-check when the blocked-by epics close. No blocker escalation is needed — this is normal program ordering, not a decision for Drew.