Skip to content

fix(ops): hub-native fleet-status collection on erlich (#1657) - #1659

Draft
aarontrowbridge wants to merge 1 commit into
mainfrom
opencode/1657-fleet-status-hub-native
Draft

aarontrowbridge wants to merge 1 commit into
mainfrom
opencode/1657-fleet-status-hub-native

Conversation

@aarontrowbridge

Copy link
Copy Markdown
Member

Closes #1657.

Each probe rewritten hub-native, verified by running the collector on the hub against ground truth:

AC Evidence
erich row = this machine, members ssh ok erlich/this machine + macbook,mini/ssh ok
server_guard ok with real pid + canonical DB ok, pid 2804749 = the port-4094 engine listener, db opencode.db
chat_db real values 2057 sessions = direct python3 sqlite count
vault_sync real age or explicit state state "no completed cycle in the log", clean false (the log genuinely errored out Aug 30)

Two drive-by fixes the verification run exposed (both mini-era latent bugs, both noted in the commit):

  • the ahead/behind split emitted an empty token → invalid JSON whenever a repo had a live upstream
  • the served-sessions shortfall heuristic compared a paginated, directory-filtered endpoint against a raw table count → false guard alarms; the channel-flip incident it targets is caught by the pid+db check

Also: OUT is overridable via FLEET_STATUS_OUT for testing (used for this verification).

Deploy note: the live cadence keeps running the old script until this merges and the hub copy refreshes.

The four mini-era probes all lied on the erlich hub; each is now hub-native:
this machine names itself from the fleet config's sshAlias instead of a
hardcoded first entry (erlich was labeled 'mini'); the server guard reads
the engine's /proc environment (pid holding OPENCODE_DB, no OPENCODE_PID)
instead of launchd — the DB-channel-flip incident it exists for is caught
by the pid+db check, which correctly finds the live engine; the chat DB is
read via python3 against the pinned canonical name (no sqlite3 CLI on the
hub); sync-log parsing is portable and reports an explicit state instead
of nulls from BSD date flags.

Two drive-by fixes surfaced by the verify run: the inherited ahead/behind
parameter-expansion split emitted an empty token (invalid JSON) whenever
a repo had a live upstream, now awk-split; the served-sessions shortfall
heuristic compared a paginated, directory-filtered endpoint against a raw
table count and false-alarmed the guard — served is now informational.

Verified on the hub (fresh collection): erlich=this machine + both members
ssh ok; guard ok with the real engine pid (matches the 4094 listener) and
opencode.db; chat_db 2057 sessions = direct DB count; vault_sync explicit
state; valid JSON, 57 repo rows.
@coderabbitai

coderabbitai Bot commented Oct 1, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(ops): fleet-status collector reports mini-era values on the erlich hub

1 participant