Relay: many Claude Code subagents push the device state over the MQTT payload budget, and the whole device drops off the relay #172

Hi, and thanks for hcom. We ran into a relay failure that scales with the number of Claude Code subagents. Here's what we measured and some suggestions.

Environment

• hcom 0.7.30 (the latest release). On today's main, src/relay/push.rs has not changed since the tag.
• Claude Code 2.1.x, launched with hcom claude, relay enabled, Linux.
• The trigger is one session that orchestrates many workflow agents through Claude Code's Workflow tool. Each of those agents arrives at SubagentStart with agent_type: "workflow-subagent".

What happens

1. Every subagent becomes an instance row at SubagentStart. SubagentStart is registered with an empty matcher, so it fires for every agent type (claude.rs#L2613-L2632). handle_subagent_start then calls ensure_subagent_row, which allocates a dormant row (status_context = subagent:dormant) for each child (claude.rs#L653-L689, claude.rs#L2176-L2220). No setting turns this off. The only subagent setting is subagent_timeout (config.rs#L188).
2. build_state publishes every local row, whatever its status, with the full status_detail (push.rs#L222-L284). The query has no status filter, and detail goes out untruncated. For Claude, detail is the whole Bash command or the whole Agent/Task prompt (family.rs#L12-L51).
3. When the sealed state alone is over MAX_SEALED_PAYLOAD_BYTES (112 KiB), the whole publish fails. seal_bounded_payload can only drop or shrink events. After that it returns relay state exceeds safe MQTT payload budget of 114688 bytes (push.rs#L17-L21, push.rs#L23-L71). The worker logs relay.push_err and tries again on the next 5 s push interval, plus any inbound-triggered pushes (client.rs#L192, client.rs#L323-L327, client.rs#L691-L697).
4. Peers then drop the entire device. A peer refreshes relay_sync_time_<device> only when a state message arrives (pull.rs#L520-L526). After 90 s without one, cleanup_stale_remote_instances deletes every instance from that device (instance_lifecycle.rs#L57-L58, instance_lifecycle.rs#L993-L1027). So the device's main sessions disappear from other machines too, not only the subagents. They come back only once the local state fits the budget again (rows cleaned up, or long details replaced by shorter ones).

The problem also shows before the hard failure: when the state is close to the budget, seal_bounded_payload has to drop retained and new events to make each publish fit, so less event traffic goes out per publish.

Measured (one device, one day)

• The device registered about 870 subagent rows in one day, nearly all from one orchestrating session (all but two were workflow-subagent).
• More than 1,300 relay.push_err in one day, all of them the payload-budget error, in about a dozen streaks, and it keeps recurring while the workflows run. Several streaks lasted longer than the 90 s peer timeout with no successful publish (the longest about half an hour). During each of those, every session on the device was gone from its peers.
• Row sizes in the published state: 331-404 B for a row with an empty detail. In one snapshot the mean was about 720 B and the largest about 3.5 KB. At about 380 B per row, the 112 KiB budget is reached at roughly 300 rows even with every detail empty.
• Row size matters as much as row count: several streaks began and ended at the same row count (as seen in the peers' received state), so they were driven by long details (a long prompt or command is stored in full) rather than by new rows.

Related: #144 added the payload bound and event shrinking, but the state snapshot itself is unbounded. #99 made native children visible from SubagentStart, which is where the row count comes from. We found no existing issue for this.

Steps to reproduce (from our setup; we haven't built a minimal repro yet)

1. Two devices on one relay.
2. On device A, run hcom claude and have the session start a Workflow, or many Agent calls, that spawns a few hundred subagents. A few dozen subagents running multi-KB Bash commands or prompts should also trigger it.
3. Watch hcom status --logs on A for relay.push_err ... payload budget. On B, all of A's instances, including the parent session, disappear after about 90 s.

Suggestions

1. An opt-out for eager subagent registration. This could be a config key or env var (e.g. off/on), or an agent_type deny-list such as ["workflow-subagent"]. A skipped subagent should get no row and should not be treated as its parent if it runs hcom itself.

2. Cap detail in build_state. hcom list -v shows 60 characters (list.rs#L663-L671) and the TUI shows one line, so a char-boundary-safe cap of a few hundred bytes would lose nothing peers display. Dormant subagent rows could also be sent without detail.

3. Degrade gracefully instead of failing the whole publish. When the state does not fit:

   • drop details first;
   • then drop subagent rows (rows with a parent), and publish a count such as omitted_subagents: N so peers know they exist;
   • fail only if the top-level rows alone exceed the budget.

   That way the device's main sessions stay visible.

For now we work around it with a small wrapper around the hook command, on that one session's PATH only. It answers SubagentStart for that session's subagents without creating a row, denies hcom commands from those subagents (so they cannot act as the parent), and passes everything else through to hcom unchanged. We'd be glad to drop it once there's a built-in option, and we're happy to test a fix.