Skip to content

Agentic Fleet Setup — replace wizard with a skill-guided chat session #363

Closed

Description

@jeonghun-jj-lee

Agentic Fleet Setup — replace wizard with a skill-guided chat session

Important

Problem

The Create Fleet wizard (sequential QuickPick → SSH → build) fails ungracefully:
a 60-second timeout kills a 530 MB push with a cryptic ETIMEDOUT, missing tools
on the host are reported as a failure rather than handled, and the user gets no
real-time feedback. The setup requires human judgment at every failure point —
exactly what an agent does well.

Approach

Replace the wizard with a skill-guided Amicode chat session. The "Create Fleet"
button opens a new chat where the agent handles the entire fleet lifecycle
interactively: SSHs into the host, discovers problems, installs tools, pushes repos,
monitors builds, configures services, and reports progress — adapting in real-time
to failures just like a human operator would.

Approaches Considered

  1. Agentic skill in a chat session (chosen) — the agent SSHs, adapts, retries,
    explains. Full flexibility: can install missing tools, handle timeouts, switch
    strategies. Real-time feedback in the chat. Matches the pattern that worked in
    practice (this session).
  2. Fix the existing wizard (rejected) — bump timeouts, add more QuickPick steps
    for error recovery. Still sequential, still can't adapt. Every new failure mode
    needs a new code path. Doesn't scale.
  3. Terminal-based script (rejected) — run a bash script that SSHs and provisions.
    No adaptation, no user interaction, no explanation of what's happening or why.

Scope

In scope:

  • New fleet skill (packages/extension/skills/fleet/SKILL.md)
  • "Create Fleet" button → launches new chat session with fleet skill prompt
  • Skill covers full lifecycle: create, add machine, remove, dismantle, reconfigure
  • Skill documents: SSH patterns, tool installation, repo push, build monitoring,
    service configuration (launchd/systemd), fleet.json schema, validation
  • Remove fleet_wizard.ts (the agent does it all via bash)
  • Remove the QuickPick-based createFleet/addMachine/removeMachine/dismantle commands
  • Keep: fleet_panel.ts, fleet_sync.ts, fleet_profiles.ts, fleet_launch.ts,
    fleet_bridge.ts, fleet_host_settings.ts, fleet_fallback.ts

Deferred:

  • Auto-triggering the skill on first activation (panel button is the entry for now)
  • Fully headless (AFK) fleet setup (needs the agent to handle SSH key setup too)
  • Multi-client orchestration (adding N machines at once)

Assumptions

  • The agent has bash tool access (can run ssh target "command")
  • SSH key-based auth is already configured (the agent validates but doesn't set up keys)
  • The user is present in the chat to answer questions (SSH target, confirm actions)
  • The fleet skill is always available (extension-bundled, not vault-gated)

Acceptance Criteria

Skill

  • packages/extension/skills/fleet/SKILL.md exists with full lifecycle documentation
  • Skill covers: create fleet (dev clone mode + release mode), add machine, remove
    machine, dismantle, reconfigure, sync status
  • Skill documents the SSH command patterns for each operation (not pseudocode —
    the exact commands the agent should run)
  • Skill documents error handling: missing tools (install them), push timeout (retry
    with progress), build failure (read error, fix, retry), service won't start (check
    logs)
  • Skill documents the fleet.json schema, service file templates (launchd plist,
    systemd unit), and the tunnel plist template
  • Skill documents how to detect local repo paths (walk up from extension path /
    binary path to find .git)
  • Skill documents the validation checklist (SSH reachable, tools present, repos
    pushed, build succeeds, service running, tunnel connects)

Extension Changes

  • "Create Fleet" button in the panel → opens a new chat session with the fleet
    skill loaded and an initial prompt (e.g. "I want to set up a fleet")
  • The fleet_wizard.ts module is removed
  • The QuickPick-based commands (createFleet, addMachine, removeMachine, dismantle)
    are removed from extension.ts
  • The devBinaryPath, buildServerLaunchdPlist, buildServerSystemdUnit,
    buildTunnelLaunchdPlist, buildRemoteRebuildScript functions move into the skill
    as documented templates (not TypeScript — markdown code blocks the agent copies)
  • Panel "Add Machine", "Remove", "Dismantle" actions also launch chat sessions
    with the appropriate fleet skill prompt
  • After the agent completes setup, the panel refreshes (the agent tells the user
    to reload, or triggers it programmatically)

Preserved Functionality

  • fleet_sync.ts auto-sync on activation — unchanged
  • fleet_panel.ts topology/profiles/stats — unchanged (reads fleet.json, shows state)
  • fleet_profiles.ts CRUD — unchanged
  • fleet_launch.ts session launch — unchanged
  • fleet_bridge.ts bridge protocol — unchanged
  • fleet_fallback.ts fleet.json read/write — unchanged (the agent uses it via bash)
  • fleet_compat.ts version negotiation — unchanged

Key Decisions

Decision Rationale
Full lifecycle in one skill (not create-only) The same SSH + adapt pattern applies to add/remove/dismantle. One skill, not four.
Remove wizard code entirely The agent executes raw commands — keeping a library adds indirection the agent doesn't need. The skill's markdown IS the documentation.
Extension-bundled skill Fleet setup is core functionality, not an internal-only tool. Every user should have it.
New chat session per operation Fleet setup is a multi-minute operation with decisions. It deserves its own conversation, not a side effect of a button click.
Skill documents exact commands (not pseudocode) The agent copies and runs them. Pseudocode requires the agent to invent the implementation, which is where errors creep in.

Data Contracts

Fleet skill prompt (button → chat):

I want to create a fleet with a remote machine as the server.

Fleet skill operations (the skill documents these as sections):

  • /fleet create — full setup flow
  • /fleet add <target> — add a machine to existing fleet
  • /fleet remove <target> — remove a machine
  • /fleet dismantle — tear down the fleet
  • /fleet reconfigure — change host/port/SSH alias
  • /fleet status — show current state

Constraints & Invariants

  • The agent MUST validate SSH connectivity before any remote operation
  • The agent MUST NOT store SSH passwords or private keys
  • The agent MUST report each step's outcome before proceeding to the next
  • The agent MUST offer to install missing tools (not just report them missing)
  • Fleet.json writes MUST be atomic (tmp + rename) — the skill documents this
  • The canonical server MUST be a system service (launchd/systemd), not a child process
  • Sessions MUST survive client disconnect (the skill documents the service config)

Prior Art

  • This conversation (2026-08-13) — the manual setup that proved the agentic pattern
  • fleet_wizard.ts (being removed) — the sequential implementation this replaces
  • pulse-designer score — the existing guided-interview pattern (this is similar but
    uses bash execution instead of form-filling)
  • Device Inspector pattern — proven WebviewViewProvider (the panel stays)

Source

Part of #350 (Fleet Panel & Profile UI)
ADR 0005: Managed Fleet
ADR 0007: Fleet Panel and Profile UI

Activity

  1. 9 remaining items

  2. aarontrowbridge commented on Aug 27, 2026

    @aarontrowbridge
    Member

    Hygiene note 2026-08-27: This proposes replacing the wizard in #354 with a skill-guided chat session. Marking as related-to #354 — if the team adopts the skill approach, #354 can be closed as superseded; otherwise fold this into #354 as an alternative design. Leaving both open pending owner decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions