Skip to content

[awf] docker-manager: MCP gateway fails on ARC self-hosted runners — DOCKER_HOST not propagated, hardcoded group-add #2267

Closed

Description

@lpcox

Problem

When running on ARC (actions-runner-controller) self-hosted runners using the Docker-in-Docker sidecar pattern on Kubernetes/EKS, the MCP gateway fails to start with:

[WARN] Invalid container ID format: arc-gaw-xzpj8-runner-8lthc
[WARN] Could not determine container ID
Error: Docker socket not found at /var/run/docker.sock

The agent workflow aborts before any model/tool work is performed.

Context

Root Cause

Three independent bugs surface together on ARC/Kubernetes runners:

1. DOCKER_HOST not propagated to gateway container (src/docker-manager.ts getLocalDockerEnv())
The docker run command generated to launch the gateway does not pass -e DOCKER_HOST. So when the runner exports DOCKER_HOST=unix:///var/run/docker.sock (required for dind sidecar), the gateway always falls back to /var/run/docker.sock hardcoded. Additionally, getLocalDockerEnv() intentionally strips TCP DOCKER_HOST values to force a local Unix socket, which prevents TCP-only dind setups from working at all.

2. Hardcoded --group-add 0 in the generated docker run invocation for the gateway
The fix in #26750/#26771 (v0.68.6) hardcoded --group-add 0 to match the Docker socket's GID on GitHub-hosted runners. On ARC/dind setups, the socket is created with a custom GID (e.g. 123 via DOCKER_GROUP_GID), so the hardcoded 0 is ineffective and the socket is inaccessible.

3. Container-ID regex in gh-aw-mcpg/run_containerized.sh rejects Kubernetes cgroup names
The regex ^[a-f0-9]{12,64}$ only matches Docker container hashes. On containerd/EKS, /proc/self/cgroup contains the K8s pod name (e.g. arc-gaw-xzpj8-runner-8lthc), which fails the check. This causes a [WARN] that, combined with bug #1 and #2, leads to failure.

Proposed Solution

In src/docker-manager.ts:

  1. Propagate DOCKER_HOST: Add -e DOCKER_HOST to the gateway docker run arguments when DOCKER_HOST is set in the runner environment. Remove the intentional stripping of TCP DOCKER_HOST values (or make it opt-in).
  2. Auto-detect Docker socket GID: At startup, run stat -c '%g' "$socket_path" to get the actual GID of the Docker socket and use that value for --group-add instead of hardcoding 0.

In gh-aw-mcpg/run_containerized.sh:
3. Make container-ID detection non-fatal for Kubernetes cgroup formats: Treat parse failures as informational only; do not let a [WARN] for unrecognized cgroup format influence the socket-detection logic downstream.

Documentation: Add a guide page for running gh-aw on ARC self-hosted runners (dind sidecar pattern, DOCKER_HOST propagation, socket GID requirements).

Generated by Firewall Issue Dispatcher · ● 726.9K · ◷

Activity

  1. lpcox commented on Apr 28, 2026

    @lpcox
    CollaboratorAuthor

    Closing — the three bugs identified are primarily in the gh-aw (compiler/runtime) and mcpg (MCP gateway) codebases, not in gh-aw-firewall:

    1. DOCKER_HOST not propagated to gateway container — The gateway is launched by gh-aw (not this repo). AWF's getLocalDockerEnv() intentionally strips TCP DOCKER_HOST for its own Docker operations (Squid/agent containers need the local daemon), but correctly passes DOCKER_HOST through to the agent container (see comment at line 80-81). This is upstream.

    2. Hardcoded --group-add 0 — This is in the gh-aw gateway launch code, not in gh-aw-firewall.

    3. Container-ID regex — This is in gh-aw-mcpg/run_containerized.sh, not in this repo.

    Additionally, PRs #1909, #1943, and #1924 (all merged) already improved DOCKER_HOST handling in this repo for various edge cases.

    The upstream issue gh-aw#28888 is the correct place to track these fixes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions