Skip to content

[awf] CLI: Add --diagnostic-logs flag to collect Docker container logs on failure #1928

Closed

Description

@lpcox

Problem

When AWF containers fail to start (e.g., Squid crashes during initialization in DinD environments), application-level logs (access.log, audit.jsonl) are never written because the service never became healthy. Diagnosing these failures requires asking users to manually add debug steps to their workflow.

A concrete example: a customer on ARC runners with DinD sidecars hit a Squid container crash (exit code 1) with empty access logs, requiring multiple support rounds. See github/gh-aw#18385.

Context

Root Cause

The AWF cleanup path (src/cli.ts catch blocks and signal handlers, src/docker-manager.ts cleanup function) does not run docker logs <container> on failure. Low-level Docker operational state (container stderr/stdout, exit codes, mount bindings) is discarded when containers are stopped.

Proposed Solution

Add --diagnostic-logs flag (already partially planned per the issue):

  1. src/types.ts: Add diagnosticLogs?: boolean to WrapperConfig.

  2. src/cli.ts: Add --diagnostic-logs option to the commander program and wire it into WrapperConfig. Note: --diagnostic-logs already appears at line 1554 in src/docker-manager.ts in the compose generation path — verify if this is already scaffolded and hook into the cleanup path.

  3. src/docker-manager.ts — performCleanup() / error handlers:

    • On non-zero exit with flag enabled, call docker logs --tail 200 awf-squid, awf-agent, awf-api-proxy, awf-iptables-init and write to \$\{workDir}/diagnostics/.
    • Run docker inspect --format '\{\{.State.ExitCode}}' and docker inspect --format '\{\{json .Mounts}}' for each container.
    • Emit a sanitized docker-compose.yml (redact env var values matching token|key|secret|password case-insensitively).
  4. Bundle into audit artifact: Write diagnostics to \$\{auditDir}/diagnostics/ when --audit-dir is set, else \$\{workDir}/diagnostics/.

  5. Graceful skipping: If a container no longer exists (already cleaned up), skip that container without error.

What NOT to collect

  • Raw environment variables (may contain API keys)
  • Full docker inspect output (contains env vars)
  • Host filesystem contents

Generated by Firewall Issue Dispatcher · ● 1.6M · ◷

Activity

  1. github-actions commented on Apr 12, 2026

    @github-actions
    Contributor

    👋 Hello! I noticed this issue might be a duplicate of an existing issue:

    These two issues appear very closely related:

    • Both describe the same root cause: AWF containers crashing before reaching healthy state (ARC + DinD + Squid exit code 1) leaving no diagnostic logs
    • Both reference the same upstream issues (github/gh-aw#18385, github/gh-aw/issues/25548)
    • Both propose the same solution: a --diagnostic-logs CLI flag with diagnosticLogs?: boolean in WrapperConfig, collecting docker logs + sanitized inspect output on non-zero exit

    Since #1895 was closed as completed, the feature may already be implemented. It's worth checking whether the work from #1895 covered the scope described here, or if this issue tracks additional work (e.g., the --audit-dir bundling path or the awf-diagnostic-logs frontmatter feature flag).

    This is an automated message from the issue duplication detector.

    Generated by Issue Duplication Detector for issue #1928 · ● 283K · ◷

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions