[aw-failures] BehaviorDefinedEngine.GetInstallationSteps skips AWF install for non-npm custom engines — 100% failure for Pydanti [Content truncated due to length] #51271

Emit the AWF-install step for every behavior-defined engine when network isolation is on — not just npm-installed ones

Fix BehaviorDefinedEngine.GetInstallationSteps now — it silently skips the install_awf_binary.sh step for any custom engine whose behavior.Installation is nil (and has no harness-script) or whose install.PackageManager != "npm". That's a hard, deterministic P0: the compiled lock file has no way to put awf on PATH, so the firewall wrapper call fails immediately with awf: command not found (exit 127) and the entire job dies before the agent CLI ever runs a single token.

Affected workflows and runs

• .github/workflows/smoke-pydantic.md / smoke-pydantic.lock.yml (engine: pydantic-ai, declared in .github/workflows/shared/pydantic.md with runtimes: uv, no installation block, no harness-script)
• §31231477754 — fails at "Execute Pydantic AI CLI": awf: command not found, exit 127
• §31229465314 — same root defect, fails one step earlier at "Predownload Pydantic AI CLI" (Failed to spawn: pai) because this run's lock file also predates the pip install pydantic-ai step; either way the run never reaches a working agent
• audit-diff between these two runs confirms zero token usage on both — the agent CLI never executed in either run
• Any future custom/behavior-defined engine declared with a non-npm installation.package-manager (pip, cargo, uv, etc.) or with no installation block and no harness script will hit the identical failure the moment network isolation/firewall is enabled — this is not Pydantic-AI-specific, it's a gap in the generic engine framework

Probable root cause

pkg/workflow/behavior_defined_engine.go:166-213:

func (e *BehaviorDefinedEngine) GetInstallationSteps(workflowData *WorkflowData) []GitHubActionStep {
	behavior := e.behavior()
	if behavior == nil {
		return nil
	}
	if workflowData != nil && workflowData.EngineConfig != nil && workflowData.EngineConfig.Command != "" {
		return nil
	}

	// Behavior-defined engines that execute via a harness script (e.g. Goose) run the
	// harness through Node.js, so Node.js (and, when the firewall is enabled, the AWF
	// binary) must always be installed even when no package-manager based installation
	// is declared for the engine's CLI itself.
	if behavior.Installation == nil {
		if behavior.HarnessScript == "" {
			return nil   // <-- BUG: no harness script AND no installation block => no AWF install, ever
		}
		return BuildNpmEngineInstallStepsWithAWF([]GitHubActionStep{GenerateNodeJsSetupStep()}, workflowData)
	}

	install := behavior.Installation
	if install.PackageManager != "npm" {
		return nil   // <-- BUG: any non-npm package manager => no AWF install, ever
	}
	...
	return BuildNpmEngineInstallStepsWithAWF(npmSteps, workflowData)
}

The AWF binary install is bundled inside BuildNpmEngineInstallStepsWithAWF, which only ever gets called from the npm/harness-script branches. Every other path (return nil at lines 169, 181, 188) drops the AWF install entirely, with no fallback. Confirmed empirically:

$ grep -c install_awf_binary .github/workflows/smoke-copilot.lock.yml   # npm-based engine
3
$ grep -c install_awf_binary .github/workflows/smoke-pydantic.lock.yml  # uv-based custom engine
0

Proposed remediation

Factor the AWF-binary install out of BuildNpmEngineInstallStepsWithAWF into its own step-builder (it's already effectively generateAWFInstallationStep + npm-specific wrapping — see pkg/workflow/copilot_engine_installation.go:415), and call it unconditionally from GetInstallationSteps whenever network isolation is enabled for the workflow, appending it after whatever package-manager-specific install steps (npm, pip/uv, or none) the engine declares. Concretely:

1. Extract the generateAWFInstallationStep(...) call out of BuildNpmEngineInstallStepsWithAWF so it can be composed with non-npm install steps too.
2. In GetInstallationSteps, always append the AWF step (respecting the existing --rootless / custom-command skip logic) regardless of which branch produced the package-manager install steps, including the Installation == nil && HarnessScript == "" branch.
3. Regenerate .lock.yml files for all behavior-defined/custom engines and confirm install_awf_binary.sh now appears in each one that has network isolation enabled.

Success criteria / verification

• smoke-pydantic.lock.yml contains an install_awf_binary.sh step before the "Execute Pydantic AI CLI" step.
• A manual/scheduled re-run of "Smoke Pydantic AI" completes with agent job conclusion success and non-zero token usage (verifiable via audit-diff against §31231477754).
• Add a compiler test asserting GetInstallationSteps includes an AWF-install step for a synthetic behavior-defined engine with PackageManager: "pip" (or no Installation at all) when network isolation is on — to prevent regression for any future non-npm custom engine.

Filed as a sub-task of this run's 6h Failure Investigation parent report issue (same run window, 2026-08-08 ~01:00 UTC).

Generated by 🔍 [aw] Failure Investigator (6h) · agent · 191.8 AIC · ⌖ 55.2 AIC · ⊞ 5.5K · ◷

• expires on Aug 14, 2026, 5:22 PM UTC-08:00