Repository navigation
Enable Atomic built-in tools and schedule the Atomic smoke test - #77020
Conversation
… its cooldown - Replace the read,bash,edit,write,mcp__* allowlist with --exclude-tools ask_user_question, so Atomic's default and bundled-extension tools (workflows, subagents, Intercom, web access, native MCP) are available; only the interactive question tool is excluded. - Run Smoke Atomic every two days as well as on manual dispatch. - Set engine.cooldown: 1 in the Atomic smoke and conformance workflows (the minimum strict mode allows), so a new pin installs the day after release instead of after three days.
| - --no-approve | ||
| - --tools | ||
| - read,bash,edit,write,mcp__* | ||
| - --exclude-tools |
There was a problem hiding this comment.
Agreed on the risk, addressed in cbf78863 without the suggested change. Allowing only ask_user_question would remove every tool the agent needs. The exclude list is now an explicit allowlist of exactly the reviewed tools: read,bash,edit,write,find,search,kill,todo,workflow,subagent,intercom,web_search,code_search,fetch_content,get_search_content,mcp__*. Tools added by a later Atomic release stay off until a reviewed bump lists them. Verified against the published 0.9.79: it offers exactly these 15 tools plus MCP, and ask_user_question is not among them.
There was a problem hiding this comment.
Update (cbc7349b): at the maintainer's direction the sample now runs Atomic with its full feature set, minus the interactive ask_user_question, so this is back to --exclude-tools ask_user_question, now with --approve. The exclude list is deliberate: repositories add custom tools and workflows by committing them to .atomic/, which an allowlist in the engine definition would block. .atomic/ is restored from the base branch before the agent starts, so only merged extensions load. Impact stays bounded by the firewall's network.allowed, read-only GitHub MCP, threat detection and safe outputs, and strict mode.
| description: Smoke test for Atomic headless execution and native MCP | ||
| name: Smoke Atomic | ||
| on: | ||
| schedule: every 2 days |
There was a problem hiding this comment.
The schedule was requested by the maintainer, and it doesn't add a new kind of exposure: each scheduled run does exactly what a manual dispatch already does. The containment is deterministic, not prompt text: strict: true, GitHub MCP in read-only mode (GITHUB_READ_ONLY: 1), the AWF firewall with network.allowed limited to defaults, github and go, threat detection on the agent output before anything is published, and safe outputs limited to create_issue and noop. The titles come from PRs merged in this repository, which requires maintainer action. gh-aw's other smoke workflows (for example smoke-agent-all-merged) already run every 2 days under the same model. Keeping the schedule.
List Atomic's default and bundled-extension tools explicitly instead of excluding ask_user_question, so tools added by later Atomic releases stay off until a reviewed bump lists them. The enabled set is unchanged.
| - --no-approve | ||
| - --tools | ||
| - read,bash,edit,write,mcp__* | ||
| - read,bash,edit,write,find,search,kill,todo,workflow,subagent,intercom,web_search,code_search,fetch_content,get_search_content,mcp__* |
There was a problem hiding this comment.
Keeping this set; it is intentional for the Atomic sample. The suggested subset (read,find,search,todo) would break Smoke Atomic itself: step 1 needs mcp__* for native GitHub MCP retrieval, and the build check needs bash. The impact of a steered tool call is bounded by deterministic controls outside the model, not by trimming the list: the AWF firewall limits network egress to the workflow's network.allowed (defaults, github and go here), so the web tools cannot reach arbitrary hosts; GitHub MCP runs read-only (GITHUB_READ_ONLY: 1); workflow and subagent run inside the same sandbox with the same tool policy; nothing leaves the run except safe outputs (create_issue, noop), which threat detection reviews before publication; and strict mode applies. The allowlist is explicit, so tools from later Atomic releases stay off until a reviewed bump lists them.
There was a problem hiding this comment.
Update (cbc7349b): at the maintainer's direction the sample now runs Atomic with its full feature set, minus the interactive ask_user_question, so this is back to --exclude-tools ask_user_question, now with --approve. The exclude list is deliberate: repositories add custom tools and workflows by committing them to .atomic/, which an allowlist in the engine definition would block. .atomic/ is restored from the base branch before the agent starts, so only merged extensions load. Impact stays bounded by the firewall's network.allowed, read-only GitHub MCP, threat detection and safe outputs, and strict mode.
|
✅ PR Code Quality Reviewer completed the code quality review. PR Code Quality ReviewerOutcome: The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ PR Specification Reviewer completed successfully! PR Specification ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Design Decision Gate 🏗️Outcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅ Matt Pocock Skills ReviewerOutcome: The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Ponytail Reviewer completed successfully! Ponytail ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
Comment MemoryPeek at saved memory (pr-code-quality-reviewer)Note This comment is managed by comment memory.Expand the saved memory block to view or edit the persistent context for this thread.
|
There was a problem hiding this comment.
The explicit tool allowlist and scheduled trigger are documented and consistent with the stated Atomic smoke-test goals. However, the new one-day npm cooldown weakens supply-chain protection for an unattended, recurring workflow with copilot-requests: write; please preserve the stronger default gate or pin a reviewed package version.
🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~2 AIC · ⌖ ~2 AIC · ⊞ 1.6K
Comment /review to run again
| engine: | ||
| id: atomic | ||
| # Validate a newly pinned Atomic release the day after it ships; strict mode allows at least one day. | ||
| cooldown: 1 |
There was a problem hiding this comment.
Lowering the npm release-age guard to one day for this unattended scheduled workflow materially increases supply-chain exposure because a newly published compromised Atomic package can be installed before the normal three-day delay while the job has copilot-requests: write.
💡 Keep the stronger install gate
The workaround addresses a transient ETARGET failure, but it trades away protection on a recurring job with write-capable permissions. Prefer pinning a reviewed Atomic version (or otherwise preserving the default cooldown) and update the pin through the normal review process.
There was a problem hiding this comment.
Addressed in c0e994bc: Atomic keeps cooldown: 1, while the threat-detection Copilot engine has an explicit cooldown: 3 override.
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /diagnosing-bugs, /codebase-design, and /tdd. No blocking defects found. Three follow-ups, inline.
📋 Key Themes & Highlights
Key Themes
- Unscoped cooldown side effect:
engine.cooldown: 1also lowers the detection job's Copilot install from 3 to 1 days. - Duplicated allowlist: the Atomic tests still hard-code the old tool list.
- Brittle schedule assertion: the test pins source text rather than the compiled cron.
Positive Highlights
- ✅ The lock files regenerate byte-for-byte from HEAD, so the committed artifacts match the sources.
- ✅ The allowlist explicitly excludes
ask_user_question, which is the right call for headless runs. - ✅ The
shared/atomic.mddoc comment now matches the allowlist.
The GHAS threads on the broad mcp__* allowlist and the scheduled prompt-injection surface are already covered by the author's replies, so I have not repeated them.
@copilot please address the review comments above.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~7 AIC · ⌖ ~2 AIC · ⊞ 6.6K
Comment /matt to run again
| engine: | ||
| id: atomic | ||
| # Validate a newly pinned Atomic release the day after it ships; strict mode allows at least one day. | ||
| cooldown: 1 |
There was a problem hiding this comment.
[/diagnosing-bugs] engine.cooldown: 1 is applied globally, not only to Atomic. The detection job in the compiled lock also installs Copilot with this cooldown, so its --cooldown-days drops from 3 to 1 (smoke-atomic.lock.yml line 2157). That weakens the supply-chain delay for the detection job, and the PR does not mention it. Confirm this is intended, or scope the setting to Atomic only.
💡 Evidence
Compiling HEAD with the cooldown line removed yields --cooldown-days 3 in the detection step and NPM_CONFIG_MIN_RELEASE_AGE: '3' for Atomic. Compiling HEAD as-is yields --cooldown-days 1 in the detection step. The detection job uses detection-engine: copilot from shared/atomic.md.
@copilot please address this.
There was a problem hiding this comment.
Addressed in c0e994bc: the Atomic install retains its one-day age, and the detector now has a separate three-day cooldown override.
| - --no-approve | ||
| - --tools | ||
| - read,bash,edit,write,mcp__* | ||
| - read,bash,edit,write,find,search,kill,todo,workflow,subagent,intercom,web_search,code_search,fetch_content,get_search_content,mcp__* |
There was a problem hiding this comment.
[/codebase-design] The allowlist now lives in shared/atomic.md, but the Atomic tests still hard-code the old read,bash,edit,write,mcp__* list. Those tests pass because they supply their own argument lists, so nothing checks the real allowlist.
💡 Suggested test
Add a test that reads the --tools value from shared/atomic.md and asserts it matches the documented set. It should also assert that ask_user_question is absent, as the doc comment states. Stale copies are at pkg/workflow/atomic_environment_test.go lines 47 and 110 and pkg/workflow/atomic_output_redaction_test.go lines 72 and 86. A single source of truth keeps these in sync.
@copilot please address this.
There was a problem hiding this comment.
Addressed in c0e994bc: the test pins the exact parsed execution args and rendered exclude flag; the harness fixtures now derive their args from the shared Atomic definition instead of carrying stale copies.
| require.NoError(t, err) | ||
| assert.Contains(t, string(source), "shared/atomic.md") | ||
| assert.NotContains(t, string(source), "schedule:") | ||
| assert.Contains(t, string(source), "on:\n schedule: every 2 days\n workflow_dispatch:\n") |
There was a problem hiding this comment.
[/tdd] This assertion matches exact source text, including blank lines and indentation, so harmless reformatting of the frontmatter breaks it. It also does not check the compiled cron. Assert the parsed schedule instead, or check that the lock file contains 0 0 */2 * *.
@copilot please address this.
There was a problem hiding this comment.
Addressed in c0e994bc: the smoke contract now asserts the generated lock's 0 0 */2 * * cron instead of matching exact frontmatter formatting.
Return to --exclude-tools ask_user_question so every Atomic tool is available, including tools a repository adds through its own extensions. Replace --no-approve with --approve and drop --no-skills and --no-prompt-templates, so the repository's .atomic/ extensions, workflows, skills, prompt templates and settings load, along with the skills gh-aw installs into .atomic/skills. gh-aw restores .atomic/ from the base branch before the agent starts, so a pull request cannot supply its own.
| - --no-approve | ||
| - --tools | ||
| - read,bash,edit,write,mcp__* | ||
| - --approve |
There was a problem hiding this comment.
🟡 Changes recommended
Atomic still enables native web fetching and search when workflows explicitly disable them, leaving those restrictions unenforced.
4 open issues
What changed in this PR
Extends the experimental Atomic engine sample following the 0.9.79 integration and adds recurring smoke validation.
Changes:
- Expands the explicit native-tool allowlist while excluding interactive questions.
- Schedules Smoke Atomic every two days and sets a one-day install cooldown for smoke and conformance workflows.
- Updates generated workflows, schedule assertions, and documentation of headless behavior.
| File | Description |
|---|---|
pkg/workflow/atomic_workflow_test.go |
Updates the smoke contract to require the new schedule. |
.github/workflows/smoke-atomic.md |
Adds the recurring schedule and one-day cooldown. |
.github/workflows/smoke-atomic.lock.yml |
Regenerates scheduling, tool selection, cooldowns, and metadata. |
.github/workflows/shared/atomic.md |
Expands the tool allowlist and documents headless behavior and limits. |
.github/workflows/engine-conformance-atomic.md |
Sets the Atomic conformance install cooldown to one day. |
.github/workflows/engine-conformance-atomic.lock.yml |
Regenerates tool selection, cooldown, and metadata. |
.changeset/patch-atomic-builtin-tools-smoke-schedule.md |
Records the sample changes in a patch release note. |
Review effort: Max (auto)
| - --no-approve | ||
| - --tools | ||
| - read,bash,edit,write,mcp__* | ||
| - read,bash,edit,write,find,search,kill,todo,workflow,subagent,intercom,web_search,code_search,fetch_content,get_search_content,mcp__* |
| - --no-approve | ||
| - --tools | ||
| - read,bash,edit,write,mcp__* | ||
| - read,bash,edit,write,find,search,kill,todo,workflow,subagent,intercom,web_search,code_search,fetch_content,get_search_content,mcp__* |
| engine: | ||
| id: atomic | ||
| # Validate a newly pinned Atomic release the day after it ships; strict mode allows at least one day. | ||
| cooldown: 1 |
| tools are enabled inside the gh-aw sandbox. This sample does not advertise | ||
| bundled extensions remain loaded. An explicit allowlist enables Atomic's default | ||
| and bundled-extension tools inside the gh-aw sandbox: read, bash, edit, write, | ||
| find, search, kill, todo, workflow, subagent, intercom, web_search, code_search, |
|
🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:
Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering. Structured data{
"completed_task": 0,
"head": "cbc7349b64fdb1d6bd4d1e6e4a6302ef6905ffa0",
"kind": "requested",
"schema_version": "1",
"state": "b9c6f4d32af83056884c3982ed6feb94a962c47837678ea6ade5d5bf6405d3cf",
"work": [
"43fb2fab4792659ce8a9a5979d64e8fe6e6a43f3219b7bf3a0fabc4ac29749da",
"a78befba6a8b7b6b0660b3a9ca59aff1129d1db3007f8f15c90b659e5242d9c0",
"a9ad9ae28dd89de35bca3e9353f293e215067bdc97e069540bcaec8bd06d9005",
"aeb0e4d812c55b9adeaa6ca02542f622f4350409c2cdbc19cbf26614539e6251"
],
"workflow": "pr-sous-chef"
}
|
…-storage Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
|
🎉 This pull request is included in a new release. Release: |



Follow-up to #77011. cc @pelikhan
Full Atomic, minus its one interactive tool
gh-aw always runs headless, so the sample now runs Atomic with its whole feature set except
ask_user_question:--exclude-tools ask_user_questionreplaces theread,bash,edit,write,mcp__*allowlist. Every Atomic tool is available (workflows, subagents, Intercom, web tools, todo, native MCP), including tools a repository registers through its own extensions. An exclude list rather than an allowlist is deliberate: a repository adds a custom tool by committing an extension, without editing the engine definition.--approvereplaces--no-approve, and--no-skillsand--no-prompt-templatesare dropped, so the repository's.atomic/extensions, workflows, skills, prompt templates and settings load, along with the skills gh-aw installs into.atomic/skills..atomic/with the base-branch copy before the agent starts, so a pull request cannot supply its own extensions or workflows; only merged ones load.Smoke schedule and cooldown
schedule: every 2 days, compiled to0 0 */2 * *).TestAtomicSmokeContractpreviously asserted there was no schedule; it now pins the new trigger.ETARGET … No matching version found for @bastani/atomic@0.9.79 with a date before 11/9/2026, from the default 3-day npm release-age cooldown.behaviors.installation.cooldownis deprecated and ignored, and strict mode rejectsengine.cooldown: 0, so Smoke Atomic and the Atomic conformance workflow setengine.cooldown: 1, the minimum strict mode allows. Their Atomic install now runs withNPM_CONFIG_MIN_RELEASE_AGE: '1'.Validation
go test ./pkg/workflow -run Atomicpasses.gh-aw compile smoke-atomic engine-conformance-atomic daily-declarative-engine-improver --validate: 3 succeeded, no deprecation warning..atomic/extensions/repo-probe.ts(a custom tool),.atomic/workflows/repo-probe-flow.tsand.atomic/skills/repo-probe-skill/SKILL.md, against a local OpenAI-compatible stand-in for the AWF proxy that issued aworkflowlistcall: the model was offered 16 tools includingrepo_probeand notask_user_question, the skill was in the system prompt,workflow listreturnedrepo-probe-flowwith the built-in workflows, and the run exited 0. Not yet run on Actions.