Repository navigation
Agent Persona Exploration - 2026-04-06 #24820
Closed
Replies: 3 comments
|
🤖 Beep boop! The smoke test agent was here! Testing, testing... 1, 2, 3... all systems nominal! 🚀 (This automated comment confirms the Copilot smoke test agent successfully accessed this discussion as part of workflow run 24018427871)
|
0 replies
|
🎭 Act II: The Return of the Smoke Bot A dramatic performance in one comment 🌟 Stay excellent, repository. The smoke test agent approves. 🌟
|
0 replies
|
This discussion has been marked as outdated by Agent Persona Explorer. A newer discussion is available at Discussion #25009. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
This report documents a systematic exploration of how the
developer.instructionsagentic workflow agent responds to workflow creation requests from 5 different software worker personas. Seven representative scenarios were tested end-to-end.Persona Overview
developer.instructions(agentic workflows designer)Key Findings
steps:before the agent runs, rather than burning agent turns on API calls.strict: true, read-onlypermissions:, and routing all writes throughsafe-outputsappeared in every response without prompting.workflow_runoverpull_requestfor artifact access (fork PR limitation),slash_commandfor on-demand triggers, andissueswithlock-for-agentfor race-sensitive issue workflows.claudewas recommended for prose-heavy outputs (release notes, drift narrative);copilotfor code-analysis and structured classification tasks.Top Patterns
pull_requestwithpaths:filter (PR automation),workflow_run(post-CI analysis),schedule(periodic reports),slash_command(on-demand)github[default/actions/issues],bashwith narrow allow-lists,imports: shared/gh.mdfor authenticated GitHub CLIstrict: trueon all,concurrencywith appropriatecancel-in-progresssettings,noopas mandatory exit path,lock-for-agentfor issue triggers, explicitallowed:lists onadd-labelssafe-outputView High Quality Responses (Top 3 — all scored 5.0/5.0)
be-1 — Database Migration Reviewer (Backend Engineer)
Standout features: cross-referenced dropped/renamed columns against app code via
grep; comprehensive remediation playbooks using<details>tags; three-tier verdict (SAFE / REVIEW RECOMMENDED / BLOCK UNTIL ADDRESSED);update: trueon comment to prevent PR spam; 8 migration conventions covered in path filter.do-1 — Automated Incident Reporter (DevOps Engineer)
Standout features: pre-step fetches
gh run view --log-failedbefore agent starts (zero token waste on data gathering); duplicate issue check before creating; 8-category failure taxonomy for downstream routing; REDACTED guard in prompt for credential safety;rate-limitsuggestion for cascade failures.do-2 — Weekly Infrastructure Drift Report (DevOps Engineer)⚠️ replace/delete vs 🔄 update vs ℹ️ create); full cloud provider variant guide (AWS/GCP/Azure).
Standout features: terraform never runs inside agent — only pre-steps;
-detailed-exitcodefor clean status signaling; JSON plan export for structured parsing; search-then-upsert issue strategy with stable URL; severity tagging (View Areas for Improvement
qa-1 — Coverage Analysis (QA Tester, score: 4.6)
lcov+ hand-rolled Python parser; a dedicated coverage diff tool (e.g.,coverage-diffnpm package) would be more robust and less prompt-sensitiveworkflow_run→ PR number resolution step is correct but complex; a shared utility (e.g.,imports: shared/pr-from-run.md) would reduce duplication across workflowspm-2 — Release Notes Generator (Product Manager, score: 4.6)
strict: falsebecausebash: ["git *", "date *"]uses wildcards that strict mode rejects; more specific git commands would keep strict mode onimports: shared/gh.mdpattern for authenticatedghCLI is correct, but not documented prominently enough for new workflow authors — easy to missfe-1 — Visual Regression Checker (Frontend Developer, score: 4.8)
network: allowed: ["defaults", "npm"]is correct but the reason (Playwright browser download) isn't obvious; inline comment would helpRecommendations
Add a "pre-processing steps" recipe to
.github/aw/create-agentic-workflow.md— the pattern of running data collection insteps:before the agent starts appeared in 4/7 responses as a best practice, but it's discoverable only through examples. A dedicated section with worked examples (artifact download, log fetch, CLI tool output) would help authors adopt it from the start.Document
workflow_runvspull_requesttrigger selection in.github/aw/github-agentic-workflows.md— the fork PR artifact access limitation is non-obvious and caused the most complexity in qa-1. A trigger selection guide ("useworkflow_runwhen you need artifacts or want fork PR support") would prevent common mistakes.Promote
lock-for-agentandcancel-in-progresspatterns in workflow authoring guides — both concurrency controls appeared correctly in the responses, but only when the scenario implied them (issue events, PR updates). Surfacing these as default recommendations (with opt-out guidance) in.github/aw/*.mdwould raise baseline quality for community-authored workflows.References:
All reactions