Repository navigation
Closed
Labels
enhancementFeature requestsFeature requestsneeds-human-reviewFlagged by automated PR review agentFlagged by automated PR review agent
Description
Bring dev-lead under the persona standard. Part of the framework rollout tracked in combsmith/.github#755.
dev-lead is the highest-blast-radius agent we have — it is the only one that writes code — and it is currently the least governed by the framework. That inversion is the reason to do this one carefully rather than first.
What already exists (verified)
| Piece | State |
|---|---|
| Reusable | ✅ .github/workflows/dev-lead-reusable.yml |
| Caller stub | ✅ .github/workflows/dev-lead.yml |
canary-rings.json entry |
✅ agents.dev-lead registered |
| Prompt library | ✅ prompts/dev-lead/ — 7 files, including on-mention.md |
| Labels | ✅ dev-lead, dev-lead:hands-off, dev-lead:needs-human |
| Role-based name | ✅ already dev-lead, not a person name (§1.6) |
What is missing
| Piece | State |
|---|---|
personas/dev-lead/persona.yml |
❌ does not exist |
address block + petry-projects/dev-lead team |
❌ |
evals/dev-lead/ |
❌ does not exist at all (verified 404) |
Blockers — do not start until these clear
gate_labelis declared but never enforced (feat: implement issue #737 — [Phase 1] Instrument auto-rebase: agentic-conflict-resolution rate + fan-out CI-run volume report #755, finding 2). dev-lead is awritepersona. Onboarding it withmention+writebefore the router enforces the gate label means@petry-projects/dev-lead do Xtriggers an ungated write. This is the single reason dev-lead must not be the first persona onboarded.- The eval gate is unenforced (feat: implement issue #737 — [Phase 1] Instrument auto-rebase: agentic-conflict-resolution rate + fan-out CI-run volume report #755, finding 5) and dev-lead has no evals. Principle 5 says no persona reaches
stablewithout one. The agent that writes code is the one where a held-out eval matters most.
Design questions to settle first
- Which surfaces are
writevsadvisory? dev-lead writes on issues (implements stories) but arguably only advises on review threads. Eachwritesurface needs its owngate_label— the existingdev-leadlabel is the obvious candidate, and it is already the live trigger. on-mention.mdalready exists. How does the personamentionsurface relate to the current mention path? Reuse, or is it superseded?- What is
dev-lead's eval set? It has none. Building a meaningful held-out set for a code-writing agent is real work — likely the largest task here, and it should not be faked. See dev-lead fabricates 'captured telemetry' data when it lacks the real source (caught on #1102) — add a provenance guard #1160 (dev-lead fabricated telemetry once already; invented eval cases would be the same failure).
Acceptance criteria
-
personas/dev-lead/persona.ymlvalidates;id/nameare the role (already true). -
definition.layers[]points at the realprompts/dev-lead/library andagents/profile if applicable. - Trigger matrix declares every surface honestly — no surface claimed that has no implementation (feat: implement issue #737 — [Phase 1] Instrument auto-rebase: agentic-conflict-resolution rate + fan-out CI-run volume report #755, finding 7).
- Every
writesurface carries agate_label, and the router enforces it (blocked on feat: implement issue #737 — [Phase 1] Instrument auto-rebase: agentic-conflict-resolution rate + fan-out CI-run volume report #755). -
canary.agentpoints at the existingagents.dev-leadentry; nothing is restated. -
runtime.permissionsrecorded; the caller stub grants a superset. -
evals/dev-lead/{dev,holdout}exist with real, de-identified cases ≥min_cases. - If
mentionis enabled:address.handle: petry-projects/dev-lead, team createdclosed+notifications_disabled+ 0 members, and CI verifies it (blocked on Verify persona addressing teams in CI #1280).
Metadata
Metadata
Assignees
Labels
enhancementFeature requestsFeature requestsneeds-human-reviewFlagged by automated PR review agentFlagged by automated PR review agent
Activity
github-actions commentedon Jul 17, 2026
Thanks for the detailed writeup — the blocker analysis and design questions make the risks very clear and will save a lot of back-and-forth. Before any implementation starts, what is the current thinking on sourcing real eval cases for
dev-lead? The issue flags this as the largest piece of work and explicitly calls out the risk of invented cases, so understanding whether there are candidate de-identified runs already available (or whether collection needs to be planned as a separate step) seems like the right thing to nail down first.Onboard dev-lead, pr-review, business-analyst as personas (#1303)
don-petry commentedon Sep 1, 2026
Closing — the persona is onboarded and its compliance is enforced by CI
Verified on
origin/main2026-09-01.Manifest artifacts exist for this persona:
Compliance is machine-checked, not asserted.
.github/workflows/lint.ymlcarries a dedicatedvalidate-personasjob (backed bypersonas/validate-personas.py) plus the bats suitestests/test_validate_personas.bats,tests/verify_persona_teams.batsandtests/persona_runner.bats, and the workflow path-filter includespersonas/**so any manifest change re-runs them.lint.ymlis green onmainacross its last five runs (latest 2026-09-01T02:37), so the persona validates againststandards/personas/persona.schema.jsonand its interaction contract passes the model validator today — which is exactly the "compliant persona" bar this issue set.Any future drift is caught by that job rather than by this tracking issue. Post-rollout hardening continues under #1318.