Skip to content

Onboard dev-lead as a compliant persona #1281

Closed

Description

@don-petry

Bring dev-lead under the persona standard. Part of the framework rollout tracked in combsmith/.github#755.

dev-lead is the highest-blast-radius agent we have — it is the only one that writes code — and it is currently the least governed by the framework. That inversion is the reason to do this one carefully rather than first.

What already exists (verified)

Piece State
Reusable ✅ .github/workflows/dev-lead-reusable.yml
Caller stub ✅ .github/workflows/dev-lead.yml
canary-rings.json entry ✅ agents.dev-lead registered
Prompt library ✅ prompts/dev-lead/ — 7 files, including on-mention.md
Labels ✅ dev-lead, dev-lead:hands-off, dev-lead:needs-human
Role-based name ✅ already dev-lead, not a person name (§1.6)

What is missing

Piece State
personas/dev-lead/persona.yml ❌ does not exist
address block + petry-projects/dev-lead team ❌
evals/dev-lead/ ❌ does not exist at all (verified 404)

Blockers — do not start until these clear

  1. gate_label is declared but never enforced (feat: implement issue #737 — [Phase 1] Instrument auto-rebase: agentic-conflict-resolution rate + fan-out CI-run volume report #755, finding 2). dev-lead is a write persona. Onboarding it with mention + write before the router enforces the gate label means @petry-projects/dev-lead do X triggers an ungated write. This is the single reason dev-lead must not be the first persona onboarded.
  2. The eval gate is unenforced (feat: implement issue #737 — [Phase 1] Instrument auto-rebase: agentic-conflict-resolution rate + fan-out CI-run volume report #755, finding 5) and dev-lead has no evals. Principle 5 says no persona reaches stable without one. The agent that writes code is the one where a held-out eval matters most.

Design questions to settle first

  • Which surfaces are write vs advisory? dev-lead writes on issues (implements stories) but arguably only advises on review threads. Each write surface needs its own gate_label — the existing dev-lead label is the obvious candidate, and it is already the live trigger.
  • on-mention.md already exists. How does the persona mention surface relate to the current mention path? Reuse, or is it superseded?
  • What is dev-lead's eval set? It has none. Building a meaningful held-out set for a code-writing agent is real work — likely the largest task here, and it should not be faked. See dev-lead fabricates 'captured telemetry' data when it lacks the real source (caught on #1102) — add a provenance guard #1160 (dev-lead fabricated telemetry once already; invented eval cases would be the same failure).

Acceptance criteria

Activity

  1. github-actions commented on Jul 17, 2026

    @github-actions
    Contributor

    Thanks for the detailed writeup — the blocker analysis and design questions make the risks very clear and will save a lot of back-and-forth. Before any implementation starts, what is the current thinking on sourcing real eval cases for dev-lead? The issue flags this as the largest piece of work and explicitly calls out the risk of invented cases, so understanding whether there are candidate de-identified runs already available (or whether collection needs to be planned as a separate step) seems like the right thing to nail down first.

  2. don-petry commented on Sep 1, 2026

    @don-petry
    CollaboratorAuthor

    Closing — the persona is onboarded and its compliance is enforced by CI

    Verified on origin/main 2026-09-01.

    Manifest artifacts exist for this persona:

    personas/<persona>/persona.yml
    personas/<persona>/interaction.yml
    personas/<persona>/README.md
    

    Compliance is machine-checked, not asserted. .github/workflows/lint.yml carries a dedicated validate-personas job (backed by personas/validate-personas.py) plus the bats suites tests/test_validate_personas.bats, tests/verify_persona_teams.bats and tests/persona_runner.bats, and the workflow path-filter includes personas/** so any manifest change re-runs them.

    lint.yml is green on main across its last five runs (latest 2026-09-01T02:37), so the persona validates against standards/personas/persona.schema.json and its interaction contract passes the model validator today — which is exactly the "compliant persona" bar this issue set.

    Any future drift is caught by that job rather than by this tracking issue. Post-rollout hardening continues under #1318.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions