Skip to content

feat(atb2): add Slack intake and approved babysitter fixes - #4766

Draft
ATX24 wants to merge 6 commits into
canaryfrom
baml/feedback-slack-ready
Draft

feat(atb2): add Slack intake and approved babysitter fixes#4766
ATX24 wants to merge 6 commits into
canaryfrom
baml/feedback-slack-ready

Conversation

@ATX24

@ATX24 ATX24 commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Slack mentions now enter the feedback runner through a signed HTTP endpoint. @bammy babysit <PR URL> investigates CI failures and review comments, posts a proposed fix in the Slack thread, and links to a private proposal page. An explicitly mapped shepherd can approve the exact Slack message; website approval requires current GitHub maintain/admin access. Requester status alone grants no approval rights.

Each approval authorizes one implementation, test gate and push, bound to the PR head and feedback. Approval is consumed atomically; changed feedback or a changed head requires a new proposal. The runner announces the push in Slack before updating the branch with an explicit lease. Later review feedback requires another approval. The final message reports CI and reviewer-feedback status without automatically merging or posting review-bot resolve commands.

Also includes shepherd approval for newly triaged issues, website status badges and private proposal/authentication routes, Fly HTTP intake, and root-only Infisical Universal Auth using the existing machine identity. CLI update notices (part 6) are outside this PR.

Security boundaries:

  • Claude project settings, hooks and ambient MCP configuration are disabled.
  • Agent commands and gates run in Linux bubblewrap mount/PID namespaces with an isolated HOME and independent Git clone. Controller credentials and processes are excluded. Startup fails closed if namespaces are unavailable.
  • A local fixed-origin Claude messages broker keeps the persistent OAuth login outside the agent sandbox. The agent receives only a temporary broker capability.
  • Pushes export objects without credentials and import them into fresh trusted Git metadata. Credential-bearing Git uses a fixed repository URL, explicit source SHA/destination ref and expected-head lease, ignoring agent-controlled origin, URL rewrites and helpers.
  • Bootstrap compiler builds use a separate builder UID. Infisical authentication credentials stay out of the runtime environment.

Validation:

  • BAML formatting/check and 97 token-free tests; existing unreachable-code warning in models.baml.
  • 8 offline sandbox/broker/push security tests, including a real Claude CLI with fake upstream responses, host file/process isolation, hook suppression and hostile Git configuration.
  • 9 launcher, 6 compiler-cache, 3 local Slack HTTP and 6 website authorization tests.
  • Runner image build and real Cargo/bootstrap isolation fixture. Nested namespaces require a privileged disposable test container; fixtures have networking disabled and no production credentials.
  • Website typecheck/build; the existing Next/ESLint compatibility diagnostic is documented in the build output.
  • Infisical secret scan of the outgoing commits plus token/private-path, stray-file, conflict, installer and action-pin checks before push.

Remaining deployment risks:

  • Live Fly namespace support, real Claude OAuth and end-to-end Slack-to-GitHub execution have not been verified.
  • Future agent-generated commits still need a mandatory outgoing secret/content security gate. This PR's scan does not scan future fixes.
  • The prior review found advisories in installed website dependencies, including Next 15.5.9. Deployed versions and exploitability were not verified.
  • Proposal recovery can fail outside the worker's retry handler; listener health alone does not establish worker progress.

Keep this PR in draft while those deployment blockers are reviewed. Merging changes tools/atb2 and triggers the canary Fly deployment workflow when FLY_API_TOKEN is present.

Deployment/setup:

  1. Apply the additional proposal SQL below in the Supabase dashboard before starting this version. The owner reported the earlier Slack SQL applied; the proposal table is additional.
  2. Set runner ATB2_UI_URL to the HTTPS feedback-site origin and ATB2_SHEPHERDS to explicit GitHub-login:Slack-user-ID pairs. The owner reported the Infisical client pair staged on Fly and FLY_API_TOKEN configured in GitHub; their current state has not been verified here.
  3. Configure website FEEDBACK_SITE_URL, FEEDBACK_GITHUB_CLIENT_ID, FEEDBACK_GITHUB_CLIENT_SECRET, FEEDBACK_APPROVAL_SESSION_KEY (64 random hex characters), and FEEDBACK_APPROVAL_SUPABASE_KEY (server-only private-table access), alongside the existing Supabase URL/anonymous key.
  4. Register the GitHub OAuth callback as <FEEDBACK_SITE_URL>/auth/github/callback. Website approval checks maintain/admin permission each time.
  5. After deployment, verify namespace preflight and complete the runtime user's Claude login: fly ssh console -a atb2-runner, then runuser -u atb2 -- env HOME=/data/home claude. Verify the broker against that login before accepting live requests.
  6. Set Slack Events URL to https://atb2-runner.fly.dev/slack/events, subscribe app_mention and reaction_added, add reactions:read alongside existing bot scopes, reinstall and invite the bot to the test channel.
  7. Test @bammy babysit . Verify proposal publication, mapped-shepherd approval, tests, pre-push narration and a single branch update.

Earlier Slack schema additions (owner reported applied):

alter table public.feedback add column if not exists slack_event_id text unique;
alter table public.feedback add column if not exists slack_thread_ts text;
alter table public.babysit_requests add column if not exists slack_event_id text unique;
alter table public.babysit_requests add column if not exists kind text not null default 'babysit';
alter table public.issues add column if not exists slack_ts text;
alter table public.issues add column if not exists slack_channel text;
create unique index if not exists issues_approval_message
  on public.issues (dataset, slack_channel, slack_ts)
  where slack_ts is not null;

Additional proposal schema (not yet applied):

-- Apply in the Supabase SQL editor. Additional to the Slack intake SQL already applied.
begin;
create table if not exists public.babysit_proposals (
  id text primary key,
  request_id bigint not null references public.babysit_requests(id),
  pr text not null,
  head text not null check (head ~ '^[0-9a-f]{40}$'),
  fingerprint text not null check (fingerprint ~ '^[0-9a-f]{64}$'),
  summary text not null check (length(trim(summary)) > 0),
  plan text not null check (length(trim(plan)) > 0),
  status text not null default 'pending' check (status in ('pending','approved','executing','pushed','failed','stale')),
  requested_by text,
  channel text,
  slack_ts text,
  approved_by text,
  approved_at timestamptz,
  dataset text not null default 'live' check (dataset in ('live','eval')),
  created_at timestamptz not null default now(),
  constraint babysit_approval_required check (
    status not in ('approved','executing','pushed') or
    (approved_by ~ '^(slack|github):[A-Za-z0-9-]+$' and approved_by is not null and approved_at is not null)
  )
);
create unique index if not exists babysit_one_active_proposal
  on public.babysit_proposals(request_id) where status in ('pending','approved','executing');
create unique index if not exists babysit_proposal_message
  on public.babysit_proposals(dataset,channel,slack_ts) where slack_ts is not null;
create index if not exists babysit_proposals_status on public.babysit_proposals(dataset,status);
alter table public.babysit_proposals enable row level security;
revoke all on public.babysit_proposals from anon, authenticated;
grant select,insert,update on public.babysit_proposals to service_role;

-- A displayed plan is immutable. A revised plan must be a new proposal.
create or replace function public.guard_babysit_proposal() returns trigger
language plpgsql set search_path = '' as $$
begin
  if (new.id,new.request_id,new.pr,new.head,new.fingerprint,new.summary,new.plan,new.requested_by,new.dataset)
     is distinct from
     (old.id,old.request_id,old.pr,old.head,old.fingerprint,old.summary,old.plan,old.requested_by,old.dataset) then
    raise exception 'proposal contents are immutable';
  end if;
  if old.status <> 'pending' and
     (new.approved_by,new.approved_at,new.channel,new.slack_ts) is distinct from
     (old.approved_by,old.approved_at,old.channel,old.slack_ts) then
    raise exception 'approval identity and message are immutable';
  end if;
  if new.status <> old.status and not (
    (old.status='pending' and new.status in ('approved','stale')) or
    (old.status='approved' and new.status in ('executing','stale')) or
    (old.status='executing' and new.status in ('pushed','failed'))
  ) then raise exception 'invalid proposal transition'; end if;
  return new;
end;
$$;
drop trigger if exists guard_babysit_proposal on public.babysit_proposals;
create trigger guard_babysit_proposal before update on public.babysit_proposals
  for each row execute function public.guard_babysit_proposal();
commit;

@vercel

vercel Bot commented Sep 7, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
beps Ready Ready Preview Sep 7, 2026 4:27am UTC
developer-docs Ready Ready Preview Sep 7, 2026 4:27am UTC
promptfiddle2 Ready Ready Preview Sep 7, 2026 4:27am UTC

Request Review

@coderabbitai

coderabbitai Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

  • Add RUN_CODSPEED=1 to the PR description, or
  • Include run-perf or /perf in the PR title or any commit message.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant