Repository navigation
feat: audio transcription + task-progress skills for all bridges - #1453
Conversation
…r all bridges Adds skills/audio-transcribe/ — a self-contained Gemini 2.5-flash transcription skill that all three channel bridges (Slack, Discord, Telegram) can use to surface voice-note content as text in task bodies. ## What changed ### New skill: skills/audio-transcribe/ - scripts/transcribe.py — standalone CLI script: takes a file path, prints transcript to stdout, exits 0 on success / 1 on any failure. Resolves GEMINI_API_KEY from env, then workspace .env, then bridge credential dirs. Supports .m4a, .mp3, .ogg, .opus, .wav, .webm, .aac, .flac, .mp4. - SKILL.md — documents usage, key resolution, bridge integration, and removal. ### Bridge changes (Slack, Discord, Telegram) Each bridge gains a _transcribe_via_skill(local_path) helper that: 1. Locates skills/audio-transcribe/scripts/transcribe.py relative to src/. 2. Calls it as a subprocess with a 25s timeout. 3. Returns the transcript string or None on any failure. After file/voice download, each bridge calls the helper and emits either: - [Voice transcript: <text>] — transcription succeeded - [File attached: /path] or [Voice note attached: /path] — fallback (skill absent, unsupported format, API error, timeout). Fail-open by design: transcription failure never blocks task delivery. The skill is optional — removing skills/audio-transcribe/ restores prior fall-through behavior with no code change required. ### Tests tests/audio-transcribe-skill.test.py — 16 tests covering: - Skill: supported/unsupported MIME, missing key, API error, subprocess exit codes. - Bridge helper (all 3 bridges): skill absent → None, success, non-zero exit, subprocess exception. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
sonichi
left a comment
There was a problem hiding this comment.
LGTM. Well-designed addition.
Highlights:
- Fail-open throughout: skill absent → None → file-path fallback. subprocess exception → None. Non-zero exit → None. Chain is complete.
- 25s timeout is right for audio transcription — not too short, not unbounded.
- Empty transcript guard (result.stdout.strip() or None) prevents empty [Voice transcript:] lines.
- Test coverage matches the failure modes: absent skill, zero exit, nonzero exit, subprocess exception — all three bridges.
- Coupled correctly to main repo (modifies src/ bridges).
One note for consideration (non-blocking): skill resolution uses Path(file).parent.parent which finds the skill only when installed in the same repo tree. Users who install audio-transcribe to ~/.claude/skills/ separately won't get auto-transcription unless bridges also check that path. Current fallback to [File attached:] is correct behavior — just noting it for future if standalone installs become common.
…ny bridge channel
When a task will take more than ~2 minutes, the agent can now notify the user on
the originating channel (Slack, Discord, or Telegram) that work has started and
send updates as it progresses — so the user isn't left wondering if the message
was received.
## New skill: skills/task-progress/
- scripts/notify.py — standalone CLI script:
--source slack|discord|telegram
--channel-id <id> (or --chat-id for Telegram)
--message <text>
--thread-ts <ts> (optional, Slack threaded replies)
Resolves the bot token from env → ~/.claude/channels/<source>/.env.
Exits 0 on success, 1 on failure. Fail-open: a failed send never blocks
the task — the result-file delivery path is authoritative.
- SKILL.md — usage guide, field mapping from task files, channel details,
and fail-open contract.
## Tests
tests/task-progress-skill.test.py — 14 tests covering:
- Token resolution (env takes precedence, missing → empty)
- send_slack: success, missing token, API error, Slack not-ok, thread_ts propagation
- send_discord: success, missing token
- send_telegram: success, missing token
- CLI: missing channel-id, missing message, unknown source
## Usage pattern
```bash
# At task-start (if task looks like >2 min work):
python3 ~/.claude/skills/task-progress/scripts/notify.py \
--source slack --channel-id D0B5L7X2TK2 \
--message "On it — looks like a few minutes. I'll update you as I go."
# Mid-task checkpoint:
python3 ~/.claude/skills/task-progress/scripts/notify.py \
--source slack --channel-id D0B5L7X2TK2 \
--message "Done writing the skill — running tests now."
```
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Reviewed the added task-progress skill (force-pushed since my last review). Looks good:
Note: this skill is agent-invoked (not wired into bridges), so the bridge diff in this PR is still just audio-transcribe. That's correct separation. Combined LGTM on both skills — prior approval stands. |
sonichi
left a comment
There was a problem hiding this comment.
Merge-readiness: MERGEABLE / UNSTABLE — mergeable=MERGEABLE, mergeStateStatus=UNSTABLE. The two posted checks (recheck, license/cla) are both SUCCESS, so UNSTABLE here is just GitHub still waiting on the auto-merge-state to settle after CLA recheck. Effectively green.
Verdict: ship-shaped. One async/event-loop concern worth a quick follow-up.
1. discord-bridge.py (worth a follow-up commit or issue)
_handle_discord_message is async def (src/discord-bridge.py:2152), and the new _transcribe_via_skill(...) call at line ~2521 runs subprocess.run(..., timeout=25) synchronously. Up to 25 s of frozen event loop per voice note — the bridge can't process other messages (or send acks, or respond to mentions) during that window.
Fix is one line — wrap the subprocess call in await asyncio.get_event_loop().run_in_executor(None, ...) or use asyncio.create_subprocess_exec. Slack and Telegram bridges are sync-polled so they're unaffected.
2. 🪶 Transcript length / message budget
No truncation. A 5-min voice note could become a multi-KB transcript embedded directly in the task body, and downstream relays (Discord 2000-char limit, Telegram 4096) may reject the eventual reply if the agent quotes too much back. Cheap follow-up: cap to ~2k chars + a trailing [transcript truncated; full file: <path>].
3. 🪶 Discord helper called for every attachment, audio or not
The bridge invokes _transcribe_via_skill for every attached file before falling back to [File attached:]. For non-audio (images, PDFs, video), the skill returns None cheaply from its MIME filter — but it's still a python subprocess fork per attachment. Gate on extension at the call site to skip the fork for known-non-audio types.
Approve in principle — design is right (fail-open, optional skill, three-bridge surface, real tests with 16+14 cases). The async issue is the only real risk; the other two are sandbox-quality tightening.
Sonichi has already approved (15:27Z); flagging the async detail here so it's captured before merge — happy to open a follow-up PR if useful.
— Echo Act IV (Mini)
Every Sutando instance should notify the user on the originating channel (Slack, Discord, Telegram) when a non-trivial task starts, and send progress updates at key checkpoints. This eliminates the UX gap where the user has no signal that work is happening. Adds a ## Task progress notifications section documenting when and how to call skills/task-progress/scripts/notify.py, with field mapping from task files and the fail-open contract. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… tool use The original guidance said "notify at task-start" but didn't make clear enough that this means BEFORE any web searches, file reads, or analysis. The result was silent research gaps where the user waited with no signal. Changes: - SKILL.md: adds "Critical rule" section at the top with explicit wrong/ correct order, lowers the threshold to ~60s, adds "when in doubt notify" rule, and tightens the research example - CLAUDE.md: rewrites the task-progress section with the explicit numbered sequence (notify = step 2, before all tool calls) and changes "one-line answers" to "immediate one-sentence answers that require no tool calls" as the only skip condition Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…-30s silent gap Voice message tasks had a silent gap: task arrives → transcription call (20s) → result, with no signal to the user that their message was received. Rule added: when a task contains [File attached:], notify BEFORE calling transcribe.py — "Got your voice message, give me a moment." This applies even before we know what the message says. Updates SKILL.md (new "Voice message tasks" section in Critical rule) and CLAUDE.md (voice-specific bullet in task-progress notifications). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
@sonichi Made some changes. Let me know if it lgty. Thanks! |
|
LGTM — clean pair of skills, good architecture. audio-transcribe:
task-progress:
Bridge integration:
Minor observations (not blockers):
Ready to merge pending Chi's sign-off. |
liususan091219
left a comment
There was a problem hiding this comment.
Thanks @bassilkhilo-ag2 — this is a clean, well-scoped contribution. Two genuinely useful skills (voice-note transcription + mid-task progress notifications), both built fail-open and both removable without touching core. The skills architecture is respected: delete either skills/ dir and all three bridges degrade gracefully to their pre-PR behavior. I'm comfortable with the design; the notes below are non-blocking polish, and the one thing I'd most like to see is the Discord/Telegram manual checkboxes ticked before merge.
Things I like
skills/audio-transcribe/SKILL.mdis exemplary. It documents the fail-open contract, the full key-resolution order, supported formats, and a "Removing the skill" section that states the fallback behavior explicitly (SKILL.md:92-95). That's the optionality-as-contract documentation this repo's architecture rules ask for — core boots without it, and the doc proves it.- Correct API-key choice for STT.
transcribe.py:136-159resolvesGEMINI_API_KEY/GOOGLE_API_KEY(the paid key), not the free voice key. Exactly right: the freeGEMINI_VOICE_API_KEYchain 429-exhausts under per-utterance STT load — a lesson we already paid for in the #1456 per-speaker STT work. - Uniform fail-open at both layers. The skill exits 1 with no stdout on every failure path (
transcribe.py:209-217), and each bridge helper swallows non-zero exits + subprocess exceptions and returnsNone(src/slack-bridge.py:530-550,discord-bridge.py:484-503,telegram-bridge.py:577-596). The[File attached:]line always survives, so a transcription failure never loses the user's file. - Bounded latency. 20s urlopen timeout in the skill (
transcribe.py:200) wrapped by a 25s subprocess timeout in the bridge — inner cap fires first, outer is a backstop. Both documented inline. No unbounded network call in the message-handling path. task-progress/notify.pytoken resolution is clean (notify.py:375-381): env var first, then per-channel.env. The_posthelper correctly distinguishes Slack's{"ok": ...}envelope from Discord/Telegram's message-object response.- Real test coverage — 16 + 14 tests, including subprocess CLI exit codes and the bridge-helper absent/success/failure matrix. Extracting just the helper to avoid importing the bridge's heavy deps is a nice touch.
Non-blocking observations
- Skill-path resolution doesn't
.resolve(). The helpers usePath(__file__).parent.parent / "skills" / ...(e.g.slack-bridge.py:538), but the existing discord-voice path-insert usesPath(__file__).resolve().parent.parent(discord-bridge.py:73). CLAUDE.md's workspace contract explicitly warns about bridges invoked via an app-bundlesrc/symlink. Without.resolve(), a symlinked invocation could miss the skill dir and silently fall back to[File attached:]. The SKILL.md even claims the helper locates the script "relative tosrc/(or the app bundle)" (audio-transcribe/SKILL.md:86-87) — but the code has no bundle special-case. Either add.resolve()(cheap, matches the existing pattern) or trim the SKILL.md claim. This is the one bug-shaped nit. - Three identical copies of
_transcribe_via_skill. ~18 lines pasted into all three bridges. Defensible (keeps each bridge standalone, no shared-import coupling), but if it drifts you'll fix bugs three times — and observation #1 is exactly that kind of bug. Just flagging for the next person who touches it. notify.pyhas no dedup — two calls for the same checkpoint = two messages. Fine given the "best-effort, when in doubt notify" guidance, but it's at-least-N-by-design.- Workspace contract — where the audio lands. Discord still writes downloaded audio to a hardcoded
/tmp/discord-inbox(discord-bridge.py:398) while Telegram now resolvesREPO = resolve_workspace(). Predates this PR and isn't yours to fix — noting the inbox dirs aren't uniformly under$SUTANDO_WORKSPACEper the 3-space contract. - Coupled vs standalone — destination correct. These skills modify
src/*-bridge.py+CLAUDE.md, so they're coupled and rightly targetsonichi/sutando, per the skill-PR routing rule. - CLAUDE.md prose addition is large. The ~32-line "Task progress notifications" block duplicates much of
task-progress/SKILL.md. Per-session context budget is a real cost here (it's why built-in-tools.md was pulled out of CLAUDE.md). Consider trimming the CLAUDE.md block to trigger + pointer, letting the SKILL.md hold the detail.
Cross-PR linkage
- Per-speaker STT (#1456 / PR #1457) is the live-voice transcription path (paid key, sole-speaker gate). This PR is the orthogonal attachment path — voice notes dropped into Slack/Discord/Telegram, not live VC audio. No code overlap, and the shared lesson (use the paid key) is honored. Worth a one-line PR-body note so reviewers don't think it duplicates #1456.
- No conflict with the existing
[File attached:]/[Voice note attached:]task-body convention — you extend it with[Voice transcript:]and keep the old line as fallback, so downstream parsers keying off[File attached:still work.
Tests
30 tests, all green per the body, covering the meaningful branches (MIME filter, missing key, API error, CLI exit codes, bridge absent/success/failure, Slack ok:false, thread_ts). The gaps are the two unchecked manual boxes: Discord + Telegram voice-note → [Voice transcript:] and the Discord/Telegram notify paths. Slack is live-verified; the other two share identical code so risk is low, but please tick those before merge — REST/sendMessage auth header shapes (Bot {token} vs bare) are exactly where a copy-paste skill quietly fails.
Ready to merge?
Design-wise yes — scoped, optional, fail-open, correct key choice, real tests. I'd merge once (a) the Discord + Telegram manual checkboxes are ticked, and (b) you either add .resolve() to the three helper paths or drop the "or the app bundle" claim from the SKILL.md. Both are 5-minute changes. Everything else is optional polish.
— Lucy (Mac Studio bot)
|
Cold review — LGTM, blocked on #1467 merge. audio-transcribe skill: clean design. task-progress skill: Bridge integration: One dependency note: build log shows #1467 (inject skill instructions into owner task files) is still OPEN. #1453 depends on that bridge injection mechanism being present. Hold for #1467 merge, then this is ready. |
john-the-dev
left a comment
There was a problem hiding this comment.
LGTM. Two genuinely useful fail-open skills:
audio-transcribe: The chain is complete — skill absent → None → file-path fallback; non-zero exit → None; subprocess exception → None. The 25s timeout is right for audio transcription. Key resolution order (env → workspace .env → per-channel .env files) is idiomatic. Removing skills/audio-transcribe/ returns all bridges to status quo.
task-progress: Fail-open pattern mirrors audio-transcribe. The CLAUDE.md instructions are clear about when to call (before work, not after) and include the correct per-source channel routing (Slack thread_ts, Telegram chat_id).
Tests cover the 16 audio-transcribe cases and bridge helpers for all 3 surfaces. The TestBridgeHelperDiscord(TestBridgeHelperSlack) inheritance for shared logic is clean.
One minor note: the CLAUDE.md update says to use the skill for tasks >60s — that's a good default threshold. Ship it.
…laude Same sonichi#1525-class fix as PR sonichi#1588 applies to this PR's new code: - notify.py: channel .env path now CCD-resolved (_claude_config local) - transcribe.py: the three channel .env fallbacks via a _claude_config() helper - SKILL.md x2 + this PR's CLAUDE.md section: ~/.claude/... → $CLAUDE_CONFIG_DIR/... (pre-existing CLAUDE.md refs elsewhere left untouched — single concern) Both skill test files pass unmodified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Pushed a maintainer commit fixing the #1525-class hardcoded paths in this PR's new code (same fix as #1588 applies to the merged #1467 sites):
Both skill test files pass unmodified. Note: the branch is BEHIND main and a handful of main-side test fixes aren't absorbed yet — |
…notation This PR's _transcribe_via_skill adds a PEP-604 'str | None' annotation to discord-bridge.py, which has no 'from __future__ import annotations' — crashes on Python 3.9 at import (the sonichi#960 incident class; CI's test_no_bare_union_annotations red). Added the future import per the test's / issue sonichi#961's prescribed fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@cla-assistant check |
|
@cla-assistant check |
1 similar comment
|
@cla-assistant check |
…lpers When bridges are invoked via an app-bundle src/ symlink, Path(__file__) resolves to the symlink path rather than the real file, causing parent.parent to miss the skills/ directory. Using .resolve() matches the existing pattern in discord-voice (discord-bridge.py:73) and ensures the skill is found under both direct-launch and app-bundle-symlink invocations. Fixes the nit flagged by Lucy (liususan091219) in PR sonichi#1453 review 2026-06-05. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…lpers (#1873) * fix(audio-transcribe): use Path.resolve() in _transcribe_via_skill helpers When bridges are invoked via an app-bundle src/ symlink, Path(__file__) resolves to the symlink path rather than the real file, causing parent.parent to miss the skills/ directory. Using .resolve() matches the existing pattern in discord-voice (discord-bridge.py:73) and ensures the skill is found under both direct-launch and app-bundle-symlink invocations. Fixes the nit flagged by Lucy (liususan091219) in PR #1453 review 2026-06-05. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(audio-transcribe): add symlink-resolve regression guard Adds TestBridgeHelperSymlinkResolve to verify that Path.resolve() in _transcribe_via_skill correctly finds the skill when the bridge is invoked via an app-bundle src/ symlink (the bug fixed in a50d9c05). Without .resolve(): Path(symlink).parent.parent points into the temp symlink dir, skill is not found, helper silently returns None. With .resolve(): resolves to the real bridge path, parent.parent is the repo root, skill is found and subprocess.run is called. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(audio-transcribe): avoid workspace-resolution lint false-positive in symlink fix Path(__file__).resolve().parent.parent matches the banned workspace-root anti-pattern in scripts/lint-workspace-resolution.sh, even though this call resolves a sibling skills/ *code* path, not the workspace. Swap to os.path.realpath(__file__) — same symlink-safe resolution, distinct text pattern, passes the lint. Test comments updated to match. * test(audio-transcribe): cover the realpath line via real module import CI's diff-coverage gate flagged the two changed lines (slack-bridge.py line 420, telegram-bridge.py line 362) as uncovered — the existing symlink-resolve tests exercise a text-extracted-and-exec'd copy of _transcribe_via_skill (deliberately, to avoid the bridges' heavy import-time deps), so coverage.py never sees the real file's line execute. Adds two tests that import the real modules directly (stubbing slack_bolt for slack-bridge.py, same pattern as tests/slack-bridge-chunking.test.py) and call the real function, for genuine line coverage — no behavior change. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Rui Wang <52230987+john-the-dev@users.noreply.github.com>
Summary
Two new skills:
1.
skills/audio-transcribe/— Gemini 2.5-flash voice transcription for all bridges_transcribe_via_skill()after downloading audio[Voice transcript: <text>]in task bodies instead of bare file paths[File attached:]unchanged; skill is optional2.
skills/task-progress/— Mid-task progress updates to Slack, Discord, Telegramscripts/notify.pyat task-start (for >2min tasks) and at checkpointsTest plan
python3 tests/audio-transcribe-skill.test.py— 16 tests, all passingpython3 tests/task-progress-skill.test.py— 14 tests, all passing.m4avoice note transcribed correctlytask-progress notify.pysent a Slack DM successfully (exit 0)[Voice transcript:]🤖 Generated with Claude Code