bug(whatsapp): inbound media download always fails — SSRF allowlist rejects Twilio's own media CDN #1932

Summary

Every inbound WhatsApp attachment fails to download. Twilio serves message media from mms.twiliocdn.com, reached via a 302 from api.twilio.com. WhatsAppAdapter.download_file deliberately does not auto-follow redirects and re-validates the redirect target against an SSRF allowlist that accepts only the .twilio.com suffix — so the CDN host is refused and download_file returns None. The agent receives a <name> — download failed placeholder instead of the file.

This is not voice-specific: it breaks every inbound media type on WhatsApp — images, PDFs, documents, audio. It has been this way since the adapter shipped (#463, 2026-04-23), so inbound WhatsApp media has never worked.

Context

src/backend/adapters/whatsapp_adapter.py:

_TWILIO_MEDIA_ALLOWED_HOST_SUFFIXES = (".twilio.com",)   # line 44

# download_file, ~line 669
async with httpx.AsyncClient(timeout=30.0, follow_redirects=False) as client:
    resp = await client.get(file.url, auth=(binding["account_sid"], auth_token))
    # Twilio sometimes 302-redirects to a signed S3 URL for large media.
    # Follow only if the redirect target is still Twilio-hosted.
    if resp.status_code in (301, 302, 303, 307, 308):
        location = resp.headers.get("location", "")
        if not _is_twilio_media_url(location):
            logger.warning(
                "[WHATSAPP] Refusing off-domain media redirect to host=%s",
                urlparse(location).hostname,
            )
            return None

mms.twiliocdn.com is a different registrable domain from twilio.com, so _is_twilio_media_url returns False and the download is abandoned. The inline comment ("sometimes… for large media") understates the behavior — Twilio redirects to the CDN for ordinary media too, which is why the failure is total rather than intermittent.

Observed on a deployed instance: repeated [WHATSAPP] Refusing off-domain media redirect to host=mms.twiliocdn.com warnings in the backend log, with the agent's retained history showing media_0.ogg — download failed for each attachment and the agent replying to the sender that the voice message "didn't come through." Every attachment that instance has ever received failed this way.

Why it went unnoticed. The shipping PR (#463) tested the SSRF allowlist with six spoof vectors (eviltwilio.com, api.twilio.com.evil.com, plain-http://, garbage) — all negative cases. No test exercised a real Twilio 302, so the legitimate CDN hostname never appeared in test or review. Two issues consequently record inbound media as working: #1315's Context section states "Inbound media already works … downloads via the SSRF-gated download_file", and #468 item 3d treats download_file as a working prerequisite.

No ops workaround exists — the allowlist is a hardcoded module constant with no env override.

Acceptance Criteria

• An inbound WhatsApp image, document, and audio attachment each land in the agent's workspace as real bytes, verified end-to-end against a live Twilio sender
• _TWILIO_MEDIA_ALLOWED_HOST_SUFFIXES accepts Twilio's media CDN (.twiliocdn.com) alongside .twilio.com
• The redirect target is still validated, never blindly followed — follow_redirects=False plus the explicit allowlist check stays; only the allowlist widens
• Anti-spoof property preserved: eviltwiliocdn.com, twiliocdn.com.evil.com, and any non-https scheme are still refused — the existing feat(channels): WhatsApp via Twilio — per-agent binding (#299) #463 spoof tests keep passing, with CDN-shaped spoofs added
• New unit test covering the real-world path: api.twilio.com → 302 → mms.twiliocdn.com → 200, bytes returned
• Download failure stays non-fatal and honestly reported: the agent still receives the message, and the placeholder says the download failed rather than implying the file arrived
• Decide explicitly whether the allowlist gets an env override so a future Twilio host change is an ops fix rather than a release — record the decision either way
• docs/memory/feature-flows/whatsapp-integration.md inbound-media section reflects the actual redirect chain
• The stale "inbound media already works" claim in feat(whatsapp): outbound media attachments — deliver ChannelResponse.files to WhatsApp users via Twilio MediaUrl #1315 is corrected (a comment on the closed issue is enough)

Technical Notes

• Primary file: src/backend/adapters/whatsapp_adapter.py — constant at line 44, _is_twilio_media_url at ~151, download_file at ~647 (verified on dev)
• The suffix match (host.endswith(s) against a leading-dot suffix) is already spoof-safe; adding ".twiliocdn.com" to the tuple preserves that property
• The redirect is followed without auth (the signed CDN URL carries its own credentials) — correct as-is, should not change
• Twilio may also serve media.twiliocdn.com; a .twiliocdn.com suffix rather than a single fixed host covers both
• Prerequisite for WhatsApp voice-note transcription — without it the transcription branch would have nothing to transcribe

Related

• Introduced by feat(channels): WhatsApp via Twilio — per-agent binding (#299) #463 (WHATSAPP-001 Phase 1, feat: WhatsApp channel adapter via Twilio — per-agent integration (WHATSAPP-001) #299)
• Blocks feat(whatsapp): voice-note transcription — extend the transcription branch beyond Telegram (#468 item 3d) #1933 — WhatsApp voice-note transcription (feat(whatsapp): Phase 3 advanced features — templates, SMS, interactive, voice (WHATSAPP-001 Phase 3, #299 follow-up) #468 item 3d)
• Corrects the inbound-media premise asserted in feat(whatsapp): outbound media attachments — deliver ChannelResponse.files to WhatsApp users via Twilio MediaUrl #1315