bug(uploads): every ZIP attachment is rejected as 'file type mismatch' — libmagic 5.46 (trixie) misdetects ZIP via from_buffer #3046

Summary

ZIP attachments — accepted since #2152 and documented as supported in the user docs and chat API reference — are rejected on every channel (Telegram, web chat, Slack, WhatsApp, POST /chat / /task files) with — rejected (file type mismatch). The agent never receives the file; it only sees the rejection text in its prompt. Root cause is not Trinity code but the backend base image: python:3.13-slim now resolves to Debian 13 (trixie), which ships libmagic1t64 1:5.46-5, and libmagic 5.46 has a regression where buffer-based detection (magic_buffer, i.e. python-magic from_buffer) returns application/octet-stream for a valid ZIP while file-based detection returns application/zip. upload_service.process_file_uploads uses from_buffer, so the declared application/zip never matches and the strict-mismatch branch drops the file. Same floating-base-tag class as #1823 (tzdata-legacy).

Practical impact: since PDF is refused by UNSUPPORTED_MIMES by policy, the documented "send it as a ZIP instead" route is dead too, so a user has no way to hand a document to an agent over a channel except pasting text or a screenshot.

Component

Backend / services/upload_service.py (all inbound file paths) + docker/backend/Dockerfile

Priority

P1 — documented feature fully broken, no config workaround (the mismatch branch has no override, and python-magic cannot be disabled without dropping validation for every type).

Error

WARNING services.upload_service [UPLOAD] MIME mismatch for example.zip: declared=application/zip, detected=application/octet-stream

Prompt injected to the agent: example.zip — rejected (file type mismatch)

Location

• File: src/backend/services/upload_service.py
• Line: ~204–225 (magic-byte MIME validation block, detected_mime = magic.from_buffer(data, mime=True) → strict-mismatch continue)
• File: docker/backend/Dockerfile line 29 (apt-get install … libmagic1 … on an unpinned python:3.13-slim base)

Root Cause

Reproduced inside a backend container built from current dev (ebc2bfb4, /etc/debian_version = 13.7, python-magic 0.4.27, libmagic 5.46):

import zipfile, io, magic, tempfile
b = io.BytesIO(); z = zipfile.ZipFile(b, 'w', zipfile.ZIP_DEFLATED); z.writestr('doc.pdf', 'x'*40000); z.close(); data = b.getvalue()
magic.from_buffer(data)              # 'data'
magic.from_buffer(data, mime=True)   # 'application/octet-stream'   <-- what upload_service sees
p = tempfile.mktemp(suffix='.zip'); open(p, 'wb').write(data)
magic.from_file(p)                   # 'Zip archive data, made by v2.0 UNIX, ...'
magic.from_file(p, mime=True)        # 'application/zip'

Stored (uncompressed) ZIPs misdetect the same way. magic.from_buffer(b'%PDF-1.4 ...', mime=True) still returns application/pdf, so this is ZIP-specific. Upstream references: Debian bug #1115393 ("libmagic miscategorizes … simple archive from byte buffer", regression between 5.45 and 5.46), ahupp/python-magic#166, file project bug 622.

The Dockerfile pins nothing about the Debian release, so a rebuild on any host silently picked up trixie's libmagic when Docker Hub repointed the python:3.13-slim tag — nobody changed a line of Trinity.

Reproduction Steps

1. Build the backend image from current dev (docker compose -f docker-compose.prod.yml build backend); confirm docker exec trinity-backend cat /etc/debian_version starts with 13.
2. Attach any .zip to an agent via web chat, or send one to a Telegram-bound agent, or POST /api/agents/{name}/chat with files=[{name:"a.zip", mimetype:"application/zip", …}].
3. Backend logs [UPLOAD] MIME mismatch … declared=application/zip, detected=application/octet-stream; the agent's prompt carries a.zip — rejected (file type mismatch); nothing is written to /home/developer/uploads/.
4. tests/ — the existing feat(uploads): accept application/zip in chat file uploads #2152 ZIP-acceptance tests presumably mock python-magic or run on a bookworm host; a test that runs magic.from_buffer on a real ZIP inside the backend image would have caught this.

Suggested Fix

Any one of these closes it; (a) is the most robust because it stops depending on libmagic's buffer path at all for the ZIP case, and (c) is what #1823 did for the same class of drift:

# (a) upload_service.py — trust the container signature for a declared ZIP before consulting libmagic
_ZIP_MAGIC = (b"PK\x03\x04", b"PK\x05\x06", b"PK\x07\x08")
if mimetype == "application/zip" and data.startswith(_ZIP_MAGIC):
    actual_mime = "application/zip"
else:
    detected_mime = magic.from_buffer(data, mime=True)
    ...

# (b) detect via a temp file instead of a buffer (libmagic 5.46's file path is correct)
with tempfile.NamedTemporaryFile(delete=True) as tf:
    tf.write(data); tf.flush()
    detected_mime = magic.from_file(tf.name, mime=True)

# (c) docker/backend/Dockerfile — pin the Debian codename so the base tag cannot drift again
FROM python:3.13-slim-bookworm

Also worth adding: a unit test that builds a real ZIP in memory and asserts process_file_uploads accepts it with python-magic un-mocked, so the next libmagic regression fails CI instead of production. The same from_buffer call is used by the outbound file-share route (_INLINE_SAFE_TYPES disposition, ent#461) — worth checking whether shared ZIPs now fall to attachment/octet-stream there too (harmless direction, but same cause).

Environment

• Trinity version: dev @ ebc2bfb4 (2026-09-27); observed on an instance updated 2026-09-27
• Backend image: python:3.13-slim → Debian 13.7 (trixie), libmagic1t64 1:5.46-5, libmagic-mgc 1:5.46-5, python-magic 0.4.27
• Docker 27.x, Ubuntu 22.04 host

Related

• feat(uploads): accept application/zip in chat file uploads #2152 (ZIP acceptance in chat uploads — the feature this breaks)
• bug: legacy IANA timezone aliases (Europe/Kiev et al.) crash schedule creation with a 500 — pytz accepts, APScheduler's zoneinfo lookup can't resolve them inside the shipped image #1823 (tzdata-legacy — same floating python:3.13-slim bookworm→trixie drift)
• feat: configurable file/image exchange policy for public-channel agents #499 (configurable file/image exchange policy for public-channel agents — the PDF half of the same user complaint)
• docs/user-docs/api-reference/chat-api.md, docs/memory/feature-flows/web-chat-file-upload.md (both document ZIP as accepted)