Repository navigation
feat(runtime): materialize native PDF inputs for verified PDF-capable providers #3164
Description
Problem
Maka synchronizes PDF-input metadata and exposes resolveModelPdfSupport, but the runtime never consumes that capability. AiSdkBackend.appendImageParts filters attachments to kind === image; a PDF remains only a text placeholder even when the selected provider/model has a verified native PDF contract.
The storage side is already present: PDF attachments have a canonical kind and MIME type, are persisted as Session artifacts, and can be read through the existing bounded AttachmentByteReader. The missing boundary is provider request materialization.
Provider research
AI SDK encoding support and provider support are separate facts. The current locked SDK produces these request shapes from one generic file part:
- OpenAI-compatible Chat:
type: filewithfilenameand Base64file_data; - Anthropic Messages:
type: documentwith a Base64application/pdfsource; - OpenAI Responses:
type: input_filewithfilenameand Base64file_data.
That proves the SDK can lower a generic file part. It does not prove every OpenAI-compatible provider accepts the resulting wire.
Verified first-party contracts
- Anthropic Messages: all active Claude models support PDF processing. The API accepts URL, Base64, or Files API references. The complete request is limited to 32 MB; PDFs must be standard and unencrypted; the page limit is 600, or 100 when the request context window is below 1M tokens.
- OpenAI: native Chat Completions and Responses accept PDF file inputs. Responses uses
input_file; PDF processing includes extracted text and page images. Each file and the combined files in one request must remain below 50 MB.
Not yet verified
- Generic OpenAI-compatible providers: the adapter can emit the OpenAI Chat
file_datashape, but each provider must opt in only after its own contract is verified. Unknown compatible providers must remain unsupported. - Alibaba Token Plan /
qwen3.8-max: Alibaba Chat documentation lists text, image, audio, and video content but nofile/file_dataPDF shape. Alibaba Responses documentsinput_file, but currently states that this content type is supported only byqwen3.5-ocrand requiresfile_url. Thereforeqwen3.8-maxis not an acceptance target for this issue. Its generatedpdfmodality is insufficient authority for sending local PDF bytes. - Alibaba PDF support, if added later, needs a separate provider contract. If the service continues to require
file_url, it also needs a safe upload/temporary-URL design rather than silently exposing local files.
Scope status
This is an independent Runtime attachment-capability issue. It is not a follow-up requirement for Qwen3.8 support, #3156, #3157, #3162, or #3163, and it does not block those deliverables. Alibaba/Qwen remains in this issue only as a negative boundary showing why generic OpenAI-compatible PDF support must not be inferred without provider evidence.
Desired outcome
A model on a verified PDF-capable provider/wire receives an attached PDF as a provider-native file part on the current Turn and during durable replay. Unsupported, unknown, or unverified provider/model combinations continue to receive the bounded placeholder and never receive raw PDF bytes.
Proposed delivery
First slice: Anthropic and native OpenAI
- Replace the unused boolean-only assumption with an explicit provider/wire PDF-input contract. Do not infer generic-compatible support from a model name or generated modality alone.
- Carry the resolved contract into
AiSdkBackendfrom the production Runtime Host composition. The former Headless production path no longer exists on currentmainand is not part of this slice. - Materialize
AttachmentRef.kind === pdfthrough the existing attachment byte reader as one generic AI SDKfilepart with exactapplication/pdf, filename, and bytes. Let the verified AI SDK provider adapter perform the wire-specific lowering. - Cover native Anthropic Messages and native OpenAI Chat/Responses. Keep all generic OpenAI-compatible providers disabled unless separately verified.
- Keep current Turn, RuntimeEvent replay, StoredMessage fallback, steering, compaction replacement, and restart behavior consistent.
Budget and failure policy
- Add a PDF sub-budget and a combined raw-binary attachment budget. Separate independent image/PDF limits are not sufficient because Anthropic applies its 32 MB cap to the complete request and Base64 adds roughly one-third overhead.
- Preserve the existing 12 MB image sub-limit. Choose the PDF and combined limits from measured encoded request size, leaving room for text, schemas, JSON, and other attachments.
- Deduplicate budget decisions by durable occurrence across provider steps, as the image path does today.
- Missing, unreadable, oversized, unsupported, or locally invalid PDFs must degrade to a safe bounded explanation; PDF bytes must never enter transcript text, logs, RuntimeEvents, or diagnostics.
- Preflight parsing for encrypted/page-limit PDFs is tracked in feat(runtime): preflight native PDF inputs against provider document constraints #3284; bounded recovery after provider rejection is tracked in discussion(runtime): define bounded recovery for provider-rejected native PDF inputs #3285. Neither is a merge gate for this first slice. Ordinary provider errors remain governed by the existing bounded
ModelFailurehandling.
Acceptance criteria
resolveModelPdfSupportis no longer dead metadata; the runtime has an explicit verified provider/wire gate.- Native Anthropic receives a
documentblock with Base64application/pdfdata through the SDK. - Native OpenAI Chat receives
type: file; native OpenAI Responses receivestype: input_file. - A generic OpenAI-compatible connection does not receive PDF bytes by default.
- Current Turn, RuntimeEvent replay, StoredMessage fallback, steering, compaction, and restart all present the same durable PDF occurrence without duplicating the user message.
- Image and PDF attachments respect both their subtype caps and the combined binary-request budget.
- Unsupported models and local attachment failures keep the safe placeholder. Provider request failures use the existing bounded
ModelFailurepath; this slice does not add PDF-specific automatic retry. - Focused live verification covers at least one native Anthropic model and one native OpenAI model when protected credentials are available.
Non-goals
- Claiming PDF support for
qwen3.8-maxor another generic-compatible model without provider evidence. - Uploading local PDFs to a third-party file service or creating public temporary URLs.
- OCR fallback for models without native PDF support.
- Automatically extracting PDF text into the prompt.
- Treating arbitrary Office or binary files as provider-safe PDF/document inputs.
- Preflight parsing for encrypted PDFs/page counts (feat(runtime): preflight native PDF inputs against provider document constraints #3284) or automatic PDF-specific retry after provider rejection (discussion(runtime): define bounded recovery for provider-rejected native PDF inputs #3285).
References
- OpenAI file inputs: https://developers.openai.com/api/docs/guides/file-inputs
- Anthropic PDF support: https://platform.claude.com/docs/en/build-with-claude/pdf-support
- Alibaba Chat API: https://help.aliyun.com/zh/model-studio/qwen-api-via-openai-chat-completions
- Alibaba Responses API: https://help.aliyun.com/zh/model-studio/qwen-api-via-openai-responses
Disclosure: this issue was drafted and revised with OpenAI Codex assistance and reviewed and approved by the human contributor.
Activity
[-]feat(runtime): materialize native PDF inputs for PDF-capable models[/-][+]feat(runtime): materialize native PDF inputs for verified PDF-capable providers[/+]MoonOld commentedon Aug 19, 2026
Scope clarification: keeping this issue open because the Runtime PDF-materialization gap is real, but it is independent of the Qwen3.8 work. It is not a follow-up requirement for #3156/#3157 and does not block the Alibaba Responses or Harness work in #3162/#3163. Qwen remains mentioned only as an explicitly unsupported generic-compatible case.
tianrking commentedon Aug 20, 2026
I'd like to take the first delivery slice: an explicit PDF input contract for native Anthropic Messages and native OpenAI Chat/Responses, with generic-compatible routes denied by default. I'll reuse the durable attachment reader, keep current-turn and replay materialization consistent, and add subtype plus combined binary budgets with regression coverage for unsupported/read-failure/oversize paths.\n\nThis comment was prepared by OpenAI Codex for @tianrking. The human contributor requested this work and remains the contributor of record.
MoonOld commentedon Aug 20, 2026
Thanks for taking this. The proposed scope matches the first delivery slice in #3164. Please proceed with #3266; we’ll track the implementation and review there.
MoonOld commentedon Aug 20, 2026
Scope update after reviewing #3266: I narrowed this first slice to the production Runtime Host path and local attachment/wire failure handling. The former Headless path is absent from current
main. Encrypted/page-count preflight and PDF-specific retry after provider rejection are now explicit follow-up concerns rather than merge gates. The remaining implementation blocker is durable replay of steering attachments.MoonOld commentedon Aug 20, 2026
Follow-up tracking is now concrete:
Both remain independent of #3266's first-slice merge gate.