Skip to content

Inference Completion building block (replaces Conversation) - #114

Open
WhitWaldo wants to merge 1 commit into
dapr:mainfrom
WhitWaldo:ai-inference-proposal
Open

WhitWaldo wants to merge 1 commit into
dapr:mainfrom
WhitWaldo:ai-inference-proposal

Conversation

@WhitWaldo

Copy link
Copy Markdown
Contributor

This proposal introduces Inference Completion, a deliberately breaking, provider-neutral replacement for the preview Conversation building block. This proposal supersedes #108 and #113.

##Key Changes

  • Replaces the OpenAI-shaped messages/choices with ordered text/multimodal content, instructions, tool calls/results, and provider continuation state
  • Single always-streamed CompleteAlpha1 API coverage autoregressive, diffusion and hybrid models, with optional stored-transcript references.
  • Provider-native async batch submission, status, cancellation and streamed results
  • Component-owned per-model routing and model discovery/pricing APIs
  • Reusable runtime conventions (model defaults, access policies, resiliency, PII scrubbing, observability, idempotency, cancellation) adopted by forthcoming sibling proposals
  • Migration path from ConverseAlpha1/ConverseAlpha2

This proposal anchors a coordinated but independently proposed inference series (Decisions, Embeddings, Reranking, Image/Video Generation, etc) opening the door for Dapr to more readily adapt to new models.

The media input shape was proposed by @JoshVanL

Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant