feat(aws): native Bedrock batch inference support - #1117
Draft
Islam Elkadi (islamelkadi) wants to merge 1 commit into
Draft
feat(aws): native Bedrock batch inference support#1117Islam Elkadi (islamelkadi) wants to merge 1 commit into
Islam Elkadi (islamelkadi) wants to merge 1 commit into
Conversation
Skeleton (signatures + docstrings only) proposing BedrockBatchManager and an optional LangGraph BedrockBatchNode for native Bedrock batch inference via the Converse API format. Method bodies raise NotImplementedError to paint a design picture for maintainers.
Islam Elkadi (islamelkadi)
marked this pull request as draft
June 22, 2026 04:12
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A design-preview skeleton for native Amazon Bedrock batch inference using the Converse API format, as proposed in issue #1116 .
This PR sketches the public surface that would close that gap. Every method body is
raise NotImplementedError— the goal is to align on shape, naming, and placement with maintainers before implementing.What's included
langchain_aws/batch/_formatting.py— internal LangChain ⇄ Converse JSONL conversion + client-side validation, reusing_messages_to_bedrock/_parse_responsefrombedrock_converse.py.langchain_aws/batch/manager.py—BedrockBatchManager(plain orchestration class) andBatchJob(pydantic handle). Lifecycle:submit/get_status/get_results/cancel.langchain_aws/batch/node.py— optional LangGraphBedrockBatchNode(submit → checkpoint viainterrupt→ resume → results).langchain_aws/__init__.pyand a new optionalbatchextra (langgraph) inpyproject.toml/uv.lock.Design decisions (seeking feedback)
BedrockBatchManageris a plain class, not a pydantic model — it's an imperative orchestrator (not a Runnable), so it mirrorsAmazonS3Vectors: clients built in__init__viacreate_aws_client, with optional client injection for testing.ChatBedrockConverserather than re-declaring the credential block.BatchJobstays pydantic and is threaded through the lifecycle methods (not just returned), so it serializes into LangGraph state and carries the context (output_s3_uri,record_count) needed to locate/validate results.BedrockBatchNode, soimport langchain_aws.batchstays light and raises a friendly install hint only on construction.submit()fails fast in four stages (client-side validation → S3 upload →CreateModelInvocationJob→ a brief infra-verification poll) so a misconfigured job surfaces in seconds, not hours.Intentionally NOT in this PR
raise NotImplementedError).Areas that need careful review
s3client + first S3 object I/O in the package. Repo-wide there is no existing"s3"client or S3 helper (only ans3vectorsclient), so the credential path,s3://parsing, and output-file discovery in_upload_jsonl/_iter_output_linesare net-new surface area.BatchJob(strict) rather than a raw ARN string — is that the right contract, or should they acceptstr | BatchJob?langgraph>=1.0.0lower bound for thebatchextra.submit()blocking forverify_seconds(default 30s) — acceptable, or should verification be opt-in / async?Additional food for thought
Likely out of scope due to the potential requirement of external infrastructure:
That being said, in general does this direction align with the roadmap? Happy to iterate on the design here before filling in the implementation.