feat(aws): native async for ChatBedrockConverse, ChatBedrock and BedrockEmbeddings

[roi hezkiyahu (roihezkiRaven)]

Closes #663.

Why

ainvoke/astream/aembed_* bridge to the synchronous boto3 client with run_in_executor. The thread is held for the whole model call, so concurrency is capped at the default executor size, min(32, os.cpu_count() + 4) — the ceiling Lucas Kim (@kimnamu) measured in #663, and the cause of the latency growth Norman Luengo (@norman-luengo) reported.

Against a local fake bedrock-runtime with 500 ms of injected latency, on a 12-core box (16-thread default executor):

concurrent ainvoke	today	this PR	max in flight, before → after
8	0.52 s	0.61 s	8 → 8
32	1.12 s	0.55 s	16 → 32
64	2.16 s	0.56 s	16 → 64
128	4.21 s	0.64 s	16 → 128

Below the ceiling there is no gain, and slightly more overhead; above it the executor serializes into waves while the native path stays flat. A fake endpoint is used deliberately: against real Bedrock the wall clock is dominated by per-account service quotas, so it measures the account rather than the client.

Try it

import asyncio
from langchain_aws import ChatBedrockConverse

async def main():
    async with ChatBedrockConverse(
        model="us.amazon.nova-micro-v1:0", region_name="us-east-1",
        use_async_transport=True, max_tokens=8,
    ) as model:
        print(await model.ainvoke("hi"))
        async for chunk in model.astream("count to three"):
            print(chunk.text, end="", flush=True)
        await asyncio.gather(*(model.ainvoke("hi") for _ in range(32)))

asyncio.run(main())

Integration tests covering the same ground live in tests/integration_tests/test_async_client.py; they use Nova Micro and Titan v2 with one-token prompts, so a full run costs a fraction of a cent.

Why not aiobotocore

The issue asks for aiobotocore/aioboto3 support, but neither can be a dependency of this package:

• aiobotocore 3.9.0 (latest) pins botocore>=1.43.3,<1.43.57; this package requires boto3>=1.43.64. pip install langchain-aws aiobotocore has no solution today.
• That is structural, not transient. aiobotocore's ceiling trails botocore by design, while this package's floor moves forward for new AWS APIs (1.42.42 → 1.43.32 → 1.43.64, the last in feat(aws): add DynamoDBVectorStore backed by DynamoDB vector indexes #1200 for DynamoDB vector indexes). The ranges cross and stay crossed.
• aioboto3 is further behind: its latest release pins aiobotocore==2.25.1, i.e. botocore<1.40.62.

So BedrockAsyncClient signs with botocore and sends with httpx — the approach Anthropic's SDK takes, and the one ChatAnthropicBedrock already benefits from for Anthropic models only. This covers the rest of the surface (Nova, Titan, Cohere, Llama, embeddings). Event stream framing is parsed by botocore's own EventStreamBuffer rather than reimplemented.

No new dependencies are declared — httpx is already in the tree via langchain-core. async_client is duck-typed on the boto3 method names, so anyone who prefers aiobotocore can pass their own entered client and manage its botocore pin themselves.

Reading it

Five commits, in dependency order:

1. feat: add a native async Bedrock runtime client — the new async_client.py and _async_transport.py, plus a factory in utils.py. Purely additive (+919/−3); nothing else in the package changes behavior.
2. feat: ... ChatBedrockConverse — the bulk of the diff is an extraction: _generate and _stream duplicated ~50 lines of request building verbatim, now shared. _agenerate/_astream reuse it.
3. feat: ... BedrockEmbeddings — same shape: _embedding_func split into request building and response parsing so both paths share the per-provider logic.
4. feat: ... ChatBedrock and BedrockBase — same again for the Invoke API.
5. test: cover the native async Bedrock paths — 52 unit tests and 9 integration tests.

Commits 2–4 are refactor-then-reuse. The refactors are meant to be behavior-preserving, and the check for that is mechanical: all 1307 pre-existing unit tests pass untouched, and each model has a test asserting the sync and async paths send byte-identical requests, so the two cannot drift apart.

Opt-in and additive throughout: with neither use_async_transport nor async_client set, every async method keeps its current executor behavior.

Decisions worth a second opinion

• Retries. The client retries transient errors with jittered backoff honoring retries["max_attempts"]. This is not botocore's adaptive mode and shares no retry quota; retries["mode"] is ignored and documented as such. I included retries because boto3 retries by default, so shipping none would silently regress the async path against the sync one. Happy to drop them if you would rather retries stayed with the caller.
• Cross-loop reuse raises. An httpx pool binds to the loop that first drives it. Sharing one client across loops previously hung forever with nothing raised (reproduced: 25 threads each running asyncio.run on one shared model — 14 of 75 calls completed, the rest deadlocked in selectors.select). It now raises immediately with an actionable message. Louder than before, and deliberate, but it is a behavior choice.
• Bearer tokens are refused, not ignored. The built client signs SigV4 only, so use_async_transport=True together with a bedrock_api_key raises rather than silently authenticating as a different identity. Same approach as fix(aws): Raise error on unsupported guardrail header usage with Mantle #1225.

Known deviations from boto3

• Bearer-token authentication is unsupported on the async path (see above).
• Responses are the raw JSON, so members botocore would drop as unmodeled may be present — usage is a superset of the sync path's.
• ReasoningContentBlock.redactedContent stays a base64 str rather than bytes. It is passed through opaquely so it round-trips, but it differs from the sync path.
• Header-bound Invoke members are transcribed from the service model into _INVOKE_HEADER_MEMBERS. If that mapping drifts upstream, guardrails would quietly stop being applied — worth a check by someone who knows the model well.

Testing done

• 1324 unit tests (52 new), --disable-socket; make lint clean, and mypy clean under each of 3.10 / 3.11 / 3.12 / 3.13.
• 9 integration tests against real Bedrock, all passing.
• Verified against botocore's own service model that Converse/ConverseStream have no header-bound members while the two Invoke operations have eight, and that InvokeModelWithResponseStream moves accept to X-Amzn-Bedrock-Accept.

---

Disclaimer: this contribution was developed with the assistance of AI agents (Claude), including adversarial review passes that found several defects in my own first draft (silently dropped guardrail headers, swallowed event-stream error frames, and a cross-loop deadlock). The analysis, benchmarks and code were produced under my direction, and I have reviewed the result.

[roi hezkiyahu (roihezkiRaven)]

Hi Michael Chin (@michaelnchin), is this something that we can push somehow? Would love to get this out with your help :)
True async is extremely important at scale; we see this as a blocker for multiple high-throughput agents we run.