Skip to content

Don't send binary grpc-trace-bin header on InvokeBinding calls - #1900

Closed
DSpirit wants to merge 1 commit into
dapr:masterfrom
DSpirit:fix/invoke-binding-grpc-trace-bin
Closed

DSpirit wants to merge 1 commit into
dapr:masterfrom
DSpirit:fix/invoke-binding-grpc-trace-bin

Conversation

@DSpirit

@DSpirit DSpirit commented Aug 21, 2026 •

Copy link
Copy Markdown

Closes #1897

InvokeBindingAsync was sending the binary grpc-trace-bin header (added in #1858) on every call. The sidecar forwards a call's incoming metadata verbatim into the invoked binding component's metadata, and components like Azure Blob Storage and Service Bus require that metadata to be valid UTF-8 text. Since grpc-trace-bin's bytes come from the random trace/span ID, this caused intermittent failures - Blob request signing broke (403 AuthenticationFailed) and Service Bus rejected the message outright (
ot a valid UTF-8 string).

This strips any binary (-bin suffixed) gRPC metadata from the call options right before invoking InvokeBinding, so it can't be forwarded into binding component metadata regardless of source. raceparent/ racestate are still sent since they're plain text and don't cause this issue.

Root cause is really in the runtime (InvokeBinding shouldn't forward binary metadata into component metadata at all) - tracked in dapr/dapr#10387. This is a client-side mitigation so .NET SDK users aren't affected while that's being worked on.

Checklist

  • Code compiles correctly
  • Created/updated tests
  • Extended the documentation

Since v1.18.5, the SDK attaches a grpc-trace-bin binary metadata header
to every outgoing gRPC call. The sidecar forwards InvokeBinding call
metadata verbatim into the output binding component's metadata (e.g.
Azure Blob x-ms-meta-*, Service Bus application properties), which
must be valid UTF-8. The binary header broke Blob request signing
(403 AuthenticationFailed) and violated Service Bus's UTF-8
requirement.

Strip any binary ("-bin" suffixed) gRPC metadata from the call options
right before invoking InvokeBinding, leaving the shared trace-context
helper used by every other Dapr client untouched. traceparent/tracestate
are still sent since they're valid UTF-8 text.
@DSpirit

DSpirit commented Aug 21, 2026

Copy link
Copy Markdown
Author

For reference, the runtime-side fix is now up for review: dapr/dapr#10388 (fixes dapr/dapr#10387 by skipping -bin\ suffixed metadata keys in InvokeBinding's forwarding loop). This client-side change remains useful as defense-in-depth once that merges - protects users on older runtimes and avoids sending metadata that serves no purpose on this call.

@WhitWaldo

Copy link
Copy Markdown
Contributor

This PR is astonishingly similar to #1901 but the other adds a null check on the headers, so I'm going to take it. Thank you for taking the time to contribute this though!

@WhitWaldo WhitWaldo closed this Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Publish to servicebusqueue binding throws error invoking output binding <binding name>: (connlost): not a valid UTF-8 string

2 participants