perf(trace): write agentless JSON directly - #2454
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits intoSep 10, 2026
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits into
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits into
Conversation
Contributor
|
✅ All CI checks and tests passed. 🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 07eafe9 | Docs | View more details | Give us feedback! |
BenchmarksComparisonBenchmark execution time: 2026-09-10 21:36:59 Comparing candidate commit 07eafe9 in PR branch Found 7 performance improvements and 2 performance regressions! Performance is the same for 119 metrics, 0 unstable metrics.
|
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
BridgeAR
marked this pull request as ready for review
September 9, 2026 21:22
BridgeAR
force-pushed
the
BridgeAR/2026-09-03-agentless-hex-ids
branch
from
September 9, 2026 22:07
03bcf5c to
5853139
Compare
yannham
approved these changes
Sep 10, 2026
yannham
left a comment
yannham
left a comment
Contributor
There was a problem hiding this comment.
Overall LGTM. The code is not much more complicated than the serde one. I am surprised by the result, I would somehow wishfully assume that the compiler and LLVM would make the serde bloat somehow less noticeable than that.
BridgeAR
force-pushed
the
BridgeAR/2026-09-03-agentless-direct-json
branch
from
September 10, 2026 09:50
3906b52 to
81b9974
Compare
Member
Author
|
/merge |
|
View all feedbacks in Devflow UI.
Progress updates will be sent here directly.
PR already in the queue with status waiting
The following PRs won't be merged. The current stack is: |
BridgeAR
force-pushed
the
BridgeAR/2026-09-03-agentless-hex-ids
branch
from
September 10, 2026 12:19
22bda7d to
7525711
Compare
Base automatically changed from
BridgeAR/2026-09-03-agentless-hex-ids
to
main
September 10, 2026 13:36
Write the fixed agentless payload structure directly while retaining serde_json for all dynamic keys and values. With production-deduplicated spans, position-balanced Criterion runs on macOS arm64 improved the fixed-ID baseline by 35.44% for common HTTP traces and 20.18% for span-link traces.
An arbitrary two-second deadline can expire under parallel CI load before the loopback gRPC exchange completes.
BridgeAR
force-pushed
the
BridgeAR/2026-09-03-agentless-direct-json
branch
from
September 10, 2026 21:07
1b1fbb8 to
07eafe9
Compare
Member
Author
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
gh-worker-dd-devflow-36fce6
Bot
added
mergequeue-status: queued
mergequeue-status: in_progress
and removed
mergequeue-status: queued
labels
gh-worker-dd-mergequeue-cf854d
Bot
deleted the
BridgeAR/2026-09-03-agentless-direct-json
branch
September 10, 2026 23:08
paullegranddc
added a commit
that referenced
this pull request
Sep 24, 2026
…ibdd-data-pipeline, libdd-li... (#2571) <!-- release-proposal-inputs: {"crates":"libdd-capabilities-impl, libdd-common, libdd-data-pipeline, libdd-library-config, libdd-profiling-heap-allocator, libdd-remote-config, libdd-sampling, libdd-shared-runtime, libdd-telemetry, libdd-tinybytes, libdd-trace-utils","main_start_ref":"","level_overrides":"","bypass_standard_checks":false} --> # Release proposal for libdd-capabilities-impl, libdd-common, libdd-data-pipeline, libdd-library-config, libdd-profiling-heap-allocator, libdd-remote-config, libdd-sampling, libdd-shared-runtime, libdd-telemetry, libdd-tinybytes, libdd-trace-utils and their dependencies This PR contains version bumps based on public API changes and commits since last release. ###⚠️ Crates left out of this proposal affected by its major bumps These publishable workspace crates are not part of this release but their dependency requirement was rewritten on this branch while their published version still requires the old major. If they are a dependency on your deployment not including them in the release could result in duplicate packages or symbol incompatibility. - `libdd-capabilities` `3.0.1` → `4.0.0` affects: `libdd-crashtracker`, `libdd-ffe`, `libdd-live-debugger` - `libdd-capabilities-impl` `5.0.0` → `6.0.0` affects: `libdd-crashtracker`, `libdd-live-debugger`, `libdd-tracer-flare` - `libdd-common` `6.0.0` → `7.0.0` affects: `libdd-crashtracker`, `libdd-ffe`, `libdd-http-client`, `libdd-ipc`, `libdd-live-debugger`, `libdd-profiling`, `libdd-tracer-flare` - `libdd-data-pipeline` `10.0.0` → `11.0.0` affects: `libdd-live-debugger` - `libdd-remote-config` `5.0.0` → `6.0.0` affects: `libdd-ffe`, `libdd-live-debugger`, `libdd-tracer-flare` - `libdd-telemetry` `8.0.0` → `9.0.0` affects: `libdd-crashtracker` - `libdd-trace-stats` `9.0.0` → `10.0.0` affects: `libdd-ipc` - `libdd-trace-utils` `12.0.0` → `13.0.0` affects: `libdd-tracer-flare` ## libdd-capabilities **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-capabilities-v4.0.0` ### Commits - feat(trace-exporter)!: add API to force flush stats on the trace exporter (#2538) - refactor!: apply small timeout pooling strategy to libdd-http-client as well (#2449) - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-common **Next version:** `7.0.0` **Semver bump:** `major` **Tag:** `libdd-common-v7.0.0` ### Commits - build: enforce workspace-level dependency declarations (#2514) - chore!: remove unused dependencies (v2) (#2511) - chore: move all remaining external deps to workspace-level dependencies (#2476) - chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318) - fix!: benchmark was doing many samples for deterministic heap usage (#2465) ## libdd-ddsketch **Next version:** `1.1.2` **Semver bump:** `patch` **Tag:** `libdd-ddsketch-v1.1.2` ### Commits - chore: move all remaining external deps to workspace-level dependencies (#2476) - chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318) ## libdd-profiling-heap-sampler **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-profiling-heap-sampler-v1.1.0` ### Commits - chore: move all remaining external deps to workspace-level dependencies (#2476) - refactor: migrate to workspace dependencies, phase 4 (#2296) - feat(profiling): add USDT semaphores to skip slow path around heap profile sampler (#2266) ## libdd-tinybytes **Next version:** `1.1.4` **Semver bump:** `patch` **Tag:** `libdd-tinybytes-v1.1.4` ### Commits - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-trace-protobuf **Next version:** `5.0.1` **Semver bump:** `patch` **Tag:** `libdd-trace-protobuf-v5.0.1` ### Commits - chore: move all remaining external deps to workspace-level dependencies (#2476) - fix(build): build script change detection (#2467) ## libdd-capabilities-impl **Next version:** `6.0.0` **Semver bump:** `major` **Tag:** `libdd-capabilities-impl-v6.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 ### Commits - refactor!: apply small timeout pooling strategy to libdd-http-client as well (#2449) - fix(sidecar)!: stabilize the sidecar for macos usage (#2475) - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-profiling-heap-allocator **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-profiling-heap-allocator-v1.1.0` ### Commits - refactor(benchmarks): measure allocator thread CPU time (#2500) - refactor: migrate to workspace dependencies, phase 4 (#2296) - feat(profiling): add USDT semaphores to skip slow path around heap profile sampler (#2266) ## libdd-library-config **Next version:** `4.1.0` **Semver bump:** `minor` **Tag:** `libdd-library-config-v4.1.0` ### Commits - chore!: remove unused dependencies (v2) (#2511) - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-trace-normalization **Next version:** `4.1.0` **Semver bump:** `minor` **Tag:** `libdd-trace-normalization-v4.1.0` ### Commits - chore: move all remaining external deps to workspace-level dependencies (#2476) - chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318) ## libdd-remote-config **Next version:** `6.0.0` **Semver bump:** `major` **Tag:** `libdd-remote-config-v6.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 ### Commits - refactor!: apply small timeout pooling strategy to libdd-http-client as well (#2449) - chore!: remove unused dependencies (v2) (#2511) - fix(rem-cfg)!: remove AsmRawResponseBody (#2540) - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-shared-runtime **Next version:** `5.0.0` **Semver bump:** `major` **Tag:** `libdd-shared-runtime-v5.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 ### Commits - feat(trace-exporter)!: add API to force flush stats on the trace exporter (#2538) - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-trace-utils **Next version:** `13.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-utils-v13.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 ### Commits - fix(trace-utils)!: add forgotten css to agentless v1 encoder (#2557) - fix(otlp): use potential dedupe in otlp serialization path (#2531) - feat(trace-utils)!: add v1-native OTLP encoder brick (#2369) - perf(serialization): bench the actual path used by the trace exporter (#2512) - perf(trace): write agentless JSON directly (#2454) - chore: move all remaining external deps to workspace-level dependencies (#2476) - perf(trace): encode agentless IDs without formatting (#2452) - perf(trace-utils)!: use pooled spans on the send path (#2382) - feat(data-pipeline)!: OTLP gRPC trace export (#2171) - chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318) - fix!: benchmark was doing many samples for deterministic heap usage (#2465) ## libdd-dogstatsd-client **Next version:** `7.0.0` **Semver bump:** `major` **Tag:** `libdd-dogstatsd-client-v7.0.0` ###⚠️ major bump forced due to: - `libdd-common`: ^6.0.0 → ^7.0.0 - `libdd-shared-runtime`: ^4.0.0 → ^5.0.0 ### Commits - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-telemetry **Next version:** `9.0.0` **Semver bump:** `major` **Tag:** `libdd-telemetry-v9.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 - `libdd-shared-runtime`: ^4.0.0 → ^5.0.0 ### Commits - refactor!: apply small timeout pooling strategy to libdd-http-client as well (#2449) - chore!: remove unused dependencies (v2) (#2511) - chore: move all remaining external deps to workspace-level dependencies (#2476) - fix(telemetry): fall back to the proper intake telemetry domain when direct submission is enabled (#2484) ## libdd-sampling **Next version:** `7.0.0` **Semver bump:** `major` **Tag:** `libdd-sampling-v7.0.0` ###⚠️ major bump forced due to: - `libdd-common`: ^5.2.0 → ^7.0.0 - `libdd-trace-utils`: ^10.1.0 → ^13.0.0 ### Commits - chore(benchmarks): remove batched-loop sampling noise (#2498) - chore: move all remaining external deps to workspace-level dependencies (#2476) - fix!: benchmark was doing many samples for deterministic heap usage (#2465) - feat(trace-utils)!: add from owned to SpanText (#2403) - test(sampling): make rate_limiter thread-safety test deterministic (#2354) ## libdd-trace-obfuscation **Next version:** `9.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-obfuscation-v9.0.0` ###⚠️ major bump forced due to: - `libdd-common`: ^6.0.0 → ^7.0.0 - `libdd-trace-utils`: ^12.0.0 → ^13.0.0 ### Commits - feat(data-pipeline)!: refactor agent's /info obfuscation config format (#2490) - fix(obfuscation): keep URLs whose path or fragment holds a bracket (#2527) - fix(obfuscation): stop forcing serde_json/preserve_order on dependents (#2529) - fix(obfuscation): prevent crash and cap recursion in SQL/HTTP obfuscation (#2441) - chore: move all remaining external deps to workspace-level dependencies (#2476) - fix(data-pipeline-ffi): harden agentless config setters and parse obfuscation config once (#2474) ## libdd-trace-stats **Next version:** `10.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-stats-v10.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 - `libdd-shared-runtime`: ^4.0.0 → ^5.0.0 - `libdd-trace-obfuscation`: ^8.0.0 → ^9.0.0 - `libdd-trace-utils`: ^12.0.0 → ^13.0.0 ### Commits - feat(trace-exporter)!: add API to force flush stats on the trace exporter (#2538) - fix(trace-stats): cardinality limit telemetry name (#2559) - feat(data-pipeline)!: refactor agent's /info obfuscation config format (#2490) - chore!: remove unused dependencies (v2) (#2511) - feat(data-pipeline)!: generate agentless trace stats (#2488) - chore: move all remaining external deps to workspace-level dependencies (#2476) - chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318) ## libdd-data-pipeline-core **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-data-pipeline-core-v2.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 - `libdd-trace-obfuscation`: ^8.0.0 → ^9.0.0 - `libdd-trace-utils`: ^12.0.0 → ^13.0.0 ### Commits - refactor!: apply small timeout pooling strategy to libdd-http-client as well (#2449) - feat(data-pipeline)!: generate agentless trace stats (#2488) - perf(data-pipeline): avoid parsing static entity header names (#2457) - chore: move all remaining external deps to workspace-level dependencies (#2476) - perf(trace-utils)!: use pooled spans on the send path (#2382) ## libdd-data-pipeline **Next version:** `11.0.0` **Semver bump:** `major` **Tag:** `libdd-data-pipeline-v11.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 - `libdd-data-pipeline-core`: ^1.0.0 → ^2.0.0 - `libdd-shared-runtime`: ^4.0.0 → ^5.0.0 - `libdd-trace-obfuscation`: ^8.0.0 → ^9.0.0 - `libdd-trace-stats`: ^9.0.0 → ^10.0.0 - `libdd-trace-utils`: ^12.0.0 → ^13.0.0 ### Commits - feat(trace-exporter)!: add API to force flush stats on the trace exporter (#2538) - refactor!: apply small timeout pooling strategy to libdd-http-client as well (#2449) - feat(data-pipeline)!: refactor agent's /info obfuscation config format (#2490) - test(data-pipeline): stop OTLP gRPC post-connect test from hanging (#2549) - chore!: remove unused dependencies (v2) (#2511) - feat(data-pipeline)!: generate agentless trace stats (#2488) - fix(otlp): use potential dedupe in otlp serialization path (#2531) - refactor(benchmarks): isolate enqueue timing in data-pipeline (#2499) - perf(trace): write agentless JSON directly (#2454) - test(data-pipeline): fix flaky OTLP gRPC exporter integration tests (#2505) - chore: move all remaining external deps to workspace-level dependencies (#2476) - perf(trace-utils)!: use pooled spans on the send path (#2382) - feat(data-pipeline)!: OTLP gRPC trace export (#2171) - chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318) [APMSP-3884]: https://datadoghq.atlassian.net/browse/APMSP-3884?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: paullegranddc <82819397+paullegranddc@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Writes fixed agentless JSON framing and field names directly into the output buffer. serde_json still owns every dynamic key, value, escape, and MessagePack conversion.
Motivation
The generic serializer state machine adds overhead for a fixed wire schema.
Additional Notes
This draft is stacked on the fixed-ID branch and contains only the JSON writer layer. Position-balanced Criterion runs improved that baseline by 35.44% for common HTTP traces and 20.18% for span-link traces.
How to test the change?
Run
cargo test -p libdd-trace-utils --lib agentless_encoderandcargo bench -p libdd-trace-utils --bench main -- agentless_encoding.