perf(trace): encode agentless IDs without formatting - #2452
gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits into
Conversation
|
✅ All CI checks and tests passed. 🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 7525711 | Docs | View more details | Give us feedback! |
BenchmarksComparisonBenchmark execution time: 2026-09-10 12:48:15 Comparing candidate commit 7525711 in PR branch Found 3 performance improvements and 1 performance regressions! Performance is the same for 120 metrics, 0 unstable metrics.
|
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
03bcf5c to
5853139
Compare
|
Is this PR superseded by #2454? Or still worth reviewing? |
|
@yannham it is additional to the other PR, it just needs a rebase after it lands |
|
/merge |
|
View all feedbacks in Devflow UI.
It will be processed automatically as soon as GitHub reports it as mergeable. View in MergeQueue UI.
The expected merge time in
|
Replace formatting allocations with fixed-width stack buffers for agentless trace, span, parent, and link IDs. With production-deduplicated spans, position-balanced Criterion runs on macOS arm64 improved common HTTP traces by 32.35% and span-link traces by 44.48%. # Conflicts: # libdd-trace-utils/benches/main.rs
Keep the agentless and JSON log encoders on one stack-backed implementation so their fixed-width hexadecimal wire formats cannot drift.
Release LLVM IR contains no calls to the annotated helper chain, so the hints do not affect the hot path.
22bda7d to
7525711
Compare
…ibdd-data-pipeline, libdd-li... (#2571) <!-- release-proposal-inputs: {"crates":"libdd-capabilities-impl, libdd-common, libdd-data-pipeline, libdd-library-config, libdd-profiling-heap-allocator, libdd-remote-config, libdd-sampling, libdd-shared-runtime, libdd-telemetry, libdd-tinybytes, libdd-trace-utils","main_start_ref":"","level_overrides":"","bypass_standard_checks":false} --> # Release proposal for libdd-capabilities-impl, libdd-common, libdd-data-pipeline, libdd-library-config, libdd-profiling-heap-allocator, libdd-remote-config, libdd-sampling, libdd-shared-runtime, libdd-telemetry, libdd-tinybytes, libdd-trace-utils and their dependencies This PR contains version bumps based on public API changes and commits since last release. ###⚠️ Crates left out of this proposal affected by its major bumps These publishable workspace crates are not part of this release but their dependency requirement was rewritten on this branch while their published version still requires the old major. If they are a dependency on your deployment not including them in the release could result in duplicate packages or symbol incompatibility. - `libdd-capabilities` `3.0.1` → `4.0.0` affects: `libdd-crashtracker`, `libdd-ffe`, `libdd-live-debugger` - `libdd-capabilities-impl` `5.0.0` → `6.0.0` affects: `libdd-crashtracker`, `libdd-live-debugger`, `libdd-tracer-flare` - `libdd-common` `6.0.0` → `7.0.0` affects: `libdd-crashtracker`, `libdd-ffe`, `libdd-http-client`, `libdd-ipc`, `libdd-live-debugger`, `libdd-profiling`, `libdd-tracer-flare` - `libdd-data-pipeline` `10.0.0` → `11.0.0` affects: `libdd-live-debugger` - `libdd-remote-config` `5.0.0` → `6.0.0` affects: `libdd-ffe`, `libdd-live-debugger`, `libdd-tracer-flare` - `libdd-telemetry` `8.0.0` → `9.0.0` affects: `libdd-crashtracker` - `libdd-trace-stats` `9.0.0` → `10.0.0` affects: `libdd-ipc` - `libdd-trace-utils` `12.0.0` → `13.0.0` affects: `libdd-tracer-flare` ## libdd-capabilities **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-capabilities-v4.0.0` ### Commits - feat(trace-exporter)!: add API to force flush stats on the trace exporter (#2538) - refactor!: apply small timeout pooling strategy to libdd-http-client as well (#2449) - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-common **Next version:** `7.0.0` **Semver bump:** `major` **Tag:** `libdd-common-v7.0.0` ### Commits - build: enforce workspace-level dependency declarations (#2514) - chore!: remove unused dependencies (v2) (#2511) - chore: move all remaining external deps to workspace-level dependencies (#2476) - chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318) - fix!: benchmark was doing many samples for deterministic heap usage (#2465) ## libdd-ddsketch **Next version:** `1.1.2` **Semver bump:** `patch` **Tag:** `libdd-ddsketch-v1.1.2` ### Commits - chore: move all remaining external deps to workspace-level dependencies (#2476) - chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318) ## libdd-profiling-heap-sampler **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-profiling-heap-sampler-v1.1.0` ### Commits - chore: move all remaining external deps to workspace-level dependencies (#2476) - refactor: migrate to workspace dependencies, phase 4 (#2296) - feat(profiling): add USDT semaphores to skip slow path around heap profile sampler (#2266) ## libdd-tinybytes **Next version:** `1.1.4` **Semver bump:** `patch` **Tag:** `libdd-tinybytes-v1.1.4` ### Commits - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-trace-protobuf **Next version:** `5.0.1` **Semver bump:** `patch` **Tag:** `libdd-trace-protobuf-v5.0.1` ### Commits - chore: move all remaining external deps to workspace-level dependencies (#2476) - fix(build): build script change detection (#2467) ## libdd-capabilities-impl **Next version:** `6.0.0` **Semver bump:** `major` **Tag:** `libdd-capabilities-impl-v6.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 ### Commits - refactor!: apply small timeout pooling strategy to libdd-http-client as well (#2449) - fix(sidecar)!: stabilize the sidecar for macos usage (#2475) - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-profiling-heap-allocator **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-profiling-heap-allocator-v1.1.0` ### Commits - refactor(benchmarks): measure allocator thread CPU time (#2500) - refactor: migrate to workspace dependencies, phase 4 (#2296) - feat(profiling): add USDT semaphores to skip slow path around heap profile sampler (#2266) ## libdd-library-config **Next version:** `4.1.0` **Semver bump:** `minor` **Tag:** `libdd-library-config-v4.1.0` ### Commits - chore!: remove unused dependencies (v2) (#2511) - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-trace-normalization **Next version:** `4.1.0` **Semver bump:** `minor` **Tag:** `libdd-trace-normalization-v4.1.0` ### Commits - chore: move all remaining external deps to workspace-level dependencies (#2476) - chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318) ## libdd-remote-config **Next version:** `6.0.0` **Semver bump:** `major` **Tag:** `libdd-remote-config-v6.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 ### Commits - refactor!: apply small timeout pooling strategy to libdd-http-client as well (#2449) - chore!: remove unused dependencies (v2) (#2511) - fix(rem-cfg)!: remove AsmRawResponseBody (#2540) - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-shared-runtime **Next version:** `5.0.0` **Semver bump:** `major` **Tag:** `libdd-shared-runtime-v5.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 ### Commits - feat(trace-exporter)!: add API to force flush stats on the trace exporter (#2538) - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-trace-utils **Next version:** `13.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-utils-v13.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 ### Commits - fix(trace-utils)!: add forgotten css to agentless v1 encoder (#2557) - fix(otlp): use potential dedupe in otlp serialization path (#2531) - feat(trace-utils)!: add v1-native OTLP encoder brick (#2369) - perf(serialization): bench the actual path used by the trace exporter (#2512) - perf(trace): write agentless JSON directly (#2454) - chore: move all remaining external deps to workspace-level dependencies (#2476) - perf(trace): encode agentless IDs without formatting (#2452) - perf(trace-utils)!: use pooled spans on the send path (#2382) - feat(data-pipeline)!: OTLP gRPC trace export (#2171) - chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318) - fix!: benchmark was doing many samples for deterministic heap usage (#2465) ## libdd-dogstatsd-client **Next version:** `7.0.0` **Semver bump:** `major` **Tag:** `libdd-dogstatsd-client-v7.0.0` ###⚠️ major bump forced due to: - `libdd-common`: ^6.0.0 → ^7.0.0 - `libdd-shared-runtime`: ^4.0.0 → ^5.0.0 ### Commits - chore: move all remaining external deps to workspace-level dependencies (#2476) ## libdd-telemetry **Next version:** `9.0.0` **Semver bump:** `major` **Tag:** `libdd-telemetry-v9.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 - `libdd-shared-runtime`: ^4.0.0 → ^5.0.0 ### Commits - refactor!: apply small timeout pooling strategy to libdd-http-client as well (#2449) - chore!: remove unused dependencies (v2) (#2511) - chore: move all remaining external deps to workspace-level dependencies (#2476) - fix(telemetry): fall back to the proper intake telemetry domain when direct submission is enabled (#2484) ## libdd-sampling **Next version:** `7.0.0` **Semver bump:** `major` **Tag:** `libdd-sampling-v7.0.0` ###⚠️ major bump forced due to: - `libdd-common`: ^5.2.0 → ^7.0.0 - `libdd-trace-utils`: ^10.1.0 → ^13.0.0 ### Commits - chore(benchmarks): remove batched-loop sampling noise (#2498) - chore: move all remaining external deps to workspace-level dependencies (#2476) - fix!: benchmark was doing many samples for deterministic heap usage (#2465) - feat(trace-utils)!: add from owned to SpanText (#2403) - test(sampling): make rate_limiter thread-safety test deterministic (#2354) ## libdd-trace-obfuscation **Next version:** `9.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-obfuscation-v9.0.0` ###⚠️ major bump forced due to: - `libdd-common`: ^6.0.0 → ^7.0.0 - `libdd-trace-utils`: ^12.0.0 → ^13.0.0 ### Commits - feat(data-pipeline)!: refactor agent's /info obfuscation config format (#2490) - fix(obfuscation): keep URLs whose path or fragment holds a bracket (#2527) - fix(obfuscation): stop forcing serde_json/preserve_order on dependents (#2529) - fix(obfuscation): prevent crash and cap recursion in SQL/HTTP obfuscation (#2441) - chore: move all remaining external deps to workspace-level dependencies (#2476) - fix(data-pipeline-ffi): harden agentless config setters and parse obfuscation config once (#2474) ## libdd-trace-stats **Next version:** `10.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-stats-v10.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 - `libdd-shared-runtime`: ^4.0.0 → ^5.0.0 - `libdd-trace-obfuscation`: ^8.0.0 → ^9.0.0 - `libdd-trace-utils`: ^12.0.0 → ^13.0.0 ### Commits - feat(trace-exporter)!: add API to force flush stats on the trace exporter (#2538) - fix(trace-stats): cardinality limit telemetry name (#2559) - feat(data-pipeline)!: refactor agent's /info obfuscation config format (#2490) - chore!: remove unused dependencies (v2) (#2511) - feat(data-pipeline)!: generate agentless trace stats (#2488) - chore: move all remaining external deps to workspace-level dependencies (#2476) - chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318) ## libdd-data-pipeline-core **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-data-pipeline-core-v2.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 - `libdd-trace-obfuscation`: ^8.0.0 → ^9.0.0 - `libdd-trace-utils`: ^12.0.0 → ^13.0.0 ### Commits - refactor!: apply small timeout pooling strategy to libdd-http-client as well (#2449) - feat(data-pipeline)!: generate agentless trace stats (#2488) - perf(data-pipeline): avoid parsing static entity header names (#2457) - chore: move all remaining external deps to workspace-level dependencies (#2476) - perf(trace-utils)!: use pooled spans on the send path (#2382) ## libdd-data-pipeline **Next version:** `11.0.0` **Semver bump:** `major` **Tag:** `libdd-data-pipeline-v11.0.0` ###⚠️ major bump forced due to: - `libdd-capabilities`: ^3.0.1 → ^4.0.0 - `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0 - `libdd-common`: ^6.0.0 → ^7.0.0 - `libdd-data-pipeline-core`: ^1.0.0 → ^2.0.0 - `libdd-shared-runtime`: ^4.0.0 → ^5.0.0 - `libdd-trace-obfuscation`: ^8.0.0 → ^9.0.0 - `libdd-trace-stats`: ^9.0.0 → ^10.0.0 - `libdd-trace-utils`: ^12.0.0 → ^13.0.0 ### Commits - feat(trace-exporter)!: add API to force flush stats on the trace exporter (#2538) - refactor!: apply small timeout pooling strategy to libdd-http-client as well (#2449) - feat(data-pipeline)!: refactor agent's /info obfuscation config format (#2490) - test(data-pipeline): stop OTLP gRPC post-connect test from hanging (#2549) - chore!: remove unused dependencies (v2) (#2511) - feat(data-pipeline)!: generate agentless trace stats (#2488) - fix(otlp): use potential dedupe in otlp serialization path (#2531) - refactor(benchmarks): isolate enqueue timing in data-pipeline (#2499) - perf(trace): write agentless JSON directly (#2454) - test(data-pipeline): fix flaky OTLP gRPC exporter integration tests (#2505) - chore: move all remaining external deps to workspace-level dependencies (#2476) - perf(trace-utils)!: use pooled spans on the send path (#2382) - feat(data-pipeline)!: OTLP gRPC trace export (#2171) - chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318) [APMSP-3884]: https://datadoghq.atlassian.net/browse/APMSP-3884?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: paullegranddc <82819397+paullegranddc@users.noreply.github.com>
What does this PR do?
Replaces formatting-backed agentless trace, span, parent, and link ID serialization with fixed-width lowercase hexadecimal buffers. Exact wire tests and lasting Criterion workloads cover common HTTP and span-link traces.
Motivation
Formatting allocated temporary strings for every serialized ID in the agentless hot path.
Additional Notes
With production-deduplicated spans, position-balanced Criterion runs improved common HTTP traces by 32.35% and span-link traces by 44.48% on macOS arm64.
How to test the change?
Run
cargo test -p libdd-trace-utils --lib agentless_encoderandcargo bench -p libdd-trace-utils --bench main -- agentless_encoding.