Skip to content

fix(trace-utils)!: fix forward compatibility for string streaming - #2295

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 10 commits into
mainfrom
anais/fix-forward-compatibility-stream-streaming
Aug 20, 2026
Merged

gh-worker-dd-mergequeue-cf854d[bot] merged 10 commits into
mainfrom
anais/fix-forward-compatibility-stream-streaming

Conversation

@anais-raison

@anais-raison anais-raison commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes a forward-compatibility bug in the v1 msgpack decoder: when skipping an unknown key (e.g. a field added by a newer tracer version), inline strings inside the skipped value were not interned into the shared StringTable. This desynced the table's indices, so any later reference to a previously-interned string by ID would resolve to the wrong value.

Motivation

APMSP-3836

@dd-octo-sts

dd-octo-sts Bot commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 89.47 MB 89.47 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.20 MB 8.20 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 100.65 MB 100.65 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.06 MB 11.06 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 26.62 MB 26.62 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 94.56 KB 94.56 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 180.69 MB 180.68 MB -0% (-8.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 772.58 MB 772.61 MB +0% (+36.81 KB) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.72 MB 8.72 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 94.56 KB 94.56 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 25.64 MB 25.64 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 51.06 MB 51.06 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 23.20 MB 23.20 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 96.04 KB 96.04 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 185.60 MB 185.66 MB +.03% (+64.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 759.08 MB 759.02 MB -0% (-65.30 KB) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.74 MB 6.74 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 96.04 KB 96.04 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 27.58 MB 27.58 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 48.60 MB 48.60 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 79.74 MB 79.74 MB +0% (+8 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 9.15 MB 9.15 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 95.38 MB 95.38 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.16 MB 11.16 MB 0% (0 B) 👌

@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Jul 29, 2026 •

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 97.37%
• Overall Coverage: 76.56% (+0.08%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ecbc9d5 | Docs | View more details | Give us feedback!

@ajgajg1134
ajgajg1134 marked this pull request as ready for review July 29, 2026 15:14
@ajgajg1134
ajgajg1134 requested review from a team as code owners July 29, 2026 15:14

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac8e79eff6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread libdd-trace-utils/src/msgpack_decoder/v1/mod.rs Outdated
@Eldolfin

Copy link
Copy Markdown
Contributor

Code lgtm but the tests are hard to read, and it might be over-tested.
Maybe replace it all with 1 test looking more like this: https://github.com/DataDog/datadog-agent/pull/53918/changes#diff-b911ac5eb14f27a9241c4e2fee3769ca5c57f8d66e05dfb5328a253400170855R229

(I don't think we need 1 test per field)

@ajgajg1134

Copy link
Copy Markdown
Contributor

Code lgtm but the tests are hard to read, and it might be over-tested. Maybe replace it all with 1 test looking more like this: https://github.com/DataDog/datadog-agent/pull/53918/changes#diff-b911ac5eb14f27a9241c4e2fee3769ca5c57f8d66e05dfb5328a253400170855R229

(I don't think we need 1 test per field)

(This is my fault lol as I originally provided these tests for this ticket after a quick claude generation and didn't clean them up to be PR ready before sharing them :P Sorry!)

Comment thread libdd-trace-utils/src/msgpack_decoder/v1/mod.rs Outdated
@pr-commenter

pr-commenter Bot commented Aug 6, 2026 •

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-08-20 12:52:09

Comparing candidate commit ecbc9d5 in PR branch anais/fix-forward-compatibility-stream-streaming with baseline commit 1765c7c in branch main.

Found 0 performance improvements and 3 performance regressions! Performance is the same for 165 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:alloc_free/system/4096

  • 🟥 execution_time [+12.649ns; +12.846ns] or [+13.557%; +13.769%]

scenario:glob_matcher/ascii_wildcard_heavy_backtrack/wall_time

  • 🟥 execution_time [+1.933ns; +1.950ns] or [+4.720%; +4.761%]

scenario:profiler_attached/fast_path_system/4096

  • 🟥 execution_time [+9.185ns; +9.319ns] or [+9.493%; +9.631%]

Candidate

Omitted due to size.

Baseline

Omitted due to size.

Comment thread libdd-trace-utils/src/msgpack_decoder/v1/mod.rs
Comment thread libdd-trace-utils/src/msgpack_decoder/decode/buffer.rs Outdated
@ekump
ekump dismissed VianneyRuhlmann’s stale review August 12, 2026 19:09

Vianney is on PTO, Yann and Anais addressed his comments

@anais-raison anais-raison changed the title fix(trace-utils): fix forward compatibility for string streaming fix!(trace-utils): fix forward compatibility for string streaming Aug 18, 2026
@anais-raison anais-raison changed the title fix!(trace-utils): fix forward compatibility for string streaming fix(trace-utils)!: fix forward compatibility for string streaming Aug 18, 2026
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 24022a9 into main Aug 20, 2026
109 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the anais/fix-forward-compatibility-stream-streaming branch August 20, 2026 13:39
hoolioh added a commit that referenced this pull request Aug 26, 2026
…ker, libdd-data-pipeline, li... (#2409)

# Release proposal for libdd-capabilities-impl, libdd-crashtracker,
libdd-data-pipeline, libdd-http-client, libdd-ipc, libdd-library-config,
libdd-otel-thread-ctx, libdd-remote-config, libdd-shared-runtime,
libdd-telemetry, libdd-tracer-flare and their dependencies

This PR contains version bumps based on public API changes and commits
since last release.

## libdd-gotter
**Next version:** `1.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-gotter-v1.1.0`

### Commits

- fix(crashtracker): fix vdso recognition during GOT patching on musl
(#2379)

## libdd-ipc-macros
**Next version:** `1.0.0`
**Semver bump:** `major`
**Tag:** `libdd-ipc-macros-v1.0.0`

**Warning:** this is an initial release. Please verify that the version
and commits included are correct.


## libdd-trace-protobuf
**Next version:** `5.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-protobuf-v5.0.0`

### Commits

- chore(trace-protobuf)!: sync span.proto with datadog-agent (#2356)

## libdd-library-config
**Next version:** `4.0.0`
**Semver bump:** `major`
**Tag:** `libdd-library-config-v4.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.0 → ^5.0.0

### Commits

- fix(library-config): omit absent process context attributes (#2337)
- refactor: migrate to workspace dependencies, phase 4 (#2296)
- refactor: migrate to workspace dependencies, phase 3 (#2283)
- refactor: consolidate core dependencies at workspace level (phase 1)
(#2253)
- fix(library-config): update Linux process context (#2237)
- refactor(library-config): reorganize Linux process context (#2228)
- feat(library-config): otel process context reader (#2176)

## libdd-remote-config
**Next version:** `4.0.0`
**Semver bump:** `major`
**Tag:** `libdd-remote-config-v4.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0

### Commits

- feat(rc): add overrides for RC config/director roots (#2404)
- fix(rc): avoid depending on the crypto nodejs API for remote config
(#2407)
- fix(rc)!: fixup rc to be wasm compatible (#2393)
- feat: Revert vendoring of tuf-rust (#2365), use new crates.io package
(#2374)
- feat(agentless)!: vendor rust-tuf crate in libdatadog for release
(#2365)
- chore(rc)!: add DEBUG product (#2306)
- feat(remote-config)!: agentless RC fetcher (#2112)

## libdd-trace-utils
**Next version:** `11.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-utils-v11.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0

### Commits

- feat(compression): use zrip for wasm targets (#2386)
- fix(trace-utils)!: fix forward compatibility for string streaming
(#2295)

## libdd-tracer-flare
**Next version:** `2.0.0`
**Semver bump:** `major`
**Tag:** `libdd-tracer-flare-v2.0.0`

### ⚠️ major bump forced due to:

- `libdd-remote-config`: ^3.0.0 → ^4.0.0
- `libdd-trace-utils`: ^10.0.0 → ^11.0.0

### Commits

- feat(remote-config)!: agentless RC fetcher (#2112)

## libdd-crashtracker
**Next version:** `2.0.1`
**Semver bump:** `patch`
**Tag:** `libdd-crashtracker-v2.0.1`

### Commits

- fix(crashtracker): hold the collector connection open through
symbolization (#2384)
- fix(crashtracking): stop resolving thread symbols with libunwind in
the receiver (#2361)

## libdd-data-pipeline
**Next version:** `9.0.0`
**Semver bump:** `major`
**Tag:** `libdd-data-pipeline-v9.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0
- `libdd-trace-utils`: ^10.1.0 → ^11.0.0

### Commits

- feat(compression): use zrip for wasm targets (#2386)

## libdd-ipc
**Next version:** `1.0.0`
**Semver bump:** `major`
**Tag:** `libdd-ipc-v1.0.0`

**Warning:** this is an initial release. Please verify that the version
and commits included are correct.


## libdd-trace-normalization
**Next version:** `4.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-normalization-v4.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0


## libdd-trace-obfuscation
**Next version:** `7.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-obfuscation-v7.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0
- `libdd-trace-utils`: ^10.1.0 → ^11.0.0


## libdd-trace-stats
**Next version:** `8.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-stats-v8.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0
- `libdd-trace-utils`: ^10.1.0 → ^11.0.0

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: hoolioh <107922352+hoolioh@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants