Skip to content

fix(data-pipeline)!: revert changes that made /info un-parsable - #2586

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits into
mainfrom
oscarld/fix-info-schema
Oct 1, 2026
Merged

gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits into
mainfrom
oscarld/fix-info-schema

Conversation

@Eldolfin

Copy link
Copy Markdown
Contributor

What does this PR do?

Revert some changes from #2490 that made /info un-parsable in system tests

Motivation

What inspired you to submit this pull request?

Additional Notes

Uses a new dependency serde_ignored as an alternative to putting serde(deny_unknown_fields) on structs to make parsing "strict" only for tests and we accept unknown fields otherwise.

How to test the change?

Describe here in detail how the change can be validated.

@datadog-datadog-us1-prod

datadog-datadog-us1-prod Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 75.00%
• Overall Coverage: 80.02% (+1.10%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 269c523 | Docs | View more details | Give us feedback!

@Eldolfin Eldolfin changed the title fix(data-pipeline): revert changes that made /info un-parsable fix(data-pipeline)!: revert changes that made /info un-parsable Sep 29, 2026
@dd-octo-sts

dd-octo-sts Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 9.02 MB 9.02 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 96.05 MB 96.08 MB +.02% (+23.27 KB) 🔍
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 12.19 MB 12.19 MB +0% (+224 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 107.43 MB 107.45 MB +.02% (+23.00 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 29.08 MB 29.09 MB +.04% (+13.50 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 96.08 KB 96.08 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 191.55 MB 191.65 MB +.04% (+96.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 815.49 MB 819.37 MB +.47% (+3.88 MB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 9.72 MB 9.72 MB +.02% (+2.50 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 96.08 KB 96.08 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 27.51 MB 27.51 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 55.62 MB 55.63 MB +.01% (+9.81 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 25.42 MB 25.44 MB +.05% (+14.50 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 97.58 KB 97.58 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 196.76 MB 196.91 MB +.07% (+144.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 802.14 MB 801.36 MB --.09% (-794.21 KB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 7.54 MB 7.55 MB +.02% (+2.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 97.58 KB 97.58 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 29.62 MB 29.62 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 52.56 MB 52.57 MB +.01% (+8.21 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 86.06 MB 86.08 MB +.02% (+21.60 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 10.06 MB 10.06 MB +.03% (+4.00 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 101.90 MB 101.92 MB +.02% (+21.85 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 12.28 MB 12.29 MB +.03% (+4.07 KB) 🔍

@pr-commenter

pr-commenter Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-10-01 12:17:56

Comparing candidate commit 269c523 in PR branch oscarld/fix-info-schema with baseline commit e8cdc85 in branch main.

📊 Benchmarking dashboard

Found 8 performance improvements and 6 performance regressions! Performance is the same for 163 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:agentless_encoding/common_http/20x5

  • 🟥 execution_time [+14.460µs; +14.518µs] or [+14.163%; +14.221%]
  • 🟥 throughput [-121973.916op/s; -121493.921op/s] or [-12.453%; -12.404%]

scenario:agentless_encoding/http_with_span_links/20x5

  • 🟥 execution_time [+15.239µs; +15.324µs] or [+13.088%; +13.161%]
  • 🟥 throughput [-99934.452op/s; -99355.950op/s] or [-11.636%; -11.569%]

scenario:vec_map/as_deduped_map/already_deduped/16

  • 🟥 execution_time [+1.200ns; +1.214ns] or [+4.975%; +5.033%]

scenario:vec_map/as_deduped_map/already_deduped/8

  • 🟥 execution_time [+1.201ns; +1.213ns] or [+8.300%; +8.382%]

scenario:vec_map/get_hit/128

  • 🟩 execution_time [-1.313µs; -1.306µs] or [-9.542%; -9.490%]
  • 🟩 throughput [+975329.830op/s; +980421.635op/s] or [+10.489%; +10.544%]

scenario:vec_map/get_hit/16

  • 🟩 execution_time [-22.498ns; -22.224ns] or [-9.565%; -9.448%]
  • 🟩 throughput [+7102135.150op/s; +7189040.834op/s] or [+10.441%; +10.569%]

scenario:vec_map/get_hit/64

  • 🟩 execution_time [-294.850ns; -291.573ns] or [-7.964%; -7.875%]
  • 🟩 throughput [+1478542.872op/s; +1494803.316op/s] or [+8.553%; +8.647%]

scenario:vec_map/get_hit/8

  • 🟩 execution_time [-3.384ns; -3.245ns] or [-5.219%; -5.006%]
  • 🟩 throughput [+6512356.509op/s; +6801520.928op/s] or [+5.277%; +5.512%]

Benchmark execution time: 2026-10-01 12:12:59

Comparing candidate commit 269c523 in PR branch oscarld/fix-info-schema with baseline commit e8cdc85 in branch main.

📊 Benchmarking dashboard

Found 13 performance improvements and 7 performance regressions! Performance is the same for 146 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:credit_card/is_card_number/ 378282246310005

  • 🟩 execution_time [-5.338µs; -5.273µs] or [-7.276%; -7.187%]
  • 🟩 throughput [+1055958.498op/s; +1068739.553op/s] or [+7.748%; +7.842%]

scenario:credit_card/is_card_number/378282246310005

  • 🟩 execution_time [-5.348µs; -5.284µs] or [-7.617%; -7.525%]
  • 🟩 throughput [+1159758.927op/s; +1173134.007op/s] or [+8.143%; +8.237%]

scenario:credit_card/is_card_number/37828224631000521389798

  • 🟩 execution_time [-6.478µs; -6.441µs] or [-12.393%; -12.322%]
  • 🟩 throughput [+2689093.428op/s; +2706185.730op/s] or [+14.056%; +14.145%]

scenario:credit_card/is_card_number/x371413321323331

  • 🟥 execution_time [+742.473ns; +759.975ns] or [+12.642%; +12.940%]
  • 🟥 throughput [-19573245.912op/s; -19075633.983op/s] or [-11.495%; -11.202%]

scenario:credit_card/is_card_number_no_luhn/ 378282246310005

  • 🟩 execution_time [-4.110µs; -4.076µs] or [-7.137%; -7.079%]
  • 🟩 throughput [+1323535.206op/s; +1334007.410op/s] or [+7.621%; +7.682%]

scenario:credit_card/is_card_number_no_luhn/378282246310005

  • 🟩 execution_time [-4.609µs; -4.566µs] or [-8.402%; -8.323%]
  • 🟩 throughput [+1656077.756op/s; +1670661.702op/s] or [+9.085%; +9.165%]

scenario:credit_card/is_card_number_no_luhn/37828224631000521389798

  • 🟩 execution_time [-6.512µs; -6.473µs] or [-12.458%; -12.384%]
  • 🟩 throughput [+2704414.046op/s; +2722273.202op/s] or [+14.136%; +14.230%]

scenario:credit_card/is_card_number_no_luhn/x371413321323331

  • 🟥 execution_time [+749.139ns; +766.432ns] or [+12.757%; +13.051%]
  • 🟥 throughput [-19725710.180op/s; -19231685.236op/s] or [-11.583%; -11.293%]

scenario:glob_matcher/ascii_case_insensitive_match/wall_time

  • 🟥 execution_time [+1.342ns; +1.366ns] or [+4.684%; +4.768%]

scenario:glob_matcher/ascii_exact_miss/wall_time

  • 🟥 execution_time [+0.987ns; +1.064ns] or [+7.357%; +7.929%]

scenario:thread_cpu/profiler_attached/fast_path_system/4096

  • 🟥 execution_time [+7.897ns; +8.025ns] or [+8.128%; +8.260%]

scenario:thread_cpu/profiler_attached/slow_path_system/4096

  • 🟩 execution_time [-7.289ns; -7.207ns] or [-4.665%; -4.613%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:datadog_sample_span/parent_not_sampled_short_circuit/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+556.312%; -556.007%]

scenario:datadog_sample_span/parent_sampled_short_circuit/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+554.392%; -555.103%]

scenario:glob_matcher/ascii_case_insensitive_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+563.182%; -559.254%]

scenario:glob_matcher/ascii_exact_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+570.032%; -562.516%]

scenario:glob_matcher/ascii_exact_miss/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+566.037%; -560.611%]

scenario:glob_matcher/ascii_wildcard_backtrack_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+563.670%; -559.486%]

scenario:glob_matcher/ascii_wildcard_heavy_backtrack/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+556.600%; -556.142%]

scenario:glob_matcher/ascii_wildcard_question_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+559.703%; -557.607%]

scenario:glob_matcher/ascii_wildcard_star_match/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+562.874%; -559.108%]

scenario:glob_matcher/star_short_circuit/allocated_bytes

  • unstable execution_time [-0.000ns; +0.000ns] or [+555.639%; -555.689%]

Candidate

Omitted due to size.

Baseline

Omitted due to size.

@Eldolfin
Eldolfin marked this pull request as ready for review September 29, 2026 14:28
@Eldolfin
Eldolfin requested review from a team as code owners September 29, 2026 14:28
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-29T14:31:41.387038Z 88c22b0 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 88c22b0cfb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Cargo.lock

@bantonsson bantonsson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but a small question about avoiding breaking change in structs.

Comment thread libdd-trace-obfuscation/src/obfuscation_config.rs Outdated
Comment thread libdd-trace-obfuscation/src/obfuscation_config.rs Outdated
Comment thread libdd-data-pipeline/src/agent_info/schema.rs
@Eldolfin Eldolfin changed the title fix(data-pipeline)!: revert changes that made /info un-parsable fix(data-pipeline): revert changes that made /info un-parsable Oct 1, 2026
@Eldolfin Eldolfin changed the title fix(data-pipeline): revert changes that made /info un-parsable fix(data-pipeline)!: revert changes that made /info un-parsable Oct 1, 2026

@bantonsson bantonsson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I don't think that you need the ! in the PR title anymore.

Comment thread libdd-trace-protobuf/src/lib.rs
@Eldolfin

Eldolfin commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

I don't think that you need the ! in the PR title anymore.

I tried removing the ! but because I add new public fields in libdd-data-pipeline/src/agent_info/schema.rs and made the deserializer mod public it still counts as a breaking change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants