Skip to content

feat(data-pipeline)!: refactor agent's /info obfuscation config format - #2490

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 13 commits into
mainfrom
oscarld/refactor-agent-info-obfuscation-config-format
Sep 21, 2026
Merged

feat(data-pipeline)!: refactor agent's /info obfuscation config format#2490
gh-worker-dd-mergequeue-cf854d[bot] merged 13 commits into
mainfrom
oscarld/refactor-agent-info-obfuscation-config-format

Conversation

@Eldolfin

@Eldolfin Eldolfin commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Parse the obfuscation config from the agent /info endpoint with a new, more complete format.
It now includes the missing parts, these are not used in stats computation yet it will come in a follow up PR.

Agent side PR: DataDog/datadog-agent#56014

Motivation

2 main motivation:

Client side stats obfuscates resources without using the agent config. This is a breaking change for customers enabling CSS when they have a custom obfuscation config.
See https://docs.google.com/document/d/1i-CQfkF-5B_8vLYepIJ6A16aRISH-Px7XIBSq0HKMaY/edit?tab=t.0#heading=h.i4txi29o2zao

We might want to obfuscate span derived primary tags in client side stats in the future.

Additional Notes

How to test the change?

I tried the fetch_info function manually using a released agent version and with the version in the mentioned PR.

@pr-commenter

pr-commenter Bot commented Sep 9, 2026 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-09-18 12:17:51

Comparing candidate commit 4526320 in PR branch oscarld/refactor-agent-info-obfuscation-config-format with baseline commit 308e5c2 in branch main.

📊 Benchmarking dashboard

Found 2 performance improvements and 2 performance regressions! Performance is the same for 20 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:trace_buffer/2_senders/no_delay

  • 🟥 execution_time [+168.808µs; +183.549µs] or [+10.816%; +11.760%]
  • 🟥 throughput [-121790.671op/s; -112068.180op/s] or [-10.555%; -9.713%]

scenario:trace_buffer/8_senders/no_delay

  • 🟩 execution_time [-622.041µs; -609.841µs] or [-8.814%; -8.641%]
  • 🟩 throughput [+96586.270op/s; +98514.854op/s] or [+9.467%; +9.656%]

Benchmark execution time: 2026-09-18 12:16:09

Comparing candidate commit 4526320 in PR branch oscarld/refactor-agent-info-obfuscation-config-format with baseline commit 308e5c2 in branch main.

📊 Benchmarking dashboard

Found 16 performance improvements and 3 performance regressions! Performance is the same for 17 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:credit_card/is_card_number/ 3782-8224-6310-005

  • 🟩 execution_time [-5.106µs; -4.957µs] or [-6.408%; -6.221%]
  • 🟩 throughput [+833624.164op/s; +858527.377op/s] or [+6.642%; +6.841%]

scenario:credit_card/is_card_number/ 378282246310005

  • 🟩 execution_time [-5.578µs; -5.459µs] or [-7.579%; -7.418%]
  • 🟩 throughput [+1089968.815op/s; +1113115.135op/s] or [+8.022%; +8.192%]

scenario:credit_card/is_card_number/378282246310005

  • 🟥 execution_time [+8.450µs; +8.552µs] or [+12.008%; +12.152%]
  • 🟥 throughput [-1541920.140op/s; -1521819.367op/s] or [-10.851%; -10.709%]

scenario:credit_card/is_card_number/37828224631000521389798

  • 🟩 execution_time [-8.144µs; -8.069µs] or [-15.360%; -15.219%]
  • 🟩 throughput [+3387899.273op/s; +3420555.460op/s] or [+17.963%; +18.137%]

scenario:credit_card/is_card_number/x371413321323331

  • 🟩 execution_time [-404.158ns; -401.662ns] or [-6.263%; -6.224%]
  • 🟩 throughput [+10287426.499op/s; +10351187.971op/s] or [+6.639%; +6.680%]

scenario:credit_card/is_card_number_no_luhn/ 378282246310005

  • 🟩 execution_time [-5.612µs; -5.557µs] or [-9.490%; -9.397%]
  • 🟩 throughput [+1755302.837op/s; +1771326.256op/s] or [+10.380%; +10.474%]

scenario:credit_card/is_card_number_no_luhn/378282246310005

  • 🟩 execution_time [-5.392µs; -5.343µs] or [-9.680%; -9.592%]
  • 🟩 throughput [+1906077.613op/s; +1922805.499op/s] or [+10.617%; +10.710%]

scenario:credit_card/is_card_number_no_luhn/37828224631000521389798

  • 🟩 execution_time [-8.182µs; -8.107µs] or [-15.432%; -15.290%]
  • 🟩 throughput [+3407079.173op/s; +3439593.415op/s] or [+18.064%; +18.236%]

scenario:credit_card/is_card_number_no_luhn/x371413321323331

  • 🟩 execution_time [-404.877ns; -402.204ns] or [-6.272%; -6.230%]
  • 🟩 throughput [+10295326.207op/s; +10363342.376op/s] or [+6.646%; +6.690%]

scenario:sql/obfuscate_sql_string

  • 🟥 execution_time [+12.852µs; +13.100µs] or [+4.376%; +4.460%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 4526320 1789733038 oscarld/refactor-agent-info-obfuscation-config-format
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
trace_buffer/1_senders/10us_delay execution_time 59.957ms 59.989ms ± 0.030ms 59.982ms ± 0.007ms 59.988ms 60.049ms 60.113ms 60.121ms 0.23% 2.696 7.672 0.05% 0.002ms 1 200
trace_buffer/1_senders/10us_delay throughput 14969.904op/s 15002.701op/s ± 7.466op/s 15004.419op/s ± 1.837op/s 15006.353op/s 15009.233op/s 15010.561op/s 15010.822op/s 0.04% -2.692 7.652 0.05% 0.528op/s 1 200
trace_buffer/1_senders/1us_delay execution_time 50.173ms 50.714ms ± 0.172ms 50.710ms ± 0.108ms 50.825ms 50.980ms 51.048ms 51.098ms 0.77% -0.168 -0.234 0.34% 0.012ms 1 200
trace_buffer/1_senders/1us_delay throughput 17613.140op/s 17746.945op/s ± 60.076op/s 17748.077op/s ± 37.806op/s 17780.334op/s 17843.632op/s 17880.583op/s 17937.838op/s 1.07% 0.186 -0.218 0.34% 4.248op/s 1 200
trace_buffer/1_senders/no_delay execution_time 340.967µs 341.921µs ± 0.895µs 341.840µs ± 0.348µs 342.173µs 342.783µs 343.442µs 352.277µs 3.05% 7.872 87.903 0.26% 0.063µs 1 200
trace_buffer/1_senders/no_delay throughput 2554808.424op/s 2632201.974op/s ± 6752.464op/s 2632813.156op/s ± 2679.003op/s 2635538.284op/s 2638427.810op/s 2639222.354op/s 2639552.377op/s 0.26% -7.656 84.537 0.26% 477.471op/s 1 200
trace_buffer/2_senders/10us_delay execution_time 59.960ms 59.999ms ± 0.035ms 59.991ms ± 0.013ms 60.005ms 60.040ms 60.145ms 60.192ms 0.33% 2.949 10.830 0.06% 0.002ms 1 200
trace_buffer/2_senders/10us_delay throughput 29904.498op/s 30000.493op/s ± 17.290op/s 30004.518op/s ± 6.339op/s 30010.267op/s 30017.023op/s 30019.011op/s 30019.837op/s 0.05% -2.942 10.783 0.06% 1.223op/s 1 200
trace_buffer/2_senders/1us_delay execution_time 50.893ms 51.105ms ± 0.071ms 51.100ms ± 0.041ms 51.135ms 51.241ms 51.288ms 51.353ms 0.50% 0.538 0.820 0.14% 0.005ms 1 200
trace_buffer/2_senders/1us_delay throughput 35051.352op/s 35221.748op/s ± 49.050op/s 35224.900op/s ± 28.382op/s 35253.391op/s 35288.288op/s 35326.614op/s 35368.085op/s 0.41% -0.528 0.811 0.14% 3.468op/s 1 200
trace_buffer/2_senders/no_delay execution_time 1.626ms 1.737ms ± 0.041ms 1.735ms ± 0.029ms 1.766ms 1.811ms 1.825ms 1.840ms 6.06% 0.027 -0.338 2.37% 0.003ms 1 200
trace_buffer/2_senders/no_delay throughput 978169.326op/s 1036897.698op/s ± 24668.688op/s 1037438.116op/s ± 17486.171op/s 1054049.883op/s 1075956.081op/s 1092926.528op/s 1106945.171op/s 6.70% 0.092 -0.304 2.37% 1744.340op/s 1 200
trace_buffer/4_senders/10us_delay execution_time 59.984ms 60.040ms ± 0.040ms 60.034ms ± 0.017ms 60.047ms 60.136ms 60.164ms 60.262ms 0.38% 2.115 6.042 0.07% 0.003ms 1 200
trace_buffer/4_senders/10us_delay throughput 59739.237op/s 59959.839op/s ± 39.547op/s 59966.363op/s ± 16.636op/s 59984.997op/s 60000.360op/s 60005.783op/s 60015.972op/s 0.08% -2.108 5.997 0.07% 2.796op/s 1 200
trace_buffer/4_senders/1us_delay execution_time 51.041ms 51.283ms ± 0.202ms 51.227ms ± 0.075ms 51.338ms 51.608ms 52.215ms 52.543ms 2.57% 3.265 14.915 0.39% 0.014ms 1 200
trace_buffer/4_senders/1us_delay throughput 68515.118op/s 70199.332op/s ± 273.476op/s 70275.800op/s ± 103.335op/s 70358.050op/s 70443.976op/s 70491.100op/s 70531.731op/s 0.36% -3.192 14.334 0.39% 19.338op/s 1 200
trace_buffer/4_senders/no_delay execution_time 3.505ms 3.566ms ± 0.025ms 3.564ms ± 0.018ms 3.584ms 3.610ms 3.617ms 3.619ms 1.56% 0.089 -0.611 0.69% 0.002ms 1 200
trace_buffer/4_senders/no_delay throughput 994656.924op/s 1009666.321op/s ± 7004.557op/s 1010165.241op/s ± 5203.028op/s 1014777.860op/s 1020657.486op/s 1023974.081op/s 1027216.149op/s 1.69% -0.060 -0.607 0.69% 495.297op/s 1 200
trace_buffer/8_senders/10us_delay execution_time 60.008ms 60.119ms ± 0.078ms 60.096ms ± 0.051ms 60.166ms 60.238ms 60.341ms 60.509ms 0.69% 1.235 2.647 0.13% 0.006ms 1 200
trace_buffer/8_senders/10us_delay throughput 118991.061op/s 119763.324op/s ± 155.101op/s 119807.583op/s ± 101.498op/s 119884.622op/s 119946.970op/s 119974.271op/s 119983.600op/s 0.15% -1.223 2.582 0.13% 10.967op/s 1 200
trace_buffer/8_senders/1us_delay execution_time 51.397ms 51.827ms ± 0.239ms 51.773ms ± 0.148ms 51.993ms 52.287ms 52.405ms 52.516ms 1.43% 0.670 -0.192 0.46% 0.017ms 1 200
trace_buffer/8_senders/1us_delay throughput 137101.001op/s 138926.800op/s ± 637.524op/s 139067.418op/s ± 398.759op/s 139396.270op/s 139720.837op/s 140003.456op/s 140085.351op/s 0.73% -0.651 -0.220 0.46% 45.080op/s 1 200
trace_buffer/8_senders/no_delay execution_time 6.334ms 6.442ms ± 0.029ms 6.441ms ± 0.019ms 6.462ms 6.486ms 6.503ms 6.534ms 1.44% -0.213 0.668 0.45% 0.002ms 1 200
trace_buffer/8_senders/no_delay throughput 1101886.552op/s 1117760.977op/s ± 5064.931op/s 1117767.723op/s ± 3360.080op/s 1120779.522op/s 1125515.012op/s 1130447.376op/s 1136776.292op/s 1.70% 0.249 0.703 0.45% 358.145op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
trace_buffer/1_senders/10us_delay execution_time [59.985ms; 59.993ms] or [-0.007%; +0.007%] None None None
trace_buffer/1_senders/10us_delay throughput [15001.667op/s; 15003.736op/s] or [-0.007%; +0.007%] None None None
trace_buffer/1_senders/1us_delay execution_time [50.690ms; 50.737ms] or [-0.047%; +0.047%] None None None
trace_buffer/1_senders/1us_delay throughput [17738.619op/s; 17755.271op/s] or [-0.047%; +0.047%] None None None
trace_buffer/1_senders/no_delay execution_time [341.797µs; 342.045µs] or [-0.036%; +0.036%] None None None
trace_buffer/1_senders/no_delay throughput [2631266.148op/s; 2633137.801op/s] or [-0.036%; +0.036%] None None None
trace_buffer/2_senders/10us_delay execution_time [59.994ms; 60.004ms] or [-0.008%; +0.008%] None None None
trace_buffer/2_senders/10us_delay throughput [29998.096op/s; 30002.889op/s] or [-0.008%; +0.008%] None None None
trace_buffer/2_senders/1us_delay execution_time [51.095ms; 51.115ms] or [-0.019%; +0.019%] None None None
trace_buffer/2_senders/1us_delay throughput [35214.950op/s; 35228.545op/s] or [-0.019%; +0.019%] None None None
trace_buffer/2_senders/no_delay execution_time [1.731ms; 1.743ms] or [-0.329%; +0.329%] None None None
trace_buffer/2_senders/no_delay throughput [1033478.855op/s; 1040316.540op/s] or [-0.330%; +0.330%] None None None
trace_buffer/4_senders/10us_delay execution_time [60.035ms; 60.046ms] or [-0.009%; +0.009%] None None None
trace_buffer/4_senders/10us_delay throughput [59954.358op/s; 59965.319op/s] or [-0.009%; +0.009%] None None None
trace_buffer/4_senders/1us_delay execution_time [51.255ms; 51.311ms] or [-0.055%; +0.055%] None None None
trace_buffer/4_senders/1us_delay throughput [70161.431op/s; 70237.233op/s] or [-0.054%; +0.054%] None None None
trace_buffer/4_senders/no_delay execution_time [3.562ms; 3.569ms] or [-0.096%; +0.096%] None None None
trace_buffer/4_senders/no_delay throughput [1008695.557op/s; 1010637.085op/s] or [-0.096%; +0.096%] None None None
trace_buffer/8_senders/10us_delay execution_time [60.108ms; 60.129ms] or [-0.018%; +0.018%] None None None
trace_buffer/8_senders/10us_delay throughput [119741.828op/s; 119784.819op/s] or [-0.018%; +0.018%] None None None
trace_buffer/8_senders/1us_delay execution_time [51.794ms; 51.860ms] or [-0.064%; +0.064%] None None None
trace_buffer/8_senders/1us_delay throughput [138838.446op/s; 139015.155op/s] or [-0.064%; +0.064%] None None None
trace_buffer/8_senders/no_delay execution_time [6.438ms; 6.446ms] or [-0.063%; +0.063%] None None None
trace_buffer/8_senders/no_delay throughput [1117059.026op/s; 1118462.928op/s] or [-0.063%; +0.063%] None None None

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 4526320 1789733038 oscarld/refactor-agent-info-obfuscation-config-format
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 21.399µs 22.119µs ± 0.409µs 22.047µs ± 0.228µs 22.288µs 22.848µs 23.628µs 23.850µs 8.18% 1.653 3.868 1.84% 0.029µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [22.063µs; 22.176µs] or [-0.256%; +0.256%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 4526320 1789733038 oscarld/refactor-agent-info-obfuscation-config-format
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.625µs 2.675µs ± 0.015µs 2.673µs ± 0.010µs 2.686µs 2.700µs 2.712µs 2.721µs 1.79% 0.026 0.440 0.56% 0.001µs 1 200
tags/replace_trace_tags_v04 execution_time 2.663µs 2.847µs ± 0.080µs 2.885µs ± 0.012µs 2.895µs 2.903µs 2.907µs 2.930µs 1.55% -1.387 0.078 2.79% 0.006µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.673µs; 2.677µs] or [-0.077%; +0.077%] None None None
tags/replace_trace_tags_v04 execution_time [2.836µs; 2.858µs] or [-0.388%; +0.388%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 4526320 1789733038 oscarld/refactor-agent-info-obfuscation-config-format
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 305.938µs 306.678µs ± 0.491µs 306.612µs ± 0.150µs 306.765µs 307.251µs 308.795µs 310.725µs 1.34% 4.127 26.029 0.16% 0.035µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [306.610µs; 306.746µs] or [-0.022%; +0.022%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 4526320 1789733038 oscarld/refactor-agent-info-obfuscation-config-format
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 30.572µs 31.277µs ± 0.945µs 30.690µs ± 0.054µs 32.459µs 32.797µs 33.769µs 33.809µs 10.16% 1.034 -0.606 3.01% 0.067µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [31.147µs; 31.408µs] or [-0.419%; +0.419%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 4526320 1789733038 oscarld/refactor-agent-info-obfuscation-config-format
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 4.974µs 5.041µs ± 0.044µs 5.026µs ± 0.017µs 5.055µs 5.153µs 5.156µs 5.157µs 2.59% 1.550 1.564 0.87% 0.003µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.035µs; 5.047µs] or [-0.122%; +0.122%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 4526320 1789733038 oscarld/refactor-agent-info-obfuscation-config-format
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 9.530ms 9.561ms ± 0.017ms 9.558ms ± 0.012ms 9.571ms 9.594ms 9.608ms 9.617ms 0.62% 0.787 0.333 0.18% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [9.558ms; 9.563ms] or [-0.024%; +0.024%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 4526320 1789733038 oscarld/refactor-agent-info-obfuscation-config-format
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 2.071µs 2.093µs ± 0.010µs 2.093µs ± 0.004µs 2.096µs 2.104µs 2.115µs 2.205µs 5.37% 6.238 65.507 0.50% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.092µs; 2.095µs] or [-0.069%; +0.069%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 4526320 1789733038 oscarld/refactor-agent-info-obfuscation-config-format
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.907µs 3.921µs ± 0.004µs 3.920µs ± 0.003µs 3.924µs 3.928µs 3.931µs 3.934µs 0.35% 0.417 0.151 0.11% 0.000µs 1 200
credit_card/is_card_number/ throughput 254216309.988op/s 255050636.241op/s ± 271249.792op/s 255095920.878op/s ± 206608.280op/s 255270875.967op/s 255403764.963op/s 255474796.628op/s 255943541.583op/s 0.33% -0.410 0.149 0.11% 19180.257op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 74.371µs 74.650µs ± 0.429µs 74.530µs ± 0.097µs 74.669µs 75.309µs 75.791µs 79.237µs 6.31% 6.661 64.406 0.57% 0.030µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 12620392.704op/s 13396323.044op/s ± 74163.353op/s 13417370.467op/s ± 17452.574op/s 13432581.541op/s 13440996.913op/s 13445425.100op/s 13446112.017op/s 0.21% -6.297 58.782 0.55% 5244.141op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 67.675µs 68.078µs ± 0.321µs 68.046µs ± 0.055µs 68.108µs 68.184µs 68.256µs 72.478µs 6.51% 12.911 174.385 0.47% 0.023µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 13797269.492op/s 14689344.719op/s ± 65399.182op/s 14695964.488op/s ± 11882.865op/s 14706518.664op/s 14712221.590op/s 14715676.479op/s 14776453.370op/s 0.55% -12.771 171.862 0.44% 4624.421op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.884µs 3.922µs ± 0.005µs 3.921µs ± 0.003µs 3.925µs 3.929µs 3.932µs 3.933µs 0.32% -1.704 13.799 0.13% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 254228884.726op/s 254979841.291op/s ± 332762.133op/s 255031715.258op/s ± 210626.101op/s 255225647.590op/s 255325062.679op/s 255420767.617op/s 257477937.492op/s 0.96% 1.755 14.246 0.13% 23529.836op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 78.677µs 78.874µs ± 0.092µs 78.864µs ± 0.057µs 78.916µs 79.054µs 79.137µs 79.300µs 0.55% 1.210 2.317 0.12% 0.007µs 1 200
credit_card/is_card_number/378282246310005 throughput 12610327.958op/s 12678486.391op/s ± 14767.602op/s 12679998.572op/s ± 9181.679op/s 12689700.406op/s 12696372.519op/s 12700675.333op/s 12710156.686op/s 0.24% -1.200 2.276 0.12% 1044.227op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 44.628µs 44.916µs ± 0.198µs 44.911µs ± 0.166µs 45.072µs 45.201µs 45.251µs 45.834µs 2.06% 0.533 0.721 0.44% 0.014µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 21817715.045op/s 22264262.179op/s ± 97844.417op/s 22266198.861op/s ± 82137.924op/s 22349766.830op/s 22394992.617op/s 22398940.109op/s 22407209.535op/s 0.63% -0.502 0.575 0.44% 6918.645op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.040µs 6.050µs ± 0.006µs 6.049µs ± 0.004µs 6.054µs 6.062µs 6.065µs 6.069µs 0.32% 0.658 -0.341 0.10% 0.000µs 1 200
credit_card/is_card_number/x371413321323331 throughput 164782418.424op/s 165280301.407op/s ± 165262.566op/s 165313530.913op/s ± 117465.621op/s 165413428.123op/s 165493794.060op/s 165517970.000op/s 165555302.832op/s 0.15% -0.654 -0.348 0.10% 11685.828op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.895µs 3.921µs ± 0.005µs 3.921µs ± 0.003µs 3.924µs 3.929µs 3.935µs 3.936µs 0.40% -0.200 5.160 0.12% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 254047051.187op/s 255013960.046op/s ± 295639.137op/s 255059241.540op/s ± 185431.141op/s 255221415.212op/s 255340814.478op/s 255436756.353op/s 256726542.354op/s 0.65% 0.225 5.275 0.12% 20904.844op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 65.421µs 65.630µs ± 0.072µs 65.623µs ± 0.048µs 65.679µs 65.756µs 65.794µs 65.870µs 0.38% 0.204 0.431 0.11% 0.005µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15181506.821op/s 15237028.077op/s ± 16791.418op/s 15238560.410op/s ± 11200.247op/s 15248495.158op/s 15260259.853op/s 15280397.137op/s 15285555.150op/s 0.31% -0.196 0.431 0.11% 1187.333op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 53.459µs 53.550µs ± 0.062µs 53.541µs ± 0.037µs 53.576µs 53.665µs 53.746µs 53.840µs 0.56% 1.415 2.883 0.11% 0.004µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 18573612.141op/s 18674169.148op/s ± 21467.036op/s 18677341.583op/s ± 13044.514op/s 18690761.624op/s 18697579.039op/s 18702503.312op/s 18706086.961op/s 0.15% -1.406 2.836 0.11% 1517.949op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.884µs 3.921µs ± 0.005µs 3.921µs ± 0.003µs 3.924µs 3.929µs 3.931µs 3.933µs 0.31% -1.907 16.766 0.12% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 254263701.260op/s 255023024.002op/s ± 315230.757op/s 255051680.461op/s ± 192672.984op/s 255232625.276op/s 255335691.502op/s 255430513.043op/s 257489866.446op/s 0.96% 1.964 17.275 0.12% 22290.181op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 50.223µs 50.332µs ± 0.093µs 50.312µs ± 0.045µs 50.362µs 50.470µs 50.754µs 50.806µs 0.98% 2.613 9.041 0.18% 0.007µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 19682730.214op/s 19868079.911op/s ± 36344.875op/s 19875910.407op/s ± 17665.958op/s 19891485.450op/s 19901753.943op/s 19906986.479op/s 19911002.820op/s 0.18% -2.589 8.898 0.18% 2569.971op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 44.627µs 44.875µs ± 0.193µs 44.831µs ± 0.152µs 45.046µs 45.196µs 45.249µs 45.490µs 1.47% 0.541 -0.793 0.43% 0.014µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 21982814.368op/s 22284539.310op/s ± 95492.753op/s 22306166.618op/s ± 75670.374op/s 22375648.714op/s 22399128.238op/s 22403562.533op/s 22408035.863op/s 0.46% -0.529 -0.818 0.43% 6752.357op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.040µs 6.052µs ± 0.006µs 6.050µs ± 0.004µs 6.055µs 6.064µs 6.071µs 6.074µs 0.39% 0.940 0.725 0.10% 0.000µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 164639450.062op/s 165238192.553op/s ± 172662.435op/s 165277084.363op/s ± 103953.614op/s 165362469.811op/s 165441424.297op/s 165501885.089op/s 165551257.143op/s 0.17% -0.934 0.709 0.10% 12209.078op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.920µs; 3.921µs] or [-0.015%; +0.015%] None None None
credit_card/is_card_number/ throughput [255013043.629op/s; 255088228.854op/s] or [-0.015%; +0.015%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [74.590µs; 74.709µs] or [-0.080%; +0.080%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [13386044.716op/s; 13406601.371op/s] or [-0.077%; +0.077%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [68.033µs; 68.123µs] or [-0.065%; +0.065%] None None None
credit_card/is_card_number/ 378282246310005 throughput [14680281.022op/s; 14698408.417op/s] or [-0.062%; +0.062%] None None None
credit_card/is_card_number/37828224631 execution_time [3.921µs; 3.923µs] or [-0.018%; +0.018%] None None None
credit_card/is_card_number/37828224631 throughput [254933723.659op/s; 255025958.922op/s] or [-0.018%; +0.018%] None None None
credit_card/is_card_number/378282246310005 execution_time [78.861µs; 78.887µs] or [-0.016%; +0.016%] None None None
credit_card/is_card_number/378282246310005 throughput [12676439.744op/s; 12680533.039op/s] or [-0.016%; +0.016%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [44.888µs; 44.943µs] or [-0.061%; +0.061%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [22250701.884op/s; 22277822.474op/s] or [-0.061%; +0.061%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.049µs; 6.051µs] or [-0.014%; +0.014%] None None None
credit_card/is_card_number/x371413321323331 throughput [165257397.605op/s; 165303205.209op/s] or [-0.014%; +0.014%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.921µs; 3.922µs] or [-0.016%; +0.016%] None None None
credit_card/is_card_number_no_luhn/ throughput [254972987.305op/s; 255054932.787op/s] or [-0.016%; +0.016%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [65.620µs; 65.640µs] or [-0.015%; +0.015%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15234700.948op/s; 15239355.206op/s] or [-0.015%; +0.015%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [53.541µs; 53.559µs] or [-0.016%; +0.016%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [18671194.024op/s; 18677144.273op/s] or [-0.016%; +0.016%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.921µs; 3.922µs] or [-0.017%; +0.017%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [254979336.051op/s; 255066711.954op/s] or [-0.017%; +0.017%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [50.319µs; 50.345µs] or [-0.025%; +0.025%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [19863042.861op/s; 19873116.961op/s] or [-0.025%; +0.025%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [44.848µs; 44.902µs] or [-0.060%; +0.060%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [22271304.933op/s; 22297773.687op/s] or [-0.059%; +0.059%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.051µs; 6.053µs] or [-0.014%; +0.014%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [165214263.200op/s; 165262121.906op/s] or [-0.014%; +0.014%] None None None

Baseline

Omitted due to size.

@Eldolfin
Eldolfin force-pushed the oscarld/refactor-agent-info-obfuscation-config-format branch from e936bbd to b3a818c Compare September 9, 2026 12:57
@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Sep 9, 2026 •
edited by datadog-prod-us1-5 Bot
Loading

Copy link
Copy Markdown

Tests

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 75.19%
• Overall Coverage: 78.09% (+0.16%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4526320 | Docs | View more details | Give us feedback!

@Eldolfin Eldolfin changed the title feat(data-pipeline): refactor agent's /info obfuscation config format feat(data-pipeline)!: refactor agent's /info obfuscation config format Sep 10, 2026
@dd-octo-sts

dd-octo-sts Bot commented Sep 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 9.02 MB 9.02 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 95.91 MB 96.01 MB +.09% (+97.89 KB) 🔍
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 107.32 MB 107.41 MB +.08% (+98.56 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 12.18 MB 12.19 MB +.05% (+6.29 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 29.00 MB 29.05 MB +.17% (+52.00 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 96.08 KB 96.08 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 191.44 MB 191.50 MB +.02% (+56.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 818.80 MB 816.18 MB --.31% (-2.61 MB) 💪
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 9.68 MB 9.71 MB +.29% (+29.50 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 96.08 KB 96.08 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 27.46 MB 27.51 MB +.17% (+48.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 55.50 MB 55.59 MB +.16% (+91.56 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 25.38 MB 25.43 MB +.20% (+54.00 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 97.58 KB 97.58 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 196.73 MB 196.76 MB +.01% (+40.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 801.69 MB 800.18 MB --.18% (-1.50 MB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 7.50 MB 7.52 MB +.26% (+20.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 97.58 KB 97.58 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 29.59 MB 29.63 MB +.13% (+40.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 52.48 MB 52.56 MB +.15% (+81.21 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 85.87 MB 85.97 MB +.11% (+99.83 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 10.02 MB 10.05 MB +.23% (+24.00 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 101.76 MB 101.85 MB +.09% (+99.42 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 12.24 MB 12.27 MB +.20% (+25.92 KB) 🔍

@Eldolfin
Eldolfin added this pull request to stack #2536 September 16, 2026 11:29
@Eldolfin
Eldolfin marked this pull request as ready for review September 16, 2026 13:30
@Eldolfin
Eldolfin requested review from a team as code owners September 16, 2026 13:30
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 16, 2026 •
edited
Loading

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-16T13:39:12.365399Z fb76516 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Sep 16, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fb76516326

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-data-pipeline/src/trace_exporter/stats.rs
Comment thread libdd-trace-obfuscation/src/obfuscation_config.rs
Comment thread libdd-data-pipeline/src/agent_info/schema.rs Outdated
ichinaski
ichinaski reviewed Sep 16, 2026
View reviewed changes

@ichinaski ichinaski left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding test coverage specially for edge cases

Comment thread libdd-data-pipeline/src/agent_info/schema.rs Outdated
VianneyRuhlmann
VianneyRuhlmann approved these changes Sep 17, 2026
View reviewed changes
Comment thread libdd-data-pipeline/src/agent_info/schema.rs Outdated
@webern webern mentioned this pull request Sep 18, 2026
Merged
yannham
yannham approved these changes Sep 18, 2026
View reviewed changes
Comment thread libdd-data-pipeline/src/agent_info/schema.rs Outdated
Comment thread libdd-data-pipeline/src/agent_info/schema.rs Outdated
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 45009c2 into main Sep 21, 2026
99 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the oscarld/refactor-agent-info-obfuscation-config-format branch September 21, 2026 12:47
@dd-octo-sts dd-octo-sts Bot mentioned this pull request Sep 24, 2026
Merged
gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Sep 24, 2026
@webern
feat(obfuscation)!: make json transforms caller-provided (#2548)
9eed262 
## Human Summary

While working on DataDog/saluki#2601, I discovered that I couldn’t use libdatadog’s JSON obfuscator as written. Its transformer was a plain function pointer whose only argument was the JSON value. Unlike a closure, a function pointer cannot capture runtime state, so there was no way to pass Saluki’s dynamically loaded SQL obfuscation configuration without putting it in static or global state. Existing users can configure ordinary SQL obfuscation, but the existing JSON transformer API provides no normal way to pass those SQL settings when transforming SQL embedded in JSON. The provided transformer uses default settings.

In fixing this, I decided to take it a little further. Transform failures are now returned as structured information, scan failures use a typed error, and the hot-path API accepts caller-owned output and scratch buffers so repeated calls can reuse their allocations. Transformations return `Cow<str>`, which also avoids forcing an allocation when they can return borrowed text.

The coding agent brought in `thiserror` as a dependency. This is idiomatic and seems like a good call. The scanner's new const fns are `clippy::missing_const_for_fn`, which the workspace enables; they have no runtime effect. `Scanner::new` becoming restart is for buffer reuse. The parse-state stack now survives between passes.

# What does this PR do?

This separates JSON obfuscation policy from transformation behavior. `JsonObfuscatorConfig` now contains only serializable, comparable data. Callers provide transforms per call through generic closures that can capture runtime SQL configuration.

The allocating API is joined by `obfuscate_into`, which reuses caller-owned output and scratch buffers. Scratch capacity is observable and explicitly trimmable so callers can choose their memory-retention policy.

JSON scan errors and transform errors are reported separately. SQL obfuscation now reports an empty result as `SqlObfuscationError::EmptyResult`, and the crate exports the Agent's exact SQL failure replacement. This also pins non-ASCII identifier behavior and fixes the JSON unquoting fallback to retain the literal when unquoting fails.

# Motivation

The stored `fn(&str) -> String` transformer cannot capture runtime configuration or report errors. This prevents Agent-compatible configured SQL transformation inside JSON values.

DataDog/saluki#2601 currently duplicates libdatadog's JSON scanner to work around that API. A call-scoped closure lets Saluki use the shared scanner while retaining its configured SQL behavior, logging, and failure policy.

# Additional Notes

Transform callbacks return `Cow<str>`, allowing borrowed, static, or owned replacements. The callback is generic and allocation-free; no transformer trait or stored trait object is introduced.

A transform error replaces that value with `"?"`, records the error, and continues. Callers that need the Agent's SQL fallback can observe and log the SQL error inside the callback, then return `SQL_OBFUSCATION_FAILURE_REPLACEMENT`.

`JsonObfuscationScratch` retains capacity from the largest prior input until the caller trims or drops it. `retained_capacity` and `trim_to` make that policy explicit.

## Rebase onto #2490

#2490 moved `SqlObfuscateConfig`, `SqlObfuscationMode` and `DbmsKind` from `sql` into `obfuscation_config` and renamed the config struct to `SqlConfig`, and it began deserializing `JsonObfuscator` directly from the Agent's `/info` payload. Resolved as follows:

- Adopted `obfuscation_config::{DbmsKind, SqlConfig, SqlObfuscationMode}` everywhere, including the `obfuscate_with` doc example. No type is reintroduced in `sql`.
- `JsonObfuscatorConfig` keeps `#[serde(default)]` but not `deny_unknown_fields`: #2490 dropped it deliberately, and an `/info` payload from a newer Agent carries fields this struct has no counterpart for. A test pins that forward compatibility.
- #2490's hand-written `PartialEq for JsonObfuscatorConfig`, which existed only to skirt the uncomparable `transformer` field, is replaced by a derive now that the field is gone. The config also derives `Eq`.
- `obfuscate_resource_for_stats` and `obfuscate_pb_span` now consume the `Result` from `obfuscate_sql`: a resource that obfuscates to nothing is left as sent rather than blanked.

## Agent `/info` field names

Checked against the Agent rather than guessed. `pkg/trace/api/info.go` serves a *reduced* view:

```go
type reducedJSONObfuscationConfig struct {
	Enabled  bool     `json:"enabled"`
	KeepKeys []string `json:"keep_keys"`
}
...
oconf.Elasticsearch = reducedJSONObfuscationConfig{Enabled: o.ES.Enabled, KeepKeys: o.ES.KeepValues}
```

So `/info` sends `keep_keys`, which is already this crate's field name, and it does not report the transform set at all. The Agent's own obfuscation config (`pkg/obfuscate`, `apm_config.obfuscation.*`) calls the same two sets `keep_values` and `obfuscate_sql_values`. Rather than rename the Rust fields — `transform_keys` is no longer SQL-specific here, so `obfuscate_sql_values` would be a lie — both Agent spellings are accepted as `#[serde(alias = ...)]`, in the same style as #2490's PascalCase aliases. `obfuscation_config::tests::test_agent_json_obfuscation_field_names` pins all of it.

BREAKING CHANGE: `JsonObfuscatorConfig::transformer` and `JsonStringTransformer` are removed. JSON transforms move to `JsonObfuscator::obfuscate_with` or `JsonObfuscator::obfuscate_into`. SQL obfuscation functions now return `Result`.

# How to test the change?

The following pass on the rebased branch:

```bash
cargo +stable clippy -p libdd-trace-obfuscation --all-targets -- -D warnings
cargo +nightly-2026-07-26 fmt --all -- --check
cargo nextest run -p libdd-trace-obfuscation           # 387 passed
cargo test -p libdd-trace-obfuscation --doc
cargo check -p libdd-trace-stats -p libdd-data-pipeline-core \
  -p libdd-data-pipeline -p libdd-data-pipeline-ffi --all-targets
cargo nextest run -p libdd-data-pipeline -p libdd-trace-stats \
  -E '!test(tracing_integration_tests::)'              # 253 passed
```

The `tracing_integration_tests::` suite needs Docker and was not run locally. `Cargo.lock` gains only `thiserror`, which `LICENSE-3rdparty.csv` already covers, so no regeneration was needed. `cargo deny check` still reports pre-existing workspace advisory and license-policy failures unrelated to this diff.

# References

- Closes: #2541
- Related: #2490
- Related: DataDog/saluki#2601


Co-authored-by: matt.briggs <matt.briggs@datadoghq.com>
paullegranddc added a commit that referenced this pull request Sep 24, 2026
@dd-octo-sts @github-actions @paullegranddc
chore(release): proposal for libdd-capabilities-impl, libdd-common, l…
6163add 
…ibdd-data-pipeline, libdd-li... (#2571)

<!-- release-proposal-inputs: {"crates":"libdd-capabilities-impl,
libdd-common, libdd-data-pipeline, libdd-library-config,
libdd-profiling-heap-allocator, libdd-remote-config, libdd-sampling,
libdd-shared-runtime, libdd-telemetry, libdd-tinybytes,
libdd-trace-utils","main_start_ref":"","level_overrides":"","bypass_standard_checks":false}
-->

# Release proposal for libdd-capabilities-impl, libdd-common,
libdd-data-pipeline, libdd-library-config,
libdd-profiling-heap-allocator, libdd-remote-config, libdd-sampling,
libdd-shared-runtime, libdd-telemetry, libdd-tinybytes,
libdd-trace-utils and their dependencies

This PR contains version bumps based on public API changes and commits
since last release.


### ⚠️ Crates left out of this proposal affected by its major
bumps

These publishable workspace crates are not part of this release but
their dependency requirement was rewritten on this branch while their
published version still requires the old major. If they are a dependency
on your deployment not including them in the release could result in
duplicate packages or symbol incompatibility.

- `libdd-capabilities` `3.0.1` → `4.0.0` affects: `libdd-crashtracker`,
`libdd-ffe`, `libdd-live-debugger`
- `libdd-capabilities-impl` `5.0.0` → `6.0.0` affects:
`libdd-crashtracker`, `libdd-live-debugger`, `libdd-tracer-flare`
- `libdd-common` `6.0.0` → `7.0.0` affects: `libdd-crashtracker`,
`libdd-ffe`, `libdd-http-client`, `libdd-ipc`, `libdd-live-debugger`,
`libdd-profiling`, `libdd-tracer-flare`
- `libdd-data-pipeline` `10.0.0` → `11.0.0` affects:
`libdd-live-debugger`
- `libdd-remote-config` `5.0.0` → `6.0.0` affects: `libdd-ffe`,
`libdd-live-debugger`, `libdd-tracer-flare`
- `libdd-telemetry` `8.0.0` → `9.0.0` affects: `libdd-crashtracker`
- `libdd-trace-stats` `9.0.0` → `10.0.0` affects: `libdd-ipc`
- `libdd-trace-utils` `12.0.0` → `13.0.0` affects: `libdd-tracer-flare`

## libdd-capabilities
**Next version:** `4.0.0`
**Semver bump:** `major`
**Tag:** `libdd-capabilities-v4.0.0`

### Commits

- feat(trace-exporter)!: add API to force flush stats on the trace
exporter (#2538)
- refactor!: apply small timeout pooling strategy to libdd-http-client
as well (#2449)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)

## libdd-common
**Next version:** `7.0.0`
**Semver bump:** `major`
**Tag:** `libdd-common-v7.0.0`

### Commits

- build: enforce workspace-level dependency declarations (#2514)
- chore!: remove unused dependencies (v2) (#2511)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)
- chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318)
- fix!: benchmark was doing many samples for deterministic heap usage
(#2465)

## libdd-ddsketch
**Next version:** `1.1.2`
**Semver bump:** `patch`
**Tag:** `libdd-ddsketch-v1.1.2`

### Commits

- chore: move all remaining external deps to workspace-level
dependencies (#2476)
- chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318)

## libdd-profiling-heap-sampler
**Next version:** `1.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-profiling-heap-sampler-v1.1.0`

### Commits

- chore: move all remaining external deps to workspace-level
dependencies (#2476)
- refactor: migrate to workspace dependencies, phase 4 (#2296)
- feat(profiling): add USDT semaphores to skip slow path around heap
profile sampler (#2266)

## libdd-tinybytes
**Next version:** `1.1.4`
**Semver bump:** `patch`
**Tag:** `libdd-tinybytes-v1.1.4`

### Commits

- chore: move all remaining external deps to workspace-level
dependencies (#2476)

## libdd-trace-protobuf
**Next version:** `5.0.1`
**Semver bump:** `patch`
**Tag:** `libdd-trace-protobuf-v5.0.1`

### Commits

- chore: move all remaining external deps to workspace-level
dependencies (#2476)
- fix(build): build script change detection (#2467)

## libdd-capabilities-impl
**Next version:** `6.0.0`
**Semver bump:** `major`
**Tag:** `libdd-capabilities-impl-v6.0.0`

### ⚠️ major bump forced due to:

- `libdd-capabilities`: ^3.0.1 → ^4.0.0
- `libdd-common`: ^6.0.0 → ^7.0.0

### Commits

- refactor!: apply small timeout pooling strategy to libdd-http-client
as well (#2449)
- fix(sidecar)!: stabilize the sidecar for macos usage (#2475)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)

## libdd-profiling-heap-allocator
**Next version:** `1.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-profiling-heap-allocator-v1.1.0`

### Commits

- refactor(benchmarks): measure allocator thread CPU time (#2500)
- refactor: migrate to workspace dependencies, phase 4 (#2296)
- feat(profiling): add USDT semaphores to skip slow path around heap
profile sampler (#2266)

## libdd-library-config
**Next version:** `4.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-library-config-v4.1.0`

### Commits

- chore!: remove unused dependencies (v2) (#2511)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)

## libdd-trace-normalization
**Next version:** `4.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-trace-normalization-v4.1.0`

### Commits

- chore: move all remaining external deps to workspace-level
dependencies (#2476)
- chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318)

## libdd-remote-config
**Next version:** `6.0.0`
**Semver bump:** `major`
**Tag:** `libdd-remote-config-v6.0.0`

### ⚠️ major bump forced due to:

- `libdd-capabilities`: ^3.0.1 → ^4.0.0
- `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0
- `libdd-common`: ^6.0.0 → ^7.0.0

### Commits

- refactor!: apply small timeout pooling strategy to libdd-http-client
as well (#2449)
- chore!: remove unused dependencies (v2) (#2511)
- fix(rem-cfg)!: remove AsmRawResponseBody (#2540)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)

## libdd-shared-runtime
**Next version:** `5.0.0`
**Semver bump:** `major`
**Tag:** `libdd-shared-runtime-v5.0.0`

### ⚠️ major bump forced due to:

- `libdd-capabilities`: ^3.0.1 → ^4.0.0
- `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0
- `libdd-common`: ^6.0.0 → ^7.0.0

### Commits

- feat(trace-exporter)!: add API to force flush stats on the trace
exporter (#2538)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)

## libdd-trace-utils
**Next version:** `13.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-utils-v13.0.0`

### ⚠️ major bump forced due to:

- `libdd-capabilities`: ^3.0.1 → ^4.0.0
- `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0
- `libdd-common`: ^6.0.0 → ^7.0.0

### Commits

- fix(trace-utils)!: add forgotten css to agentless v1 encoder (#2557)
- fix(otlp): use potential dedupe in otlp serialization path (#2531)
- feat(trace-utils)!: add v1-native OTLP encoder brick (#2369)
- perf(serialization): bench the actual path used by the trace exporter
(#2512)
- perf(trace): write agentless JSON directly (#2454)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)
- perf(trace): encode agentless IDs without formatting (#2452)
- perf(trace-utils)!: use pooled spans on the send path (#2382)
- feat(data-pipeline)!: OTLP gRPC trace export (#2171)
- chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318)
- fix!: benchmark was doing many samples for deterministic heap usage
(#2465)

## libdd-dogstatsd-client
**Next version:** `7.0.0`
**Semver bump:** `major`
**Tag:** `libdd-dogstatsd-client-v7.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^6.0.0 → ^7.0.0
- `libdd-shared-runtime`: ^4.0.0 → ^5.0.0

### Commits

- chore: move all remaining external deps to workspace-level
dependencies (#2476)

## libdd-telemetry
**Next version:** `9.0.0`
**Semver bump:** `major`
**Tag:** `libdd-telemetry-v9.0.0`

### ⚠️ major bump forced due to:

- `libdd-capabilities`: ^3.0.1 → ^4.0.0
- `libdd-common`: ^6.0.0 → ^7.0.0
- `libdd-shared-runtime`: ^4.0.0 → ^5.0.0

### Commits

- refactor!: apply small timeout pooling strategy to libdd-http-client
as well (#2449)
- chore!: remove unused dependencies (v2) (#2511)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)
- fix(telemetry): fall back to the proper intake telemetry domain when
direct submission is enabled (#2484)

## libdd-sampling
**Next version:** `7.0.0`
**Semver bump:** `major`
**Tag:** `libdd-sampling-v7.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^5.2.0 → ^7.0.0
- `libdd-trace-utils`: ^10.1.0 → ^13.0.0

### Commits

- chore(benchmarks): remove batched-loop sampling noise (#2498)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)
- fix!: benchmark was doing many samples for deterministic heap usage
(#2465)
- feat(trace-utils)!: add from owned to SpanText (#2403)
- test(sampling): make rate_limiter thread-safety test deterministic
(#2354)

## libdd-trace-obfuscation
**Next version:** `9.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-obfuscation-v9.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^6.0.0 → ^7.0.0
- `libdd-trace-utils`: ^12.0.0 → ^13.0.0

### Commits

- feat(data-pipeline)!: refactor agent's /info obfuscation config format
(#2490)
- fix(obfuscation): keep URLs whose path or fragment holds a bracket
(#2527)
- fix(obfuscation): stop forcing serde_json/preserve_order on dependents
(#2529)
- fix(obfuscation): prevent crash and cap recursion in SQL/HTTP
obfuscation (#2441)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)
- fix(data-pipeline-ffi): harden agentless config setters and parse
obfuscation config once (#2474)

## libdd-trace-stats
**Next version:** `10.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-stats-v10.0.0`

### ⚠️ major bump forced due to:

- `libdd-capabilities`: ^3.0.1 → ^4.0.0
- `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0
- `libdd-common`: ^6.0.0 → ^7.0.0
- `libdd-shared-runtime`: ^4.0.0 → ^5.0.0
- `libdd-trace-obfuscation`: ^8.0.0 → ^9.0.0
- `libdd-trace-utils`: ^12.0.0 → ^13.0.0

### Commits

- feat(trace-exporter)!: add API to force flush stats on the trace
exporter (#2538)
- fix(trace-stats): cardinality limit telemetry name (#2559)
- feat(data-pipeline)!: refactor agent's /info obfuscation config format
(#2490)
- chore!: remove unused dependencies (v2) (#2511)
- feat(data-pipeline)!: generate agentless trace stats (#2488)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)
- chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318)

## libdd-data-pipeline-core
**Next version:** `2.0.0`
**Semver bump:** `major`
**Tag:** `libdd-data-pipeline-core-v2.0.0`

### ⚠️ major bump forced due to:

- `libdd-capabilities`: ^3.0.1 → ^4.0.0
- `libdd-common`: ^6.0.0 → ^7.0.0
- `libdd-trace-obfuscation`: ^8.0.0 → ^9.0.0
- `libdd-trace-utils`: ^12.0.0 → ^13.0.0

### Commits

- refactor!: apply small timeout pooling strategy to libdd-http-client
as well (#2449)
- feat(data-pipeline)!: generate agentless trace stats (#2488)
- perf(data-pipeline): avoid parsing static entity header names (#2457)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)
- perf(trace-utils)!: use pooled spans on the send path (#2382)

## libdd-data-pipeline
**Next version:** `11.0.0`
**Semver bump:** `major`
**Tag:** `libdd-data-pipeline-v11.0.0`

### ⚠️ major bump forced due to:

- `libdd-capabilities`: ^3.0.1 → ^4.0.0
- `libdd-capabilities-impl`: ^5.0.0 → ^6.0.0
- `libdd-common`: ^6.0.0 → ^7.0.0
- `libdd-data-pipeline-core`: ^1.0.0 → ^2.0.0
- `libdd-shared-runtime`: ^4.0.0 → ^5.0.0
- `libdd-trace-obfuscation`: ^8.0.0 → ^9.0.0
- `libdd-trace-stats`: ^9.0.0 → ^10.0.0
- `libdd-trace-utils`: ^12.0.0 → ^13.0.0

### Commits

- feat(trace-exporter)!: add API to force flush stats on the trace
exporter (#2538)
- refactor!: apply small timeout pooling strategy to libdd-http-client
as well (#2449)
- feat(data-pipeline)!: refactor agent's /info obfuscation config format
(#2490)
- test(data-pipeline): stop OTLP gRPC post-connect test from hanging
(#2549)
- chore!: remove unused dependencies (v2) (#2511)
- feat(data-pipeline)!: generate agentless trace stats (#2488)
- fix(otlp): use potential dedupe in otlp serialization path (#2531)
- refactor(benchmarks): isolate enqueue timing in data-pipeline (#2499)
- perf(trace): write agentless JSON directly (#2454)
- test(data-pipeline): fix flaky OTLP gRPC exporter integration tests
(#2505)
- chore: move all remaining external deps to workspace-level
dependencies (#2476)
- perf(trace-utils)!: use pooled spans on the send path (#2382)
- feat(data-pipeline)!: OTLP gRPC trace export (#2171)
- chore(libdd-data-pipeline): solve cargo deny [APMSP-3884] (#2318)


[APMSP-3884]:
https://datadoghq.atlassian.net/browse/APMSP-3884?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: paullegranddc <82819397+paullegranddc@users.noreply.github.com>
@Eldolfin Eldolfin mentioned this pull request Sep 29, 2026
Merged
gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Oct 1, 2026
@Eldolfin
fix(data-pipeline)!: revert changes that made /info un-parsable (#2586)
8040933 
# What does this PR do?
Revert some changes from #2490 that made /info un-parsable in system tests

# Motivation

What inspired you to submit this pull request?

# Additional Notes
Uses a new dependency `serde_ignored` as an alternative to putting `serde(deny_unknown_fields)` on structs to make parsing "strict" only for tests and we accept unknown fields otherwise.

# How to test the change?

Describe here in detail how the change can be validated.


Co-authored-by: oscar.ledauphin <oscar.ledauphin@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments
@ichinaski ichinaski ichinaski approved these changes
@yannham yannham yannham approved these changes
@VianneyRuhlmann VianneyRuhlmann VianneyRuhlmann approved these changes

Assignees

No one assigned

Labels

data-pipeline mergequeue-status: done mini-agent

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Eldolfin @ichinaski @yannham @VianneyRuhlmann