Skip to content

fix(config): match the Agent's decoding of Go number literals and JSON string lists - #2767

Draft
jszwedko wants to merge 2 commits into
mainfrom
jszwedko/go-strconv-env-decoding
Draft

jszwedko wants to merge 2 commits into
mainfrom
jszwedko/go-strconv-env-decoding

Conversation

@jszwedko

@jszwedko jszwedko commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

ADP decodes its local file and environment strictly before it connects to the Datadog Agent's configuration stream, so any value the Agent accepts but ADP rejects stops ADP from starting at all, and any value ADP reads differently is wrong until the stream replaces it (or for good, when it's consumed before the stream connects, like agent_ipc.grpc_max_message_size). Today DD_DOGSTATSD_PORT=0x10 aborts the boot, 010 silently reads as 10 where the Agent reads 8, and DD_APM_OBFUSCATION_ELASTICSEARCH_KEEP_VALUES='["a","b"]' becomes one bogus element. This makes ADP read numeric strings and string lists exactly as the Agent does, by porting Go's strconv grammars into a shared lib/go-strconv crate (alongside lib/go-duration), including hexadecimal floats.

Refs #2753.

flowchart LR
    S["string value<br/>(env var, quoted YAML, stream string)"] --> T{leaf type}
    T -->|integer| I["trimZeroDecimal →<br/>go_strconv::parse_int<br/>(0x / 0o / 0b / 0-octal / _)"]
    T -->|number| F["go_strconv::parse_float<br/>(decimal, hex, _)"]
    T -->|boolean| B[go_strconv::parse_bool]
    T -->|"string list<br/>(no env_parser)"| L{"JSON list<br/>of strings?"}
    L -->|yes| J[that list]
    L -->|no| W[split on whitespace]
Loading

Out of scope, left to the related compatibility decisions: malformed inputs the Agent keeps as raw strings with a warning while ADP rejects them (bool words like on, 1.342177295e+08 for an integer, non-JSON maps/lists), covered by #2755 and #2756. Surrounding whitespace on a number stays trimmed (the Agent rejects it), and NaN/Inf stay rejected since JSON can't carry them.

The corpus-replay expectations for depth-env-hex, int-string-base-prefix (the DD_DOGSTATSD_PORT boot entry), and depth-env-json-list (keep_values) should flip once that harness lands; it isn't on main yet, so they aren't updated here.

Test plan

  • New go-strconv unit tests; every expected value was produced by Go 1.27's strconv
  • One-off differential run of go-strconv against Go on 200k random number-like strings (400k int/float checks, 0 mismatches); the harness isn't committed
  • New cast_de, env_decode, and list_de cases for base-prefixed integers, hex floats, and JSON-list strings
  • New env_reader test covering the recorded corpus inputs plus 010 → 8 and a hex float
  • every_scalar_leaf_accepts_the_agent_castable_form_of_its_type now writes every integer leaf in hexadecimal
  • New loaded.rs test drives real environment variables through LoadedConfiguration::load and connected startup (run)

🤖 Generated with Claude Code

…N string lists

ADP strictly decodes local configuration before connecting to the Agent's
configuration stream, so a value the Agent accepts but ADP rejects aborts
startup. Integers written with a base prefix (`0x10`) failed to decode, a
leading-zero integer (`010`) silently read as decimal instead of octal, and a
string-list value holding a JSON list read as a single element.

Add a `go-strconv` crate porting Go's `strconv.ParseBool`, `ParseInt(s, 0, 64)`,
and `ParseFloat(s, 64)` (including hexadecimal floats), and use it for every
string-to-scalar coercion. Read string lists as the Agent's `cast` fork does:
a JSON list of strings first, else split on whitespace.

Refs #2753

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pr-commenter

pr-commenter Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Binary Size Analysis (Agent Data Plane)

Baseline: 6e3f308 · Comparison: 89ce1df · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 38.30 MiB (baseline) vs 38.30 MiB (comparison)
Size Change: +3.37 KiB (+0.01%)

✅ Binary size difference within threshold

Changes by Module
Module File Size Symbols
anon.a6926c6c21417749435f71dba1b992d8.976.llvm.13372907567088152362 -20.54 KiB 1
anon.279cd0c8aa5ef62dc61a9f1718a6053a.971.llvm.9754541002489315505 +20.54 KiB 1
anon.87e7c38bfd928d17fa28187081c51bf0.2.llvm.13921711517418886414 -16.09 KiB 1
anon.b7e79ecc1fa08370689c421e17e1d962.2.llvm.4029315304454983450 +16.00 KiB 1
anon.87e7c38bfd928d17fa28187081c51bf0.768.llvm.13921711517418886414 -14.62 KiB 1
anon.b7e79ecc1fa08370689c421e17e1d962.768.llvm.4029315304454983450 +14.62 KiB 1
anon.53bb41fcd4d7cb0fd4749e93eb822af9.155.llvm.5834098439156321449 -14.40 KiB 1
anon.19a859942b192a51eb26b31229025fd2.167.llvm.636153364112708831 +14.40 KiB 1
anon.87e7c38bfd928d17fa28187081c51bf0.766.llvm.13921711517418886414 -12.84 KiB 1
anon.b7e79ecc1fa08370689c421e17e1d962.766.llvm.4029315304454983450 +12.75 KiB 1
core -11.88 KiB 5019
anon.9d1057ec6821611763596b0232be6ef0.693.llvm.6678353967787269721 -11.14 KiB 1
anon.37fe7b850d241588ba523c235f51112b.693.llvm.486224596522849875 +11.13 KiB 1
anon.37fe7b850d241588ba523c235f51112b.4.llvm.486224596522849875 +8.48 KiB 1
anon.9d1057ec6821611763596b0232be6ef0.4.llvm.6678353967787269721 -8.48 KiB 1
anon.87e7c38bfd928d17fa28187081c51bf0.704.llvm.13921711517418886414 -8.23 KiB 1
anon.b7e79ecc1fa08370689c421e17e1d962.704.llvm.4029315304454983450 +8.05 KiB 1
anon.87e7c38bfd928d17fa28187081c51bf0.668.llvm.13921711517418886414 -7.67 KiB 1
anon.b7e79ecc1fa08370689c421e17e1d962.668.llvm.4029315304454983450 +7.49 KiB 1
alloc +6.54 KiB 877
Detailed Symbol Changes
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW] +59.7Ki  [NEW] +59.4Ki    _<datadog_agent_config::generated::datadog_configuration::_::<impl serde_core::de::Deserialize for datadog_agent_config::generated::datadog_configuration::DatadogConfiguration>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::h236b5fc312348faf
  [NEW] +40.5Ki  [NEW] +40.4Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h9a19459df2de8b91
  [NEW] +34.9Ki  [NEW] +34.7Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::he445d971e8def247
  [NEW] +31.4Ki  [NEW] +31.2Ki    agent_data_plane::cli::dogstatsd::run_dogstatsd_command::_{{closure}}::hc5322d832104a0f1
  [NEW] +28.7Ki  [NEW] +28.6Ki    agent_data_plane::dogstatsd_contexts::artifact::for_each_record::h66586be016d80336
  [NEW] +28.4Ki  [NEW] +28.2Ki    datadog_agent_commons::ipc::client::RemoteAgentClient::connect::_{{closure}}::_{{closure}}::_{{closure}}::hb1a3d792f28832dc
  [NEW] +25.6Ki  [NEW] +25.4Ki    agent_data_plane::internal::remote_agent::run_remote_agent_registration_loop::_{{closure}}::hde6252f1951beac3
  [NEW] +25.0Ki  [NEW] +24.9Ki    agent_data_plane::main::_{{closure}}::h452b8262f2c2c7c8
  [NEW] +23.5Ki  [NEW] +23.3Ki    agent_data_plane::cli::debug::handle_debug_command::_{{closure}}::h2c5a2df53d7e5489
  [NEW] +22.3Ki  [NEW] +22.2Ki    agent_data_plane::internal::env::ADPEnvironmentProvider::from_configuration::_{{closure}}::h60e9684f98a1c42e
  +0.0% +3.84Ki  +0.2% +12.5Ki    [19331 Others]
  [DEL] -22.3Ki  [DEL] -22.2Ki    agent_data_plane::internal::env::ADPEnvironmentProvider::from_configuration::_{{closure}}::hee0e8d3a30af396e
  [DEL] -23.5Ki  [DEL] -23.3Ki    agent_data_plane::cli::debug::handle_debug_command::_{{closure}}::hc84068d640a2d683
  [DEL] -24.9Ki  [DEL] -24.8Ki    agent_data_plane::main::_{{closure}}::h3ce278d5f382d5fc
  [DEL] -25.6Ki  [DEL] -25.4Ki    agent_data_plane::internal::remote_agent::run_remote_agent_registration_loop::_{{closure}}::h1f55b7becb8f9961
  [DEL] -28.4Ki  [DEL] -28.2Ki    datadog_agent_commons::ipc::client::RemoteAgentClient::connect::_{{closure}}::_{{closure}}::_{{closure}}::h1bd6a026eeafe819
  [DEL] -28.7Ki  [DEL] -28.6Ki    agent_data_plane::dogstatsd_contexts::artifact::for_each_record::hb60d5262ffff75a0
  [DEL] -31.4Ki  [DEL] -31.2Ki    agent_data_plane::cli::dogstatsd::run_dogstatsd_command::_{{closure}}::h2f5246c806e7f392
  [DEL] -34.9Ki  [DEL] -34.7Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::haa79196f741be08a
  [DEL] -40.5Ki  [DEL] -40.4Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h02ed82f16b54dbc0
  [DEL] -60.3Ki  [DEL] -59.9Ki    _<datadog_agent_config::generated::datadog_configuration::_::<impl serde_core::de::Deserialize for datadog_agent_config::generated::datadog_configuration::DatadogConfiguration>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::h7e9578d759076d6a
  +0.0% +3.37Ki  +0.0% +12.0Ki    TOTAL

…nv-decoding

# Conflicts:
#	lib/datadog-agent/config/src/list_de.rs
@pr-commenter

pr-commenter Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Regression Detector (Agent Data Plane)

Run ID: a09bbf59-f952-44fe-9531-c73501e1a4bd
Baseline: 6e3f3086 · Comparison: 89ce1df4 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
quality_gates_rss_dsd_ultraheavy memory ⚪ +0.47 metrics profiles logs
quality_gates_rss_dsd_medium memory ⚪ +0.37 metrics profiles logs
quality_gates_rss_idle memory ⚪ +0.23 metrics profiles logs
quality_gates_rss_dsd_heavy memory ⚪ +0.19 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ +0.14 metrics profiles logs
Bounds Checks: ✅ Passed (5)
experiment check replicates observed links
quality_gates_rss_dsd_heavy memory_usage 10/10 ✅ 231 MiB ≤ 250 MiB metrics profiles logs
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 51 MiB ≤ 60 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 93.5 MiB ≤ 100 MiB metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 ✅ 383 MiB ≤ 420 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 33.3 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression. Experiments tagged (no analysis) show ⚠️ n/a: SMP ran them but produced no analysis, usually because a replicate failed and exhausted its retries. Check the SMP report for that experiment's replicate failures.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant