feat(compression): use zrip for wasm targets - #2386
Merged
Merged
Conversation
Contributor
📚 Documentation Check Results📦
|
Contributor
🔒 Cargo Deny Results📦
|
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 2a25294 | Docs | View more details | Give us feedback! |
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
BenchmarksComparisonBenchmark execution time: 2026-08-21 16:27:28 Comparing candidate commit 2a25294 in PR branch Found 1 performance improvements and 0 performance regressions! Performance is the same for 151 metrics, 0 unstable metrics.
|
rochdev
marked this pull request as ready for review
August 20, 2026 18:37
paullegranddc
approved these changes
Aug 21, 2026
bengl
reviewed
Aug 21, 2026
| // The zstd bindings use FFI so they don't work under miri. This means the | ||
| // buffer isn't compressed, so simply convert to a vec. | ||
| // Native zstd uses FFI and is unavailable under Miri, where the default | ||
| // profile codec remains uncompressed. |
Contributor
There was a problem hiding this comment.
Why change this comment? The related config change seems to be about wasm, but I don't see that in the comment.
| /// size, and the implementation may use something different. | ||
| /// - `max_capacity`: the maximum size for the output buffer (hard limit). | ||
| /// - `compression_level`: see [`zstd::Encoder::new`] for the valid range. | ||
| /// - `compression_level`: must be supported by the target's zstd encoder. |
Contributor
There was a problem hiding this comment.
Are the ranges different for each encoder? Should they be codified here as enums?
| }); | ||
| #[cfg(target_arch = "wasm32")] | ||
| let result = zrip::FrameEncoder::new(writer, level) | ||
| .map_err(std::io::Error::other) |
Contributor
There was a problem hiding this comment.
It looks like this map_err is the only difference between the two. In either case we do the write_all call. Can you shorten the length of these branches, since there's very little different between the two? For example, initialization should probably be abstracted out into a helper function elsewhere.
gh-worker-dd-mergequeue-cf854d Bot
pushed a commit
that referenced
this pull request
Aug 25, 2026
# What does this PR do? Addresses follow-up review feedback from #2386: - restores the pre-#2386 Miri FFI explanation in the profile test helper; - documents the WASM and native zstd compression-level contracts; - maps level `0` to level `3` on every target; - clamps WASM zrip levels to `-7..=4` while preserving native zstd's broader range; - reports the effective compression level from trace compression; - isolates target-specific trace encoder construction and shares the write/finish path; and - tests native passthrough and WASM clamping at both boundaries. # Motivation The zrip WASM backend supports a narrower compression-level range than native zstd and previously assigned a different meaning to level `0`. Out-of-range levels also disabled trace compression or failed profile compression under WASM. The target branches duplicated the common encoder flow. This follows up on these review threads: - #2386 (comment) - #2386 (comment) - #2386 (comment) # Additional Notes Native targets continue to accept the broader range reported by `zstd::compression_level_range()`. WASM clamps requested levels to zrip's supported range, so level `22` becomes `4`; level `0` becomes `3` on every target. The full trace-utils integration binary was not validated locally because Docker could not pull the private GHCR test-agent image. All 348 trace-utils library tests passed. Strict trace-utils clippy also reports an existing `collapsible_match` warning in unchanged `tracer_payload.rs`; clippy passes with that lint allowed. This PR was generated by Codex. # How to test the change? - `cargo test -p libdd-profiling` - `cargo test -p libdd-trace-utils --features compression --lib` - `env CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUNNER=wasm-bindgen-test-runner cargo test -p libdd-wasm-tests --target wasm32-unknown-unknown` - `cargo +stable clippy -p libdd-profiling --all-targets -- -D warnings` - `cargo +stable clippy -p libdd-wasm-tests --target wasm32-unknown-unknown --all-targets -- -D warnings` - `cargo +stable clippy -p libdd-trace-utils --all-targets --features compression -- -D warnings -A clippy::collapsible_match` - `cargo +nightly-2026-07-26 fmt --all -- --check` - `cargo test -p libdd-profiling --doc` - `cargo test -p libdd-trace-utils --features compression --doc` Co-authored-by: roch.devost <roch.devost@datadoghq.com>
hoolioh
added a commit
that referenced
this pull request
Aug 26, 2026
…ker, libdd-data-pipeline, li... (#2409) # Release proposal for libdd-capabilities-impl, libdd-crashtracker, libdd-data-pipeline, libdd-http-client, libdd-ipc, libdd-library-config, libdd-otel-thread-ctx, libdd-remote-config, libdd-shared-runtime, libdd-telemetry, libdd-tracer-flare and their dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-gotter **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-gotter-v1.1.0` ### Commits - fix(crashtracker): fix vdso recognition during GOT patching on musl (#2379) ## libdd-ipc-macros **Next version:** `1.0.0` **Semver bump:** `major` **Tag:** `libdd-ipc-macros-v1.0.0` **Warning:** this is an initial release. Please verify that the version and commits included are correct. ## libdd-trace-protobuf **Next version:** `5.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-protobuf-v5.0.0` ### Commits - chore(trace-protobuf)!: sync span.proto with datadog-agent (#2356) ## libdd-library-config **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-library-config-v4.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.0 → ^5.0.0 ### Commits - fix(library-config): omit absent process context attributes (#2337) - refactor: migrate to workspace dependencies, phase 4 (#2296) - refactor: migrate to workspace dependencies, phase 3 (#2283) - refactor: consolidate core dependencies at workspace level (phase 1) (#2253) - fix(library-config): update Linux process context (#2237) - refactor(library-config): reorganize Linux process context (#2228) - feat(library-config): otel process context reader (#2176) ## libdd-remote-config **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-remote-config-v4.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 ### Commits - feat(rc): add overrides for RC config/director roots (#2404) - fix(rc): avoid depending on the crypto nodejs API for remote config (#2407) - fix(rc)!: fixup rc to be wasm compatible (#2393) - feat: Revert vendoring of tuf-rust (#2365), use new crates.io package (#2374) - feat(agentless)!: vendor rust-tuf crate in libdatadog for release (#2365) - chore(rc)!: add DEBUG product (#2306) - feat(remote-config)!: agentless RC fetcher (#2112) ## libdd-trace-utils **Next version:** `11.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-utils-v11.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 ### Commits - feat(compression): use zrip for wasm targets (#2386) - fix(trace-utils)!: fix forward compatibility for string streaming (#2295) ## libdd-tracer-flare **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-tracer-flare-v2.0.0` ###⚠️ major bump forced due to: - `libdd-remote-config`: ^3.0.0 → ^4.0.0 - `libdd-trace-utils`: ^10.0.0 → ^11.0.0 ### Commits - feat(remote-config)!: agentless RC fetcher (#2112) ## libdd-crashtracker **Next version:** `2.0.1` **Semver bump:** `patch` **Tag:** `libdd-crashtracker-v2.0.1` ### Commits - fix(crashtracker): hold the collector connection open through symbolization (#2384) - fix(crashtracking): stop resolving thread symbols with libunwind in the receiver (#2361) ## libdd-data-pipeline **Next version:** `9.0.0` **Semver bump:** `major` **Tag:** `libdd-data-pipeline-v9.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 - `libdd-trace-utils`: ^10.1.0 → ^11.0.0 ### Commits - feat(compression): use zrip for wasm targets (#2386) ## libdd-ipc **Next version:** `1.0.0` **Semver bump:** `major` **Tag:** `libdd-ipc-v1.0.0` **Warning:** this is an initial release. Please verify that the version and commits included are correct. ## libdd-trace-normalization **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-normalization-v4.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 ## libdd-trace-obfuscation **Next version:** `7.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-obfuscation-v7.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 - `libdd-trace-utils`: ^10.1.0 → ^11.0.0 ## libdd-trace-stats **Next version:** `8.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-stats-v8.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 - `libdd-trace-utils`: ^10.1.0 → ^11.0.0 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: hoolioh <107922352+hoolioh@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Keeps the existing zstd backend for native targets and selects the pure-Rust zrip backend for wasm32 compression paths in profiling and trace utilities.
It also adds a small WASM-only runtime test package. The harness compiles the standalone production compression modules directly and runs two round-trip tests under Node, covering trace compression plus the profiling profile and observation codecs.
Motivation
Native zstd remains substantially faster, while zrip is lighter and performs better for WASM. Selecting the backend by target preserves native performance and removes the need for a C toolchain when compiling these compression paths to WASM.
Additional Notes
The wire format and Content-Encoding remain zstd-compatible.
This PR was generated by Codex.
How to test the change?
The existing WASM CI job now installs the matching wasm-bindgen test runner and executes:
Additional validation performed locally:
The WASM runtime suite executes two tests and both pass. The repository-wide cargo deny check still reports existing default-branch advisory and license-policy failures, including time 0.3.41 and spin 0.9.8.