feat(agentless)!: vendor rust-tuf crate in libdatadog for release - #2365
gh-worker-dd-mergequeue-cf854d[bot] merged 11 commits into
Conversation
Vendor the tuf crate from https://github.com/DataDog/rust-tuf at commit eb129ccad320b11e8bf99d2f0ff2c415a0795ccb into vendor/tuf/, renaming the package from 'tuf' to 'libdd-tuf-rust' (the [lib] name stays 'tuf' so all import paths in the codebase remain unchanged). libdd-remote-config now depends on the local path instead of a git reference, using a package alias so no import paths need to change: tuf = { package = "libdd-tuf-rust", path = "../vendor/tuf", ... }
- Replace deprecated chrono APIs throughout (in both production code and
tests):
- `Utc.ymd(y, m, d).and_hms(h, min, s)`
→ `Utc.with_ymd_and_hms(y, m, d, h, min, s).unwrap()`
- `DateTime::from_utc(NaiveDateTime::from_timestamp(0, 0), Utc)`
→ `DateTime::from_timestamp(0, 0).unwrap()`
- Remove now-unused `use chrono::NaiveDateTime` import.
- crypto: elide needless explicit lifetime `'a` on `retain_supported_hashes`
- crypto: fix non-canonical `PartialOrd` impls on `PublicKey` and `Signature`
to delegate to `Ord::cmp` (clippy::non_canonical_partial_ord_impl)
- crypto: replace manual `BuildHasher`/`Hasher` pair in test with
`BuildHasher::hash_one` (clippy::manual_hash_one)
- metadata: replace `.into_iter().map(|(_k, v)| v)` on a `HashMap` with
`.into_values()` (clippy::iter_kv_map)
- metadata: fix doc-comment list-continuation indentation (4 items,
clippy::doc_lazy_continuation)
- lib: fix doc-comment list-continuation indentation (4 items)
- repo_builder: remove redundant `.into_iter()` calls on `Vec` passed to
`.extend()` (clippy::useless_conversion)
- repository: replace `::std::usize::MAX` with `usize::MAX`
(clippy::legacy_numeric_constants)
- repository/ephemeral: remove redundant `.into_iter()` (clippy::useless_conversion)
- repository/file_system: add explicit `'_` lifetime to `FileSystemBatchUpdate`
return type (mismatched_lifetime_syntaxes); remove needless `&` borrows on
`Path` args to `File::open` (clippy::needless_borrows_for_generic_args)
- repository/http: elide needless `'a` lifetime on `get`; replace
`io::Error::new(io::ErrorKind::Other, e)` with `io::Error::other(e)`
(clippy::io_other_error)
Reformat all source files in vendor/tuf/ with the workspace rustfmt configuration (max_width=100, wrap_comments=true, format_code_in_doc_comments=true, etc.). This commit is specific to the libdatadog vendoring and is not intended for upstream application.
|
These lints only surface when the `unstable_rsa` feature is enabled (the CI runs `--all-features`, but the previous local clippy run did not). - Replace three uses of the deprecated `ring::rsa::KeyPair::public_modulus_len()` with `key.public().modulus_len()` (per the deprecation notice). - Remove needless `&` borrows on two array literals passed to `Command::args()` (clippy::needless_borrows_for_generic_args).
…y CSV - .github/workflows/lint.yml: add `-not -path "./vendor/*"` to the licensecheck find command, alongside the existing exclusions for symbolizer-ffi, datadog-ipc/plugins, and datadog-ipc/tarpc. Vendored code carries its own upstream licences and must not be required to carry Datadog copyright headers. - LICENSE-3rdparty.csv: regenerated with `dd-rust-license-tool dump`. The old git-sourced `tuf` entry is removed because libdd-tuf-rust is now a workspace path dependency (first-party code) and the tool correctly omits workspace members from the third-party list.
paullegranddc
changed the title
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
Replace the two late-initialized `let` bindings (`targets_version`,
`snapshot_version`) with a single `let (…, …) = if … { … } else { … }`
expression (clippy::needless_late_init, nightly only).
The crate depends on ring (FFI, assembly) and async I/O helpers that are
not Miri-compatible. Skipping is done at the crate level with two
complementary guards:
* vendor/tuf/src/lib.rs — inner attribute `#![cfg(not(all(test, miri)))]`
strips the entire compilation unit when the crate is the direct Miri
test target (`cfg(test)` is only set on the root crate under test, not
on dependencies). This removes all `#[cfg(test)]` modules without
touching each one individually.
* vendor/tuf/tests/{integration,simple_example}.rs — inner attribute
`#![cfg(not(miri))]` on each integration-test file. These are
separate crates compiled with `cfg(test)=true` themselves, so the
lib.rs guard above would not reach them.
BenchmarksComparisonBenchmark execution time: 2026-08-18 23:00:18 Comparing candidate commit 80a615d in PR branch Found 0 performance improvements and 4 performance regressions! Performance is the same for 164 metrics, 10 unstable metrics.
|
The ECDSA canonical-JSON fixture (tests/ecdsa/ecdsa_root.canonical) embeds PEM-encoded public keys whose internal LF newlines are part of the signed bytes. Git's autocrlf on Windows converts those LFs to CRLF on checkout, producing different bytes from what the signature was computed over, causing the verify_ecdsa_signatures_against_canonical_signed_body test to panic with "BadSignature". Add a vendor/tuf/.gitattributes that marks all files under tests/ecdsa/, tests/ed25519/, and tests/rsa/ as binary so Git never touches their line endings.
![datadog-datadog-prod-us1[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
There was a problem hiding this comment.
The vendoring switch cannot currently be released: libdd-remote-config’s path dependency lacks a publication version, while the copied TUF README and license symlinks resolve to nonexistent files. Cargo packaging therefore fails before the agentless crates can be published.
🤖 Datadog Autotest · Commit 535a9da · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
| hashbrown = "0.15" | ||
| # branch = "opw-develop" | ||
| tuf = { git = "https://github.com/DataDog/rust-tuf/", rev = "eb129ccad320b11e8bf99d2f0ff2c415a0795ccb", default-features = false, optional = true } | ||
| tuf = { package = "libdd-tuf-rust", path = "../vendor/tuf", default-features = false, optional = true } |
There was a problem hiding this comment.
A future release containing agentless remote config cannot package and publish libdd-remote-config.
Assertion details
- Input: Packaging libdd-remote-config for publication, even though the TUF dependency is optional.
- Expected:
Declare the vendored package's registry version alongside its local path so Cargo can rewrite the dependency for publication. - Actual: The new normal, optional path dependency has no
version. Cargo requires a version for dependencies when packaging a publishable crate, and the repository's other normal path dependencies follow that rule.
| tuf = { package = "libdd-tuf-rust", path = "../vendor/tuf", default-features = false, optional = true } | |
| tuf = { package = "libdd-tuf-rust", version = "0.3.0-beta10", path = "../vendor/tuf", default-features = false, optional = true } |
Was this helpful? React 👍 or 👎
🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
| @@ -0,0 +1 @@ | |||
| ../README.md No newline at end of file | |||
There was a problem hiding this comment.
Packaging the new libdd-tuf-rust crate fails before it can be released.
Assertion details
- Input: The release workflow packages the newly added libdd-tuf-rust workspace member.
- Expected:
Replace all three broken symlinks with the corresponding upstream README and license contents so the published crate contains readable metadata and licensing material. - Actual:
README.md, LICENSE-APACHE, and LICENSE-MIT still point one level upward as they did in the original rust-tuf layout. Under vendor/tuf those targets resolve to nonexistent vendor/README.md and vendor/LICENSE-* files.
Was this helpful? React 👍 or 👎
🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 535a9da309
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -0,0 +1 @@ | |||
| ../README.md No newline at end of file | |||
There was a problem hiding this comment.
On Unix checkouts, this symlink resolves to vendor/README.md, which does not exist; the two added license symlinks similarly resolve to missing vendor/LICENSE-APACHE and vendor/LICENSE-MIT. Because vendor/tuf/Cargo.toml declares this path as its package readme, cargo package -p libdd-tuf-rust exits with readme README.md does not appear to exist, blocking the release workflow this change is intended to enable and omitting the vendored license texts. Copy the referenced upstream files into the package or adjust the links to real repository files.
Useful? React with 👍 / 👎.
| - name: Check licenses | ||
| # Exclude symbolizer-ffi from the checks (mostly imported code) | ||
| run: '! find . \( -name "*.rs" -o -name "*.c" -o -name "*.sh" \) -not -path "./symbolizer-ffi/*" -not -path "./datadog-ipc/plugins/*" -not -path "./datadog-ipc/tarpc/*" -print0 | xargs -0 licensecheck -c ".*" | grep -v "Apache License 2.0"' | ||
| run: '! find . \( -name "*.rs" -o -name "*.c" -o -name "*.sh" \) -not -path "./symbolizer-ffi/*" -not -path "./datadog-ipc/plugins/*" -not -path "./datadog-ipc/tarpc/*" -not -path "./vendor/*" -print0 | xargs -0 licensecheck -c ".*" | grep -v "Apache License 2.0"' |
There was a problem hiding this comment.
This new exclusion hides every file under vendor/ from the license-header job, and all 22 added Rust files under vendor/tuf lack the required Apache 2.0 header. The repository convention applies to all source files except symbolizer-ffi, so either add the required headers and keep these files covered by the check or narrow the exception without bypassing that invariant.
AGENTS.md reference: AGENTS.md:L90-L94
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
No, every vendored crate should be excluded, this is voluntary
| [package] | ||
| name = "libdd-tuf-rust" | ||
| edition = "2021" | ||
| version = "0.3.0-beta10" |
There was a problem hiding this comment.
When the inspected release-proposal-dispatch.yml flow targets libdd-remote-config, publication-order.sh now includes this new publishable dependency, but there is no libdd-tuf-rust-v0.3.0-beta10 tag. The workflow therefore enters its initial-release branch and explicitly exits unless the manifest version is 0.1.0 (lines 586–598), so setting the new crate's initial version to the upstream 0.3.0-beta10 blocks release proposals before publishing. Initialize it at the version required by the workflow or seed/handle the imported version explicitly.
Useful? React with 👍 / 👎.
…rate from a workspace
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
gh-worker-dd-devflow-36fce6
Bot
added
mergequeue-status: queued
mergequeue-status: in_progress
and removed
mergequeue-status: queued
labels
gh-worker-dd-devflow-36fce6
Bot
added
mergequeue-status: done
and removed
mergequeue-status: in_progress
labels
…#2374) # Motivation Using the libdatadog release system for vendored crate might be problematic, so this PR reverts the vendoring and instead depends on the new package we pushed to crates.io from the forked repo
…ker, libdd-data-pipeline, li... (#2409) # Release proposal for libdd-capabilities-impl, libdd-crashtracker, libdd-data-pipeline, libdd-http-client, libdd-ipc, libdd-library-config, libdd-otel-thread-ctx, libdd-remote-config, libdd-shared-runtime, libdd-telemetry, libdd-tracer-flare and their dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-gotter **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-gotter-v1.1.0` ### Commits - fix(crashtracker): fix vdso recognition during GOT patching on musl (#2379) ## libdd-ipc-macros **Next version:** `1.0.0` **Semver bump:** `major` **Tag:** `libdd-ipc-macros-v1.0.0` **Warning:** this is an initial release. Please verify that the version and commits included are correct. ## libdd-trace-protobuf **Next version:** `5.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-protobuf-v5.0.0` ### Commits - chore(trace-protobuf)!: sync span.proto with datadog-agent (#2356) ## libdd-library-config **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-library-config-v4.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.0 → ^5.0.0 ### Commits - fix(library-config): omit absent process context attributes (#2337) - refactor: migrate to workspace dependencies, phase 4 (#2296) - refactor: migrate to workspace dependencies, phase 3 (#2283) - refactor: consolidate core dependencies at workspace level (phase 1) (#2253) - fix(library-config): update Linux process context (#2237) - refactor(library-config): reorganize Linux process context (#2228) - feat(library-config): otel process context reader (#2176) ## libdd-remote-config **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-remote-config-v4.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 ### Commits - feat(rc): add overrides for RC config/director roots (#2404) - fix(rc): avoid depending on the crypto nodejs API for remote config (#2407) - fix(rc)!: fixup rc to be wasm compatible (#2393) - feat: Revert vendoring of tuf-rust (#2365), use new crates.io package (#2374) - feat(agentless)!: vendor rust-tuf crate in libdatadog for release (#2365) - chore(rc)!: add DEBUG product (#2306) - feat(remote-config)!: agentless RC fetcher (#2112) ## libdd-trace-utils **Next version:** `11.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-utils-v11.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 ### Commits - feat(compression): use zrip for wasm targets (#2386) - fix(trace-utils)!: fix forward compatibility for string streaming (#2295) ## libdd-tracer-flare **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-tracer-flare-v2.0.0` ###⚠️ major bump forced due to: - `libdd-remote-config`: ^3.0.0 → ^4.0.0 - `libdd-trace-utils`: ^10.0.0 → ^11.0.0 ### Commits - feat(remote-config)!: agentless RC fetcher (#2112) ## libdd-crashtracker **Next version:** `2.0.1` **Semver bump:** `patch` **Tag:** `libdd-crashtracker-v2.0.1` ### Commits - fix(crashtracker): hold the collector connection open through symbolization (#2384) - fix(crashtracking): stop resolving thread symbols with libunwind in the receiver (#2361) ## libdd-data-pipeline **Next version:** `9.0.0` **Semver bump:** `major` **Tag:** `libdd-data-pipeline-v9.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 - `libdd-trace-utils`: ^10.1.0 → ^11.0.0 ### Commits - feat(compression): use zrip for wasm targets (#2386) ## libdd-ipc **Next version:** `1.0.0` **Semver bump:** `major` **Tag:** `libdd-ipc-v1.0.0` **Warning:** this is an initial release. Please verify that the version and commits included are correct. ## libdd-trace-normalization **Next version:** `4.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-normalization-v4.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 ## libdd-trace-obfuscation **Next version:** `7.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-obfuscation-v7.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 - `libdd-trace-utils`: ^10.1.0 → ^11.0.0 ## libdd-trace-stats **Next version:** `8.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-stats-v8.0.0` ###⚠️ major bump forced due to: - `libdd-trace-protobuf`: ^4.0.1 → ^5.0.0 - `libdd-trace-utils`: ^10.1.0 → ^11.0.0 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: hoolioh <107922352+hoolioh@users.noreply.github.com>
4 participants
What does this PR do?
libdd-tuf-rustMotivation
The
tufcrate was added as a git dependency to support agentless remote config.There is a pushed crate for
tuf, but we depend on a fork of the main repo (Datadog/rust-tuf) which does not exist as a crate.Having git dependencies, even if optional prevents pushing to crates.io so we are blocked for releases.
This is vendored under a new name so that we can push the forked code