Conversation
The Rust CLI's own dependencies are fetched by cargo from crates.io and never pass through pnpm or pnpr. This RFC adds a second ecosystem to pnpm (resolution, fetching, store, Cargo.lock, directory-source materialization) and a `cargo` registry protocol to pnpr (sparse index, download, web API, crates.io upstream), with a dogfooding plan for the pnpm repository. Ships in pacquet only, as a decided exception to the parity rule. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0167jVFmSFDDzaTKAvCEbC1Y
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…kspace trees Crate slots are unpacked once per machine under `<store>/crates/`; each workspace holds one symlink per locked crate in its directory sources. Verified against cargo 1.97 that directory sources follow symlinked entries, ignore dot-prefixed ones, and verify checksums at compile time. Records the per-workspace hardlinked tree as a rejected alternative and adds a self-contained layout as an open question. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0167jVFmSFDDzaTKAvCEbC1Y
|
Recorded the architecture exposed by the Cargo proof of concept in Written by an agent (Codex, gpt-5.6-sol). |
|
The verified-artifact ingestion slice has landed in pnpm/pnpm#14574. Commit ff49aaa updates this RFC with the implementation finding that integrity identifies immutable bytes, while projection inputs identify the extracted file map and therefore belong in every memory/disk cache key. It also records the legacy-row migration rules that fell out of review. Written by an agent (Codex, gpt-5.6-sol). |
|
Implementation update: pnpm/pnpm#14578 merged and the RFC now records the request-auth invariants learned from it. The important boundary is that the shared network layer routes already-formatted headers by URL. npm and Cargo configuration readers retain their own formats, schemes, and precedence rules. Cargo credential discovery happens once and only on the Cargo path; redirect destinations are authorized independently; parse failures do not echo credential contents. The current crates.io token is deliberately scoped to the index host, while named/private registry mapping remains a later Cargo source-identity slice. Written by an agent (Codex, gpt-5.6-sol). |
Document the real Python CLI integration in pnpm/pnpm#14586 and its validation of shared network, auth, and artifact storage. Record staged environment publication, native pylock interoperability, and remaining compatibility limits.
Route Python projects through production install and add coordination. Keep PEP 440/508 resolution and pylock.toml native to Python while sharing the HTTP/auth budget, verified ZIP ingestion, CAS, store index and reporter. Prepare complete virtual environments before publication, and retain mixed add metadata rollback. Include Python executables in run and exec paths. Exercise real wheels, native extensions, offline/frozen replay, and failures. Record the staged-publication and target-identity requirements in the RFC. New Python features are opt-in and target pnpm v12 only. Related to #14566 and pnpm/rfcs#34.
Extract workspace locking, metadata snapshots, concurrent preparation, settlement and publication into a dedicated install-coordinator crate. The lifecycle has no dependency on CLI configuration, ecosystem identity, resolver semantics or native lockfile formats. Let Cargo and Python declare metadata footprints and return prepared projections. Publish only after every participant settles. Reverse all attempted publications on failure, then restore metadata. Retain resources when rollback fails so recovery does not reference deleted generations. Move Cargo add preparation into its adapter and remove native file lists and Python-specific publication state from command pipelines. Sort Cargo results for deterministic publication despite concurrent preparation. Drain Cargo work before returning errors, including blocking store tasks. Exclude configured stores and caches from native workspace discovery before descent. Repeated installs must not reinterpret cached native manifests as workspace inputs. Keep unrelated projects with the same basenames visible. Keep npm's in-place materialization and lifecycle-script semantics explicit. Mixed add enrolls its metadata in rollback. The npm-only path creates no plan, inventory or transaction. No universal resolver or lockfile is added. Cover real cross-ecosystem preparation/publication failures and shared rollback/resource-lifetime contracts. Match Python's native Windows CRLF in the existing entry-point assertion found by remote CI. Related to #14566 and pnpm/rfcs#34.
Route tarball and ZIP ingestion through one cache and publication lifecycle. Share authenticated request setup, concurrency permits, HTTP status handling, bounded error-body draining, progress tracking and extraction retries. Keep tar streaming and ZIP decoding as container-specific strategies. Preserve npm cache keys, projection namespaces, revision-addressed requests, local tarball support and the existing in-memory hit fast path. The native adapters continue to own dependency rules and installed filesystem layouts. Input adapters borrow install-scoped handles without adding reference-count updates when constructing the shared ingestion request. Use the resolved retry policy for npm, Cargo and Python artifact requests. Replace Python's independent metadata retry loop with the shared network policy and apply it to Cargo sparse-index requests. Cover interrupted bodies, HTTP 408, credential isolation on redirects and credential-free diagnostics. Add cross-format contracts for cache validation, offline replay, projection selection, retry classification and unpublished corrupt transfers. Document the ownership audit without claiming that native algorithms are equivalent. Related to #14566 and pnpm/rfcs#34.
…es (#14586) Exercise the multi-ecosystem architecture through a usable Python integration, not test-only metadata writers. Keep Python requirement, marker, lockfile and environment semantics separate from npm and Cargo. Reuse the install-wide HTTP/auth budget, verified artifact ingestion, CAS and store index. Extract a shared install lifecycle without ecosystem-specific branches. Native tasks declare metadata footprints and return prepared projections. Settle all work before publishing Cargo or Python state, and reverse attempted publications before restoring metadata on failure. Retain resources when rollback fails so recovery remains possible. Enroll npm with its existing in-place materialization semantics, and keep npm-only dispatch on its early path. Do not unify native resolvers, lockfile formats, target identity or package layouts. Consolidate archive ingestion below the ecosystem boundary. Share cache validation, authenticated requests, extraction retries and publication across tarballs and ZIPs, retaining tar streaming and ZIP decoding. Test the shared contracts across both formats and preserve npm fast paths. Use standard pylock.toml, independently validated with uv. New features target pnpm v12 only. Shared archive URL-redaction fixes also cover pnpm v11. Related to #14566 and pnpm/rfcs#34.
Summary
Adds
text/0000-cargo-ecosystem.md, an RFC that makes pnpm a package manager for Rust crates and pnpr a crate registry.Why: since the CLI rewrite in Rust, pnpm no longer dogfoods itself. The Rust workspace's 714 crates.io dependencies (811 lockfile entries) are fetched by cargo, cached by
Swatinem/rust-cache, audited by cargo-deny and bumped by Dependabot. None of it touches pnpm or pnpr.What the RFC proposes:
.pnpm/crates/<source>/as cargodirectorysources, wired up through a pnpm-managed region of a committed.cargo/config.toml.Cargo.lockstays the one lockfile for crates, written in cargo's canonical form and accepted bycargo --locked.cargo.enabledinpnpm-workspace.yaml; cargo's[registries]andcredentials.tomlare honoured rather than duplicated.crate:specifier protocol acrossadd/remove/update/outdated/list/why/audit/licenses/publish/pack, with[workspace.dependencies]as the crate catalog.protocol: cargoon registries under the existing mounts model: sparse index, download, publish/yank/owners/search web API, bare-token auth, and an upstream mode mirroring crates.io.Decided: new ecosystem features ship in the Rust pnpm v12 CLI, following the repository's v12-only feature-development policy.
The architecture section records the real Python integration in pnpm/pnpm#14586: opt-in install/add/run/exec support, PEP 440/508 resolution, native wheels, shared HTTP/auth/CAS, and staged environment publication. Python retains standard
pylock.toml; uv independently installed a generated real-PyPI lockfile. The integration now shares a dedicated lifecycle crate across install and mixed add: native metadata ownership, concurrent preparation, settlement, publication and reverse rollback. Cargo and Python return prepared projections; npm retains explicit in-place semantics. Target identity remains native, and the boundary is internal rather than a public plugin API. Source builds, universal locking and crash recovery remain outside this slice.A duplication audit also consolidated tarball and ZIP cache validation, offline gating, HTTP request setup, extraction retries, progress and store publication. npm packages, Cargo crates, Python wheels and runtime binaries use these same implementations. Container decoding and native dependency semantics remain separate, with cross-format contract tests guarding the common behavior.
Open questions (naming,
--optional, whether to split the pnpr half intopnpr/text/) are listed at the end of the document.🤖 Generated with Claude Code, then continued with Codex.
https://claude.ai/code/session_0167jVFmSFDDzaTKAvCEbC1Y
Updated by an agent (Codex, GPT-5).