Skip to content

mcpx: where a script's packages come from -- npm/deno/bun dependency configuration chain #1913

Closed

Description

@dezren39

Relocated from dezren39/darwin-nix#103 on 2026-10-28 (placement audit): mcpx's single source lives in developing-today/code (pkgs/mcpx); re-home tracked at #309. Original body below; issue references inside it are historical (dezren39/nix numbering).

Parent: #175

Cache: deferred to #187 (#181 collision 12). #187 owns the cache mechanism
(layout, keys, byte/age eviction, leases); this issue owns only what it puts in
it — the resolved dependency set as the key, and the prepared directory as the
entry. The cache design text below stays as input to #187.

Summary

Where does a script's third-party code come from, and who decides? Today: the
runtime decides, entirely, and mcpx has no opinion, no setting and no way for a
caller to state one. This issue defines the dependency-source chain — host
default, CLI config, config directory, the working directory's
package.json/node_modules, an explicit package list over /v1, /mcp, the
CLI and the plugin, and a pre-baked set shipped with the binary — and picks a
precedence order for it.

Original notes

  • how to config npm
    • from host default
    • from cli config
    • from cli config dir
    • from pwd dir nodepackages
    • from /v1 /mcp (and cli/plugin)
      • command to pass list of packages, package.json(c) (package.lock supported too)
    • baked into binary

Current state

Nothing in mcpx configures packages. runner.Run writes the client and the
launcher next to the script, sets MCPX_RUNTIME, MCPX_ENTRY, MCPX_CLIENT
and whatever script.env carries, and spawns the runtime with cmd.Dir = opts.Dir (internal/runner/runner.go). Resolution is then whatever the runtime
does from that directory. docs/proposals.md:301 states the position
explicitly — "Scripts run on a real runtime, so imports already work: npm:,
jsr:, https: on Deno, node_modules on bun and node. What is missing is a
declared place to put shared code" — and proposes .config/mcpx/lib/ with a
generated import map. That proposal is still status: proposed.

Two consequences that matter:

  1. Working directory determines dependencies, and the working directory
    differs by surface.
    mcpx run ./x.ts inherits the caller's cwd on purpose
    ("a relative path in a script should mean what it means on the command
    line"). POST /v1/exec takes cwd from ExecOptions — the caller's
    context, which on a remote daemon names a directory that does not exist
    there. So the same script resolves packages differently depending on how it
    was invoked, and nothing says so.
  2. A remote exec sends text, not a path (docs/exec.md), so there is no
    project directory on the daemon side at all. A remote script cannot import
    anything but npm:/jsr:/https: specifiers today.

The one place mcpx does touch npm is unrelated: internal/registry/registry.go:248-255
(Runtimes) maps a registry package type to a launcher (npm → npx -y,
pypi → uvx, oci → docker, nuget → dnx) when adding an MCP server.

How each runtime resolves packages (verified, deno 2.9.6 / bun 1.3.11 / node v24.20.0)

deno bun node
bare specifier import {z} from "zod" needs deno.json imports or a node_modules/package.json resolves node_modules, auto-installs when none exists (verified: fresh dir, no node_modules, bun b.ts fetched zod and ran) node_modules upward walk only; fails otherwise
npm:zod@3 specifier native, no install step (verified) rejected — "Cannot resolve invalid URL 'npm:zod'" rejected
jsr: / https: native jsr: no, HTTP no no
cache location DENO_DIR (default ~/.cache/deno) ~/.bun/install/cache, BUN_INSTALL node_modules on disk; npm cache under ~/.npm
materialise node_modules nodeModulesDir: "auto"|"manual"|"none" in deno.json always always
declared deps deno.json imports, --import-map package.json + bun.lock, bunfig.toml package.json + lockfile
auto-install control n/a (specifier-level) --install=auto|fallback|force, -i none
registry/auth config .npmrc is honoured for npm: bunfig.toml [install], .npmrc .npmrc chain: project → ~/.npmrc → global → builtin, plus NPM_CONFIG_* env

The asymmetry to design around: deno pins in the specifier, bun and node pin
in a file.
npm:zod@3 is self-contained and travels with the script text —
which is exactly what a remote exec needs — while package.json needs a
directory. Any scheme that works on all three has to be able to produce both.

docs/story.md:1672 already states the project's stance on this: "npm versions
are pinned. A configuration that silently upgrades is one that…". Bun's
auto-install resolves to latest unless the specifier carries a range, so
auto-install-by-default contradicts that line and should be opt-in.

Approaches

Approach A — least recommended: a mcpx npm command group that shells out

mcpx npm install <pkg>, mcpx npm config set …, writing into a
mcpx-owned node_modules. The user's note ("command to pass list of packages")
reads most directly as this.

Why not, and this is the brief's point 2 in practice: it is a new command
surface that duplicates a tool the user already has, in a project whose parity
rule is CLI = /v1 = MCP = plugin — so one command becomes four surfaces, an
ops-table entry and an OpenAPI path, all to wrap npm install. It also has to
pick a package manager (npm? bun? pnpm is on this box too) and then owns every
lockfile-format question forever. And it says nothing about deno, where the
answer is not an install at all. A setting that names where dependencies come
from is strictly smaller and reaches all four surfaces automatically.

Approach B — middle: one deps.* setting group over the existing search-path machinery

Add settings in the one registry:

setting kind default meaning
deps.source enum host|config|cwd|explicit|none host which of the chain below is authoritative
paths.deps path list — directories searched for package.json/deno.json/bunfig.toml, nearest first, with the null splice
deps.packages list, repeatable — explicit specifiers, e.g. zod@3, npm:zod@3
deps.manifest source (inline or path) — a package.json/package.jsonc or deno.json given directly
deps.autoInstall enum off|fallback|always off maps to bun --install=, to a materialise step for node, ignored by deno

paths.deps reuses internal/searchpath exactly as paths.scripts,
paths.adapters and paths.placeholders do — same null splice, same
nearest-wins, same Describe() for error messages — so "from cli config dir"
and "from pwd dir nodepackages" are two entries on one list rather than two
features. KindSource already accepts "inline source, a path to a file, or a
directory", which is what deps.manifest needs, and settings.Scope already
distinguishes client from daemon, which is what makes cwd mean the right thing
on each side.

This is most of the answer and it is small. What it leaves open: nothing is
declared per script, so two scripts in one project cannot want different
versions; and the daemon still has to be told what the caller's cwd contained
on a remote run.

Approach C — recommended: Approach B, plus a resolved dependency set computed per run and carried on ExecOptions

The addition is one idea: mcpx resolves the chain once, into a concrete
dependency set — a list of pinned specifiers plus optionally a manifest and a
lockfile — and that resolved set is what travels, gets hashed, and keys the
cache. Everything else is where the set comes from.

The chain, most specific wins:

rank source how it is expressed
1 the script itself a pinned npm:/jsr: specifier in the source, or a PEP-723-style inline block (PEP 723 is #105's; #104 reads a script's own specifiers statically)
2 explicit request --package zod@3 (repeatable), --manifest ./package.json, ExecOptions.deps on /v1/exec and mcpx_exec
3 project nearest package.json/deno.json/node_modules on paths.deps from the working directory
4 mcpx config dir .config/mcpx/package.json and .config/mcpx/lib/ — docs/proposals.md:301's proposal, which this subsumes
5 user/host ~/.npmrc, DENO_DIR, ~/.bun/install/cache — whatever the host already has
6 baked in a set shipped with the binary or the Nix derivation

Why this one:

  • It is the only version that makes remote exec work. A remote run sends
    text; a resolved set is also text. ExecOptions.deps is one more field on
    an object whose stated design is "one flat object that grows, rather than a
    new endpoint per capability" (docs/exec.md), and it lands on /v1, MCP, the
    CLI and the plugin at once because that is how ExecOptions already works. No
    new endpoint is needed, which is the answer to "from /v1 /mcp (and
    cli/plugin)".
  • It gives caching something to key on. The runner already content-addresses
    the exec workdir by the hash of the generated client, precisely so two
    catalogues cannot overwrite each other's node_modules-adjacent state. A
    dependency set hashes the same way: same set, same prepared directory, warm
    runtime cache; different set, different directory, no interference. Without
    this, two projects sharing one exec workdir would fight over node_modules
    the same way they once fought over the client — a mistake this codebase has
    already made once and fixed. (The key is this issue's; where the prepared
    directory lives and how it is evicted and pinned is Phase 4: joiner with missing pre-join control messages stalls silently #187's.)
  • It normalises the deno/bun/node split at one point. From a resolved set
    mcpx can emit whichever form the selected runtime wants: rewrite to npm:
    specifiers or a deno.json imports map for deno; write package.json +
    lockfile and run bun install --frozen-lockfile for bun; materialise
    node_modules for node. The caller states what, mcpx decides how, and the
    comparison table above stops being the user's problem.
  • Lockfiles get honoured rather than reinvented. The user asks for
    package.lock support; the right reading is "if a lockfile is present next to
    the manifest, use it and do not resolve" — bun install --frozen-lockfile,
    npm ci, deno.lock. mcpx never writes one into somebody's project
    uninvited.

"Baked into binary" deserves its own note because it has three real
meanings and they are not equivalent:

  1. Vendored cache — the Nix derivation prefetches a fixed dependency set into
    a store path and mcpx points DENO_DIR/BUN_INSTALL/NODE_PATH at it. Pure,
    offline, reproducible, and the natural fit for this repo: package.nix
    already wraps the binary with PATH additions for exactly this kind of
    reason. Recommended reading of the bullet.
  2. Embedded source — go:embed a small standard library of .ts helpers into
    the binary and write it into the exec workdir. Cheap, no network ever, but
    only viable for first-party code.
  3. Self-contained executable — deno compile a script and its graph into one
    binary. Real, and interesting for shipping a recipe as a command, but it is
    a distribution feature rather than a dependency-configuration one.

Only (1) and (2) belong in this issue; (3) is worth a line in the comparison
doc.

.jsonc is asked for explicitly: deno.json is JSONC already, package.json
is not and npm will not read comments. If mcpx accepts package.jsonc it must
strip comments and write plain JSON before any package manager sees it.

Open questions

  • Precedence: does an explicit --package replace the project manifest or
    merge with it? Merging invites two versions of one package; replacing
    surprises someone in a project directory. A conflict that is reported rather
    than silently resolved is likely right, matching Bump build/buck/preludes/buck2-prelude from 3a02ea3 to 370cd4d #5's rule for config chains.
  • Does mcpx ever run an installer itself, or only ever point the runtime at
    something already prepared? Running one means network, timeouts and a lock on
    a shared directory inside what is meant to be a fast exec path.
  • Where does a prepared dependency directory live — under the content-addressed
    exec workdir (pruned by count by pruneWorkDirs, internal/runner/runner.go:808,
    keeping exec.workDirs: 8, internal/defaults/defaults.json:98) or in its own
    cache with its own retention? node_modules is far larger than a generated
    client and the existing pruning policy was not sized for it. Owned by Phase 4: joiner with missing pre-join control messages stalls silently #187
    (collision 12); kept here as input. Phase 4: joiner with missing pre-join control messages stalls silently #187's first open question — lean on the
    runtimes' own caches and store only the resolved manifest — is the one this
    issue should answer.
  • For the daemon: does ExecOptions.cwd imply the daemon may read a manifest at
    that path when it happens to share the filesystem? That is the same
    shared-filesystem question artifacts answered with an explicit
    capabilities: ["artifacts"] / sharedFs declaration rather than inference —
    the same answer probably applies.
  • Does deps need to reach mcpx api --spec adapters and launcher placeholder
    files, or is it strictly a script-execution concern?

Ease · Impact · Related

Ease L — five sources, three runtimes, and a cache key · Impact high;
without it a script cannot use a library, which is most of what scripts do.

Depends on / related: #187 (owns the cache mechanism, #181 collision 12),
#5 (paths.config chain and its two-pass rule, the model for precedence here),
#40 (closed, fixed in #57 — ExecOptions, which deps extends), #7 (launcher
placeholders, same settings machinery), #78 (daemon state/cache correctness —
a prepared dependency directory is new cached state), and siblings #100
(runtime selection), #101 (comparison doc), #104 (automatic imports; fills in
the set), #105 (Python parity; PEP 723) and #107 (flake environments; a
Nix-vendored set). docs/proposals.md "Modules and libraries" (:295) is
subsumed by this.

Security/auth deferred; see tracker.

Activity

  1. developing-today commented on Oct 28, 2026

    @developing-today
    Contributor
  2. dezren39 commented on Oct 28, 2026

    @dezren39
    MemberAuthor

    History carried over from dezren39/darwin-nix#103 (verbatim, oldest first):


    @dezren39 · 2026-09-30T07:21:40Z

    Integration pass. Placed in the DAG rooted at #181.


    @dezren39 · 2026-09-30T07:28:23Z

    Integration pass -- cross-theme collision. Surfaced only when the themes were put side by side; the authors of these issues could not see each other's work. Recorded at the root in #181.

    Three new content-addressed caches, one cache design needed

    #103, #104 and #107 each introduce a content-addressed cache, sized for node_modules trees and Nix closures -- orders of magnitude larger than what pruneWorkDirs/defaults.ExecWorkDirs was sized for. Without one design across the three, #78 gets a fourth instance of the same problem.

    Also on: #104 #107 #78. Nobody currently owns this.


    @dezren39 · 2026-09-30T11:05:58Z

    Collision ownership (issues pass, 2026-09-30, at 05c78b2). #181 listed these cross-theme collisions with no owner. Each now has one:

    #181 row Collision Owner What the owner decides
    12 three new content-addressed caches (#103, #104, #107) #187 (new) One cache mechanism (layout, keys, byte/age eviction, leases, nix GC roots) under #175; the three issues own only what they put in it.

    Where this issue is the owner, the decision is made here; where it is not, it adopts the owner's decision rather than making its own.


    @dezren39 · 2026-09-30T12:02:41Z

    Issues pass (2026-09-30, at 05c78b2).

  3. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Oct 29, 2026
  4. dezren39 commented on Oct 29, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up — triage started

  5. dezren39 commented on Oct 29, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done — triage verdict: scope CONFIRMED, rescoped to an implementable slice 1; implementation routed as queue task 3191

  6. added
    queue/queuedHas a queued task in the Supabase agent queue
    and removed on Oct 29, 2026
  7. 398 remaining items

  8. dezren39 commented on Nov 1, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  9. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    agent-note:v1 — picked up
    Agent: repo-worker-code (OpenChamber scheduled lane)
    Task: repo-health-code-pilot task 3191 (small_fix)
    At: 2026-11-02T12:46:00Z
    State: claimed; fresh verification complete
    Found via: atomic claim from repo-health-code-pilot; resumed from the saved PR checkpoint.
    Fresh state: #1913 OPEN and assigned to dezren39; canonical PR #2589 OPEN, authored and assigned to dezren39.
    Next: park at the Drew-owned hard stop; do not inspect or touch PR internals, reviews, checks, branch, or saved worktree.
    Links: #1913 · #2589 · task 3191

  10. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    agent-note:v1 — parked
    Agent: repo-worker-code (OpenChamber scheduled lane)
    Task: repo-health-code-pilot task 3191
    At: 2026-11-02T12:46:20Z
    State: parked
    Fresh evidence: #1913 OPEN and assigned to dezren39; canonical PR #2589 OPEN and authored/assigned to dezren39.
    Disposition: parked for 30 minutes at the Drew-owned hard stop. No PR internals, reviews, checks, branch, or saved worktree touched. work/assigned remains set.
    Resume: explicit Drew release/handoff or PR disposition; revalidate before any further work.
    Links: #1913 · #2589 · task 3191

  11. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    event: picked_up
    task: 3191 (repo-health-code-pilot, repo-worker-code; OpenChamber scheduled lane)
    verified: Fresh metadata-only check: #1913 OPEN and assigned to dezren39; canonical PR #2589 OPEN, non-draft, authored and assigned to dezren39. Existing checkpoint forbids inspecting/touching PR internals, reviews, checks, branch, or saved worktree absent explicit handoff/release.
    next: park 30 minutes; resume after explicit Drew release/handoff or PR disposition, then revalidate.

  12. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    event: parked
    task: 3191 (repo-health-code-pilot, OpenChamber scheduled lane)
    state: PR #2589 remains OPEN and Drew-owned; issue #1913 remains OPEN and Drew-assigned. Saved checkpoint prohibits any PR/worktree inspection or changes absent explicit handoff/release. No work performed this claim.
    resume: after explicit Drew handoff/release or PR disposition; revalidate issue and PR fresh.

  13. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    agent-note:v1 — holder: repo-worker-code (OpenChamber scheduled lane), at 2026-11-02T13:50:12Z. event: picked-up; task: 3191; queue: repo-health-code-pilot.

  14. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    agent-note:v1 — holder: repo-worker-code (OpenChamber scheduled lane), at 2026-11-02T13:50:19Z. event: parked; task: 3191; queue: repo-health-code-pilot. Canonical PR #2589 remains open and Drew-owned; hard stop. Resume after explicit handoff/release or PR disposition, then revalidate fresh. PR internals, checks, reviews, branch, worktree and source were not inspected or touched.

  15. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up + parked (queue task 3191, repo-worker-code, OpenChamber scheduled lane)

  16. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    agent-note:v1 — holder: repo-worker-code (OpenChamber scheduled lane), at 2026-11-02T14:50:02Z. event: picked-up; task: 3191; queue: repo-health-code-pilot. Fresh metadata-only revalidation: issue #1913 OPEN, assigned dezren39; canonical PR #2589 OPEN, non-draft, authored and assigned to dezren39. Existing checkpoint names this as Drew-owned hard stop; no PR checks, diff, reviews, branch, worktree or source inspected or touched.

  17. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    agent-note:v1 — holder: repo-worker-code (OpenChamber scheduled lane), at 2026-11-02T14:50:07Z. event: parked; task: 3191; queue: repo-health-code-pilot. Parked on the Drew-owned canonical PR #2589 hard stop. Resume only after explicit handoff/ownership release or PR disposition; no work touched.

  18. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    agent-note:v1 — task 3191 picked up by repo-worker-code (lane D) 2026-11-02 ~15:27Z. Revalidation: PR #2589 OPEN, non-draft, head branch queue/3191-1913-deps-config-chain (this tasks own fleet branch), no hard-stop labels on issue #1913 — resuming fleet work per the head-branch test, not a Drew-owned stop. Fresh state: PR #2589 is CONFLICTING with current main; resolving via main merge on the laptop worktree, then Codex/thread disposition and CI.

  19. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    agent-note:v1 — task 3191 (lane D) progress 2026-11-02 ~15:40Z: resumed the uncommitted inherited-env workdir-key test in the laptop worktree, reproduced its failure (workdir key ignores inherited NODE_ENV), fixed installEnvIdentity to also scan the inherited process environment for NODE_ENV / NPM_CONFIG_* (commit a312fb4). Merged origin/main (82eede0): one conflict in settings/effect.go — EffectExemptionPin resolved to 192 (merged map has 192 entries: main 191 + branch deps.autoInstall). Laptop suites pass on the merged tree: runner, deps, cli, execsvc, api. The 5 settings-suite failures (model.* effect coverage, undeclared env reads, environment doc, compendium chapters) reproduce identically on clean main — pre-existing, not from this PR. Pushed head 8d1d9d6 to PR #2589; no CI checks registered after ~5 min (runner backlog). Task 3191 parked 20m on CI; resume: checks green -> disposition the 7 open CodeRabbit threads -> merge -> close issue #1913.

  20. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    Picked up: OpenChamber scheduled lane claimed task 3191 via atomic queue claim (resume, attempt 4).

  21. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    Parked: OpenChamber scheduled lane parked task 3191 for 30 minutes — Drew-owned PR #2589 still open.

  22. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done — slice 1 shipped, PR merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions