vfs: integrate with CJS and ESM module loaders by mcollina · Pull Request #63653 · nodejs/node

mcollina · 2026-05-30T05:41:48Z

Makes require() and import resolve files served by node:vfs. Before this PR, mounted VFS files were only visible through fs.*; the loaders went straight to the real filesystem.

Design

vfs.mount() takes no arguments and returns the reserved absolute mount point of the instance: ${os.devNull}/vfs/<layerId> (for example /dev/null/vfs/0). os.devNull is a character device on POSIX and a device-namespace path on Windows; neither can have child filesystem entries, so no real path can ever exist under this root.

Everything about ownership is decidable from the path alone:

Dispatch: one path normalization, one prefix comparison against the namespace root, one map lookup on the layer id. O(1) in the number of mounted layers — the benchmark (benchmark/vfs/bench-fs-dispatch.js) reports flat per-call latency across 1..10 mounted layers.
Real-fs paths short-circuit with a single prefix check.
Unmount purges loader caches by prefix-scanning the mount point — no per-VFS ownership tracking is needed, and no other-VFS or real-fs entries are ever touched.
Symlink correctness: realpathSync of a VFS entry always resolves to another path under the same mount point, so cache entries hidden behind symlinks are captured by the prefix scan.
Dynamic-import identity: resolved URLs are plain file: URLs under the mount point; import(import.meta.resolve(x)) re-hits the same module job.

Two instances mounting simultaneously never collide (each gets its own layer-<id> segment), so there is no overlap validation and no ordering hazard between mounts.

Loader integration

Toggleable wrappers in the loaders. Null fast-path when no VFS is mounted; otherwise the VFS answers stat / readFile / realpath / legacyMainResolve / getFormatOfExtensionlessFile and the four package.json C++-binding calls.

Module identity follows the path: __filename, module.filename, and import.meta.url are the plain absolute path (or file: URL) of the module under the mount point — no synthetic decorations.

Review guide

Refs

The reserved-namespace design follows the "no interference with valid paths in the file system" requirement from the SEA VFS requirements doc.

Out of scope

SEA + VFS, overlay/stacking of multiple VFS layers under one prefix, migrating the C++ package_configs_ cache, broader permission-model integration.

nodejs-github-bot · 2026-05-30T05:41:54Z

Review requested:

@nodejs/loaders

codecov · 2026-05-30T06:50:02Z

Codecov Report

❌ Patch coverage is 91.03079% with 67 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.10%. Comparing base (0e2126d) to head (b993d9e).
⚠️ Report is 220 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/vfs/setup.js	86.07%	50 Missing ⚠️
src/node_modules.cc	77.96%	4 Missing and 9 partials ⚠️
lib/internal/vfs/file_system.js	96.07%	2 Missing ⚠️
lib/internal/vfs/router.js	95.45%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #63653      +/-   ##
==========================================
- Coverage   92.04%   90.10%   -1.95%     
==========================================
  Files         381      741     +360     
  Lines      169208   242313   +73105     
  Branches    25926    45732   +19806     
==========================================
+ Hits       155746   218325   +62579     
- Misses      13174    15467    +2293     
- Partials      288     8521    +8233

Files with missing lines	Coverage Δ
lib/fs.js	`98.36% <100.00%> (+4.13%)`	⬆️
lib/internal/modules/cjs/loader.js	`98.15% <100.00%> (+18.59%)`	⬆️
lib/internal/modules/esm/get_format.js	`95.20% <100.00%> (+20.89%)`	⬆️
lib/internal/modules/esm/load.js	`91.40% <100.00%> (+8.14%)`	⬆️
lib/internal/modules/esm/resolve.js	`99.03% <100.00%> (+12.10%)`	⬆️
lib/internal/modules/helpers.js	`98.75% <100.00%> (+7.96%)`	⬆️
lib/internal/modules/package_json_reader.js	`99.47% <100.00%> (+11.86%)`	⬆️
src/node_modules.h	`100.00% <ø> (ø)`
lib/internal/vfs/file_system.js	`99.55% <96.07%> (+0.34%)`	⬆️
lib/internal/vfs/router.js	`97.01% <95.45%> (+12.23%)`	⬆️
... and 2 more

... and 526 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

mcollina · 2026-05-30T10:27:16Z

@joyeecheung take a look, should be easier to review.

@jsumners-nr

Restore the "DO NOT depend on the patchability" warnings in esm/load.js and esm/resolve.js that were dropped along with the fs imports. The warning still applies; it now also points at node:vfs as one of the formal hook mechanisms callers should reach for instead. Addresses review feedback from @jsumners-nr in nodejs#63653

nodejs-github-bot · 2026-06-03T08:04:09Z

CI: https://ci.nodejs.org/job/node-test-pull-request/73901/

nodejs-github-bot · 2026-06-03T11:52:41Z

CI: https://ci.nodejs.org/job/node-test-pull-request/73902/

joyeecheung

A design question recently occurred to me: have we explored the versioning of the mounting?

mcollina · 2026-06-03T12:40:18Z

A design question recently occurred to me: have we explored the versioning of the mounting?

what do you mean? You mean multiple vfs layers on top of each other?

joyeecheung · 2026-06-03T13:02:48Z

what do you mean? You mean multiple vfs layers on top of each other?

For the stacks to have some kind of version number/ID to identify the current status?

BTW I just noticed that there's no mention of unmount() and mount() in the VFS docs..

mcollina · 2026-06-03T20:10:47Z

BTW I just noticed that there's no mention of unmount() and mount() in the VFS docs..

I did purge them when doing the splitting; I forgot to bring them back. I'll add them to this PR.

mcollina · 2026-06-03T20:11:45Z

For the stacks to have some kind of version number/ID to identify the current status?

No but we totally should.

Each VirtualFileSystem now exposes a per-process monotonically increasing `layerId`, assigned at construction. The id is stable across mount/unmount cycles for the lifetime of the instance and surfaces in: - debug() output for register / deregister so the layer stack is visible when NODE_DEBUG=vfs is enabled; - the overlap ERR_INVALID_STATE message, which now names the layer ids of the conflicting mounts. The id is the building block for tagging cache entries with the owning VFS, which a follow-up will use to replace the global loader-cache flush in deregisterVFS with a scoped purge. Refs: nodejs#63653 Signed-off-by: Matteo Collina <hello@matteocollina.com>

Replace the global loader-cache flush in deregisterVFS with a scope-purge that only drops entries owned by the unmounting VFS. Per-layer ownership is determined two ways: - For CJS-style filename-keyed caches (Module._cache, Module._pathCache, the CJS stat cache, the helpers.js realpath cache, and the package.json caches) entries are filtered with `vfs.shouldHandle(filename)`. __filename stays a clean absolute path so user code that does `path.dirname(__filename)` or similar is unaffected. - For the ESM cascaded loader's loadCache, entries are tagged at resolve time: when finalizeResolution() detects the resolved path is VFS-owned (via the new loaderGetLayerForPath hook), it appends `?vfs-layer=<id>` to the URL. The tag surfaces in `import.meta.url`, matching the cache-busting pattern used by HMR tooling. On deregister, entries whose URL carries the tag for the unmounting layer are deleted. Multi-mount setups no longer pay the cross-VFS cache-warmup penalty when a single VFS unmounts, and ESM modules loaded from a VFS become reachable for purge instead of leaking forever in the cascaded loader. New helpers exposed for VFS: - cjs/loader.js: clearStatCacheForVFS - helpers.js: purgeRealpathCacheForVFS, loaderGetLayerForPath - package_json_reader.js: purgePackageJSONCacheForVFS Adds test-vfs-scoped-cache-purge covering both the multi-mount isolation and the import.meta.url tag visibility. Refs: nodejs#63653 Signed-off-by: Matteo Collina <hello@matteocollina.com>

mcollina · 2026-06-05T06:47:09Z

@joyeecheung PTAL

Signed-off-by: Matteo Collina <hello@matteocollina.com>

This reverts commit a95717a.

Signed-off-by: Matteo Collina <hello@matteocollina.com>

mcollina · 2026-07-13T16:59:36Z

@joyeecheung done

joyeecheung

Can you remove the (AI generated?) comments that just restate what happens in code with prose or stating how other code calls it currently (which can get out of sync very quickly and is anti-encapsulation?)

Also I think the documentation still doesn't clarify a core question: how is the module lookup algorithm altered once an VFS is mounted. It needs to specify that precisely in the module loading spec in modules.md instead of just leaving some hand wavy descriptions like "the module loader will consult it, it will work" (but how and where in the algorithm? and when will they NOT be consulted?)

joyeecheung · 2026-07-18T12:07:22Z

+// Toggleable loader hooks for VFS support. Each `loader*` wrapper below
+// funnels through a single factory: if an override is registered for the
+// original function it is called first; anything but `undefined` short-
+// circuits the wrapper, otherwise the wrapper falls through to the
+// pinned native. This keeps the wrapper surface a one-liner per binding
+// - adding a new binding is one `wrapLoaderMethod` call, no extra state
+// variables or per-hook setter branches to keep in sync.


Suggested change

// Toggleable loader hooks for VFS support. Each `loader*` wrapper below

// funnels through a single factory: if an override is registered for the

// original function it is called first; anything but `undefined` short-

// circuits the wrapper, otherwise the wrapper falls through to the

// pinned native. This keeps the wrapper surface a one-liner per binding

// - adding a new binding is one `wrapLoaderMethod` call, no extra state

// variables or per-hook setter branches to keep in sync.

// Toggleable loader hooks for VFS support.

This is just describing what obviously happens below with 7 lines of prose?

joyeecheung · 2026-07-18T12:09:43Z

+// internal/vfs/setup.js) map directly into this array; indexes 0..6
+// use packageConfig.main as the prefix, 7..9 use pkgPath directly.


Suggested change

// internal/vfs/setup.js) map directly into this array; indexes 0..6

// use packageConfig.main as the prefix, 7..9 use pkgPath directly.

// internal/vfs/setup.js) map directly into this array.

This is repeating what comments below says.

joyeecheung · 2026-07-18T12:22:49Z

+participate in module resolution and loading. Both
+`require()` / `require.resolve()` (CommonJS) and `import` /
+`import.meta.resolve()` (ECMAScript modules) consult the VFS through
+the same toggleable hooks that `node:fs` uses, so files served from
+the VFS are first-class modules: `package.json` is honoured,
+extensionless files are sniffed for Wasm vs. JavaScript, conditional
+`exports` / `imports` work, and so on.


Suggested change

participate in module resolution and loading. Both

`require()` / `require.resolve()` (CommonJS) and `import` /

`import.meta.resolve()` (ECMAScript modules) consult the VFS through

the same toggleable hooks that `node:fs` uses, so files served from

the VFS are first-class modules: `package.json` is honoured,

extensionless files are sniffed for Wasm vs. JavaScript, conditional

`exports` / `imports` work, and so on.

participate in module resolution and loading. Both

`require()` / `require.resolve()` (CommonJS) and `import` /

`import.meta.resolve()` (ECMAScript modules) consult the VFS through

the same toggleable hooks that `node:fs` uses, so files served from

the VFS are first-class modules: `package.json` is honoured,

extensionless files are sniffed for Wasm vs. JavaScript, conditional

`exports` / `imports` work, and so on.

This paragraph is listing example without actually clarifying anything - exactly how do they appear in the module lookup algorithm, and with what precedence?

Signed-off-by: Matteo Collina <hello@matteocollina.com>

Co-authored-by: Joyee Cheung <joyeec9h3@gmail.com>

The microbenchmark drove 1000+ iterations to reach the optimizing tier, which is not representative of real workloads. Module-loading against a mounted VFS is the meaningful signal and can be measured with the existing benchmark harness.

Measures loading a large CJS or ESM module graph from a mounted VFS, relying on the unmount cache purge so every iteration is a cold load.

mcollina · 2026-07-24T10:15:38Z

@joyeecheung PTAL, I think I did all the fix you suggested.

nodejs-github-bot · 2026-07-24T14:04:06Z

CI: https://ci.nodejs.org/job/node-test-pull-request/75096/

nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels May 30, 2026

mcollina requested review from ShogunPanda, jasnell and joyeecheung May 30, 2026 10:26

ShogunPanda approved these changes May 30, 2026

View reviewed changes

jsumners-nr reviewed Jun 1, 2026

View reviewed changes

Comment thread lib/internal/modules/esm/load.js

Comment thread lib/internal/modules/esm/resolve.js

mcollina force-pushed the vfs-module-loader-integration branch from 6321e08 to 51b033a Compare June 1, 2026 15:54

ShogunPanda approved these changes Jun 2, 2026

View reviewed changes

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Jun 3, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 3, 2026

joyeecheung reviewed Jun 3, 2026

View reviewed changes

Comment thread lib/internal/vfs/setup.js Outdated

mcollina force-pushed the vfs-module-loader-integration branch from 51b033a to 294a19c Compare June 4, 2026 07:23

coderabbitai Bot mentioned this pull request Jun 12, 2026

build: migrate from Workers to Docker jsdelivr/globalping-dash#204

Merged

arcanis mentioned this pull request Jun 15, 2026

module: add --experimental-strip-private-modules #63869

Open

ShogunPanda approved these changes Jun 15, 2026

View reviewed changes

mcollina force-pushed the vfs-module-loader-integration branch from 75bb5c7 to 99a5a5c Compare June 15, 2026 09:00

mcollina added 13 commits July 13, 2026 11:36

vfs: default all read errors in getFormatOfExtensionlessFile to JS

2462d17

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: return normalized path from findVFS

902254d

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: inline isUnderMountPoint and use shouldHandleNormalized

2716996

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs,esm: share legacyMainResolveExtensions arrays

6fafe07

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: add cleanForVfsPrefix helper for cache purges

6e3f1b9

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: use indexed for loops in cleanForVfsPrefix

a95717a

Signed-off-by: Matteo Collina <hello@matteocollina.com>

Revert "vfs: use indexed for loops in cleanForVfsPrefix"

ffd68a2

This reverts commit a95717a.

vfs: fold loader wrappers into wrapLoaderMethod factory

7b6e3de

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: fix lint on wrapLoaderMethod curly braces

2e463ca

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: throw ERR_INVALID_PACKAGE_CONFIG on malformed ancestor pjson

5b71a94

Signed-off-by: Matteo Collina <hello@matteocollina.com>

src: split GetPackageJSON and expose parsePackageJSON binding

30d5115

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: use native parsePackageJSON binding, drop serializePackageJSON

337c130

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: read pjson as Buffer, skip UTF-8 decode

0d5fb61

Signed-off-by: Matteo Collina <hello@matteocollina.com>

mcollina force-pushed the vfs-module-loader-integration branch from d1d72c3 to 0d5fb61 Compare July 13, 2026 13:59

trivikr requested a review from joyeecheung July 15, 2026 17:27

joyeecheung reviewed Jul 18, 2026

View reviewed changes

mcollina added 2 commits July 18, 2026 15:10

vfs: strip verbose comments across the PR

0a98b98

Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: unexpose layerId property

4c417d0

Signed-off-by: Matteo Collina <hello@matteocollina.com>

soda0289 mentioned this pull request Jul 18, 2026

ESM imports escape the sandbox & runfiles aspect-build/rules_js#362

Open

mcollina and others added 5 commits July 24, 2026 02:34

Update doc/api/vfs.md

6d8b5f5

Co-authored-by: Joyee Cheung <joyeec9h3@gmail.com>

benchmark: remove vfs fs-dispatch microbenchmark

bdf79b2

The microbenchmark drove 1000+ iterations to reach the optimizing tier, which is not representative of real workloads. Module-loading against a mounted VFS is the meaningful signal and can be measured with the existing benchmark harness.

benchmark: add vfs module-graph cold-load benchmark

018a6da

Measures loading a large CJS or ESM module graph from a mounted VFS, relying on the unmount cache purge so every iteration is a cold load.

doc: remove unused os.devNull link definition

3b4d428

doc: specify vfs precedence in module resolution

b993d9e

ShogunPanda approved these changes Jul 24, 2026

View reviewed changes

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 24, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 24, 2026

#	File	Role
1	`lib/internal/vfs/router.js`	Reserved-namespace helpers: `getVfsRoot`, `getLayerRoot`, `getLayerIdFromPath`.
2	`lib/internal/vfs/file_system.js`	`mount()` returns the layer's reserved mount point.
3	`lib/internal/vfs/setup.js`	Heart of the PR. `findVFS` (O(1) lookup), fs handler, loader overrides with parity to `src/node_modules.cc` / `src/node_file.cc`, prefix-scan cache purge.
4	`lib/internal/modules/helpers.js`	Hook surface: `loader*` wrappers, `setLoaderFsOverrides` / `setLoaderPackageOverrides`, `purgeRealpathCacheForPrefix`.
5	`lib/internal/modules/cjs/loader.js`	`stat()` + TS read routed through wrappers; `purgeModuleCachesForPrefix` for unmount.
6	`lib/internal/modules/esm/resolve.js`	`legacyMainResolve` + `internalModuleStat` + `toRealPath` routed. No URL decoration.
7	`lib/internal/modules/esm/load.js`	`getSourceSync` reads via the wrapper.
8	`lib/internal/modules/esm/get_format.js`	Extensionless format detection routed.
9	`lib/internal/modules/package_json_reader.js`	4 C++ binding calls routed; `purgePackageJSONCacheForPrefix`.
10	`lib/fs.js`	`statSync` / `lstatSync` honour `throwIfNoEntry:false` on ENOENT from the VFS handler.
11	`doc/api/vfs.md`	Mount semantics + Module loader integration section.

		// internal/vfs/setup.js) map directly into this array; indexes 0..6
		// use packageConfig.main as the prefix, 7..9 use pkgPath directly.

Uh oh!

Uh oh!

Conversation

mcollina commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Design

Loader integration

Review guide

Refs

Out of scope

Uh oh!

nodejs-github-bot commented May 30, 2026

Uh oh!

codecov Bot commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mcollina commented May 30, 2026

Uh oh!

Uh oh!

Uh oh!

nodejs-github-bot commented Jun 3, 2026

Uh oh!

nodejs-github-bot commented Jun 3, 2026

Uh oh!

joyeecheung left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mcollina commented Jun 3, 2026

Uh oh!

joyeecheung commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcollina commented Jun 3, 2026

Uh oh!

mcollina commented Jun 3, 2026

Uh oh!

mcollina commented Jun 5, 2026

Uh oh!

mcollina commented Jul 13, 2026

Uh oh!

joyeecheung left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

joyeecheung Jul 18, 2026

Choose a reason for hiding this comment

Uh oh!

joyeecheung Jul 18, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

joyeecheung Jul 18, 2026

Choose a reason for hiding this comment

Uh oh!

mcollina commented Jul 24, 2026

Uh oh!

nodejs-github-bot commented Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

mcollina commented May 30, 2026 •

edited

Loading

codecov Bot commented May 30, 2026 •

edited

Loading

joyeecheung commented Jun 3, 2026 •

edited

Loading