Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## Unreleased

### Fixed

- Bound mock-SDK capture and profile child lifetimes, output, and process sampling; clean owned POSIX descendants through stdio close and keep timeout/cancellation outcomes unsuccessful. Flush complete capture JSON before exiting despite retained plugin timers. Thanks @SebTardif.
- Profile the default import-loop capture runner directly so its timeout also owns plugin execution. Validate fresh, bounded capture artifacts; RSS/CPU and wall-time measurements now exclude the intermediate CLI wrapper and are not directly comparable with historical profiles.
- Bound OpenClaw npm metadata and tarball downloads with a deadline through response-body reads, reject oversized responses, and release failed downloads. Resolve `latest` and `beta` through the small npm dist-tags endpoint before fetching exact-version metadata, keeping the 16 MiB metadata limit usable.

- Capture plugins that bind `api.runtime.modelAuth` during registration with credential-free defaults; auth acquisition remains an explicit synthetic failure.
Expand Down
43 changes: 43 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,49 @@ Capture one entrypoint directly:
plugin-inspector capture ./dist/index.js --mock-sdk --allow-execute
```

Mock-SDK capture and import-loop/runtime profiles give each child a 30-second
budget. Mock capture reports `capture-timeout`; timed-out profile samples
always have a nonzero `exitCode`, even if a SIGTERM handler exits zero.
Pass an `AbortSignal` as `signal` to cancel owned-child work. Cancellation is
never a successful capture or profile sample.

On POSIX, each child owns a separate process group. Completion waits for
stdout/stderr to close and cleans descendants, including after a successful
leader exit. Shutdown sends SIGTERM, then SIGKILL after a 1-second grace
period. A further 1-second close deadline fails the operation if pipes remain
open. Descendants that deliberately leave the group are not contained;
this is lifecycle supervision, not a sandbox. Windows retains direct-child
termination and the bounded close deadline, not POSIX group cleanup.

The API options `timeoutMs`, `killGraceMs`, and `maxOutputBytes` take precedence
over `PLUGIN_INSPECTOR_CAPTURE_TIMEOUT_MS`, `PLUGIN_INSPECTOR_CAPTURE_KILL_GRACE_MS`,
and `PLUGIN_INSPECTOR_CAPTURE_MAX_OUTPUT_BYTES` for mock capture. Profiles use
the corresponding `PLUGIN_INSPECTOR_PROFILE_*` variables. Values must be finite
positive numbers (zero does not disable limits); invalid values fall through
to the environment, then defaults. Durations/byte limits cannot exceed
2,147,483,647; grace cannot exceed 30,000 ms.

Each profiled stdout/stderr stream retains at most 1 MiB by default while
continuing to drain output. Mock capture retains at most 10 MiB per pipe and
fails if its JSON response is truncated; intercepted plugin stdout/stderr
inside that response retains at most 1 MiB each. The optional `ps` sampler
also has bounded output, execution, and cleanup.

Default import-loop profiles launch the mock capture runner directly under one
profile budget, for both baseline and plugin samples. Their JSON artifacts
retain capture's 10 MiB default limit (`PLUGIN_INSPECTOR_CAPTURE_MAX_OUTPUT_BYTES`,
or an explicit `maxOutputBytes` override), separately from the profile's
1 MiB stdout/stderr limits. Artifacts are accepted only after a successful
current capture. RSS and CPU now measure the actual runner, and wall time no
longer includes intermediate CLI startup. Historical measurements from the
CLI-wrapper route are not directly comparable. Custom `captureCommand` and
`captureScript` launch contracts are unchanged; custom detached groups are
outside the owned process group.

These limits apply to owned child processes only. The public in-process
`captureEntrypoint` path preserves retained handler identity and does not
claim to cancel synchronous plugin code or retained callbacks.

## CI

`plugin-inspector ci` writes the normal compatibility report plus CI-native
Expand Down
82 changes: 72 additions & 10 deletions src/import-loop-profile.js
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
import { mkdir, writeFile } from "node:fs/promises";
import { constants } from "node:fs";
import { mkdir, open, readFile, rm, writeFile } from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { renderPaddedMarkdownTable, writeJsonMarkdownArtifacts } from "./artifacts.js";
import { resolveFromRoot } from "./path-utils.js";
import { runProfiledProcess } from "./process-profile.js";
import { resolveProcessLimits, runProfiledProcess } from "./process-profile.js";
import { assertRunCount, percentile } from "./stats.js";

const defaultCliPath = fileURLToPath(new URL("./cli.js", import.meta.url));
const defaultRunnerPath = fileURLToPath(new URL("./mock-sdk-capture-runner.js", import.meta.url));

export const defaultImportLoopProfileOptions = {
entrypoint: "test/fixtures/lazy-import-plugin.mjs",
Expand Down Expand Up @@ -255,18 +256,49 @@ async function runCaptureSample(options) {
const outputPath = path.join(outputDir, `${options.sampleName ?? "capture"}-${options.index}.json`);
await mkdir(path.dirname(outputPath), { recursive: true });

const command = buildCaptureCommand({ ...options, outputPath });
const defaultCapture = typeof options.captureCommand !== "function" && !options.captureScript;
const maxOutputBytes = resolveProcessLimits({
...options,
env: { ...process.env, ...options.env, ...options.captureEnv },
}, "CAPTURE").maxOutputBytes;
// Only the built-in route owns these sample files. An early process.exit(0)
// must not turn a previous capture into this run's successful result.
if (defaultCapture) await rm(outputPath, { force: true });
const command = buildCaptureCommand({ ...options, outputPath, maxOutputBytes });
const profile = await runProfiledProcess({
command: command.command,
args: command.args,
cwd: command.cwd ?? options.rootDir,
env: { ...process.env, ...command.env },
env: { ...process.env, ...options.env, ...command.env },
timeoutMs: options.timeoutMs,
maxOutputBytes: options.maxOutputBytes,
killGraceMs: options.killGraceMs,
signal: options.signal,
});
const output = profile.exitCode === 0 ? await readCaptureOutput(outputPath) : null;
let output = null;
if (profile.exitCode === 0 && !profile.timedOut && !profile.cancelled) {
if (defaultCapture) {
try {
output = await readCaptureOutput(outputPath, maxOutputBytes);
} catch (error) {
profile.exitCode = 1;
profile.stderrPreview = `Invalid capture artifact: ${error.message}`;
}
} else {
output = await readCaptureOutput(outputPath);
}
}
if (options.signal?.aborted) {
profile.exitCode = 1;
profile.cancelled = true;
output = null;
}

return {
index: options.index,
exitCode: profile.exitCode,
timedOut: profile.timedOut === true,
cancelled: profile.cancelled === true,
status: output?.status ?? "failed",
capturedCount: output?.captured?.length ?? 0,
openClawLifecycle: output?.openClawLifecycle ?? null,
Expand Down Expand Up @@ -407,15 +439,45 @@ function buildCaptureCommand(options) {
}
return {
command: process.execPath,
args: [defaultCliPath, "capture", options.entrypoint, "--output", options.outputPath],
args: [
"--no-warnings",
"--preserve-symlinks",
defaultRunnerPath,
JSON.stringify({
entrypoint: options.entrypoint,
cwd: options.rootDir,
outputPath: options.outputPath,
maxOutputBytes: options.maxOutputBytes,
}),
],
cwd: options.rootDir,
env: { PLUGIN_INSPECTOR_EXECUTE_ISOLATED: "1", ...options.captureEnv },
};
}

async function readCaptureOutput(outputPath) {
const { readFile } = await import("node:fs/promises");
return JSON.parse(await readFile(outputPath, "utf8"));
async function readCaptureOutput(outputPath, maxOutputBytes) {
if (maxOutputBytes === undefined) return JSON.parse(await readFile(outputPath, "utf8"));
const file = await open(outputPath, constants.O_RDONLY | constants.O_NONBLOCK);
try {
const stat = await file.stat();
if (!stat.isFile()) throw new Error("expected a regular capture file");
if (stat.size > maxOutputBytes) throw new Error("capture result exceeded its byte limit");
const chunks = [];
let bytes = 0;
// end is inclusive: read at most limit + 1 even if the file grew after stat.
for await (const chunk of file.createReadStream({ end: maxOutputBytes, autoClose: false })) {
chunks.push(chunk);
bytes += chunk.length;
}
if (bytes > maxOutputBytes) throw new Error("capture result exceeded its byte limit");
const result = JSON.parse(Buffer.concat(chunks, bytes).toString("utf8"));
if (!result || typeof result.status !== "string" || !Array.isArray(result.captured)) {
throw new Error("expected a capture status and captured contracts");
}
return result;
} finally {
await file.close();
}
}

function markdownTable(rows, headers) {
Expand Down
70 changes: 32 additions & 38 deletions src/inspector.js
Original file line number Diff line number Diff line change
@@ -1,20 +1,18 @@
import { existsSync } from "node:fs";
import { execFile } from "node:child_process";
import { readdir, readFile } from "node:fs/promises";
import * as nodeModule from "node:module";
import path from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import { promisify } from "node:util";
import { createCaptureApi } from "./capture-api.js";
import { captureApiOptionsForPlugin } from "./capture-config.js";
import { fixtureCheckoutPath, fixtureSourceRoot } from "./config.js";
import { buildCompatibilityFixtureReport } from "./fixture-summary.js";
import { readOpenClawTargetSurface } from "./openclaw-target.js";
import { prepareOpenClawTarget, resolveOpenClawTargetVersion } from "./openclaw-version.js";
import { startOwnedProcess } from "./process-profile.js";
import { buildCompatibilityReport, buildReport } from "./report.js";
import { inspectSdkDeprecations } from "./sdk-deprecation-rules.js";

const execFileAsync = promisify(execFile);
const pluginFactoryNames = "defineBundledChannelEntry|defineChannelPluginEntry|createChatChannelPlugin|definePluginEntry";
// Bundlers emit unbound calls as (0, sdk.factory)(...), including inline require receivers.
const compiledFactoryCall = new RegExp(String.raw`\(\s*0\s*,\s*(?:require\s*\(\s*(?:"[^"\r\n]*"|'[^'\r\n]*')\s*\)|[$A-Z_a-z][$\w]*)(?:\s*\.\s*[$A-Z_a-z][$\w]*)*\s*\.\s*(${pluginFactoryNames})\s*\)\s*\(`, "dg");
Expand Down Expand Up @@ -232,51 +230,47 @@ export async function captureEntrypointWithMockSdk(entrypoint, options = {}) {
cwd: options.cwd ?? process.cwd(),
pluginRoot: options.pluginRoot,
apiOptions: options.apiOptions,
maxOutputBytes: options.maxOutputBytes,
};
const { result } = startOwnedProcess({
command: process.execPath,
args: ["--no-warnings", "--preserve-symlinks", runnerPath, JSON.stringify(payload)],
cwd: options.cwd ?? process.cwd(),
env: { ...process.env, ...options.env },
timeoutMs: options.timeoutMs,
killGraceMs: options.killGraceMs,
maxOutputBytes: options.maxOutputBytes,
signal: options.signal,
}, "CAPTURE");
const outcome = await result;
if (outcome.exitCode !== 0 || outcome.outputTruncated) {
const message = outcome.cancelled ? "Mock SDK capture cancelled"
: outcome.outputTruncated ? "Mock SDK capture output exceeded its byte limit"
: "Mock SDK capture child failed";
const { error: childError, ...details } = outcome;
throw classifyMockSdkCaptureError(Object.assign(childError ?? new Error(message), details));
}
try {
const { stdout } = await execFileAsync(
process.execPath,
["--no-warnings", "--preserve-symlinks", runnerPath, JSON.stringify(payload)],
{
cwd: options.cwd ?? process.cwd(),
env: {
...process.env,
...(options.env ?? {}),
},
maxBuffer: 1024 * 1024 * 10,
},
);
return JSON.parse(stdout);
return JSON.parse(outcome.stdout);
} catch (error) {
const captured = parseCaptureResultFromStdout(error?.stdout);
if (captured) {
return captured;
}
throw classifyMockSdkCaptureError(error);
}
}

function parseCaptureResultFromStdout(stdout) {
if (!stdout) {
return null;
export function classifyMockSdkCaptureError(error) {
if (error?.timedOut === true) {
return enrichCaptureError(error, {
message: `Mock SDK capture timed out after ${error.timeoutMs}ms`,
failureClass: "capture-timeout",
});
}
try {
const parsed = JSON.parse(stdout);
if (
parsed &&
typeof parsed === "object" &&
typeof parsed.status === "string" &&
Array.isArray(parsed.captured)
) {
return parsed;
}
} catch {
return null;
if (error?.cancelled || error?.outputTruncated) {
return enrichCaptureError(error, {
message: error.message,
failureClass: "mock-sdk-capture-error",
});
}
return null;
}

export function classifyMockSdkCaptureError(error) {
const rawMessage = [error?.stderr, error?.stdout, error?.message].filter(Boolean).join("\n");
const missingExport = rawMessage.match(/does not provide an export named ['"]([^'"]+)['"]/)?.[1];
if (missingExport) {
Expand Down
46 changes: 34 additions & 12 deletions src/mock-sdk-capture-runner.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,34 @@ import { register } from "node:module";
import os from "node:os";
import path from "node:path";
import { pathToFileURL } from "node:url";
import { writeArtifacts } from "./artifacts.js";
import { createCaptureApi } from "./capture-api.js";
import { captureApiOptionsForPlugin } from "./capture-config.js";
import { createCappedCollector, resolveProcessLimits } from "./process-profile.js";
import { createMockSdkPackage } from "./sdk-mock.js";

const options = JSON.parse(process.argv[2] ?? "{}");
let activeOutputCapture = null;

try {
const result = await run(options);
writeRunnerStdout(`${JSON.stringify(result, null, 2)}\n`);
const json = `${JSON.stringify(result, null, 2)}\n`;
const { maxOutputBytes } = resolveProcessLimits(options, "CAPTURE");
if (Buffer.byteLength(json) > maxOutputBytes) {
throw new Error(`Mock SDK capture result exceeded its ${maxOutputBytes}-byte limit`);
}
if (options.outputPath) {
await writeArtifacts([{ path: options.outputPath, content: json }]);
} else {
await writeRunnerStdout(json);
}
process.exit(0);
} catch (error) {
if (error.failureClass) {
writeRunnerStderr(`[plugin-inspector:${error.failureClass}]\n`);
await writeRunnerStderr(`[plugin-inspector:${error.failureClass}]\n`);
}
writeRunnerStderr(`${error.stack ?? error.message}\n`);
process.exitCode = 1;
await writeRunnerStderr(`${options.outputPath ? error.message : (error.stack ?? error.message)}\n`);
process.exit(1);
}

async function run(options) {
Expand Down Expand Up @@ -124,27 +136,27 @@ function findRegisterExport(module) {
}

function installProcessOutputCapture() {
const stdoutChunks = [];
const stderrChunks = [];
const stdout = createCappedCollector(1024 * 1024);
const stderr = createCappedCollector(1024 * 1024);
const originalStdoutWrite = process.stdout.write.bind(process.stdout);
const originalStderrWrite = process.stderr.write.bind(process.stderr);

process.stdout.write = (chunk, encoding, callback) => {
stdoutChunks.push(Buffer.isBuffer(chunk) ? chunk.toString("utf8") : String(chunk));
stdout.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(String(chunk), typeof encoding === "string" ? encoding : "utf8"));
invokeWriteCallback(encoding, callback);
return true;
};
process.stderr.write = (chunk, encoding, callback) => {
stderrChunks.push(Buffer.isBuffer(chunk) ? chunk.toString("utf8") : String(chunk));
stderr.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(String(chunk), typeof encoding === "string" ? encoding : "utf8"));
invokeWriteCallback(encoding, callback);
return true;
};

return {
originalStdoutWrite,
originalStderrWrite,
stdout: () => stdoutChunks.join(""),
stderr: () => stderrChunks.join(""),
stdout: () => stdout.text(),
stderr: () => stderr.text(),
};
}

Expand All @@ -162,9 +174,19 @@ async function drainAsyncOutput() {
}

function writeRunnerStdout(text) {
(activeOutputCapture?.originalStdoutWrite ?? process.stdout.write.bind(process.stdout))(text);
const write = activeOutputCapture?.originalStdoutWrite ?? process.stdout.write.bind(process.stdout);
return flushWrite(write, text);
}

function writeRunnerStderr(text) {
(activeOutputCapture?.originalStderrWrite ?? process.stderr.write.bind(process.stderr))(text);
const write = activeOutputCapture?.originalStderrWrite ?? process.stderr.write.bind(process.stderr);
return flushWrite(write, text);
}

// The runner exits deliberately to shed plugin timers, but only after the
// complete protocol response has reached its pipe.
function flushWrite(write, text) {
return new Promise((resolve, reject) => {
write(text, (error) => error ? reject(error) : resolve());
});
}
Loading