AutoVul

AutoVul is a host-independent vulnerability-research extension and deterministic analysis engine.

It runs under mature Agent/Harness hosts such as Pi Agent. The host understands the request, reads code, forms a vulnerability hypothesis and decides what to do next. AutoVul validates structured research inputs, executes analyzers, persists evidence and verifies results.

The host Agent owns understanding, reasoning and orchestration. AutoVul owns vulnerability-research capabilities, deterministic execution, evidence and verification.

AutoVul does not implement a general Agent, model provider, Agent Loop, conversation memory, planning system or subagent framework.

Current capability

The current TypeScript V2 implementation provides a verified CodeQL query workflow:

• structured Source/Sink-oriented query intent;
• Language Packs for Python, JavaScript/TypeScript, Java/Kotlin and C/C++;
• Source and Sink probes;
• advisory CodeQL Language Server draft diagnostics;
• authoritative CodeQL CLI compile and analyze execution;
• vulnerable/fixed differential verification;
• persisted run state, bounded candidate budgets and crash recovery;
• relocatable Query Packs with model-free replay;
• thin Pi and CLI integrations sharing the same Application API.

CodeQL databases must already exist. AutoVul currently inspects and validates them but does not create databases or execute target-project build/install scripts.

The accepted result levels are:

• generated: a candidate exists but has not passed execution verification;
• compiled: the analyzer accepted the rule;
• reproduced: the vulnerable target matched the expected behavior or location;
• differential: the vulnerable target matched and the fixed target satisfied the non-match policy;
• variant_validated: additional positive, negative or cross-project validation passed.

Probe results, model reasoning, mocks and diagnostic wrappers are not vulnerability confirmation.

Research capability direction

AutoVul is designed as a set of parallel vulnerability-research capabilities over one deterministic runtime. Flow is the first capability being completed; it is not a universal vulnerability IR.

Host Agent / Harness
  -> capability-specific hypothesis
       Flow | MissingCheck | Typestate | ...
  -> AutoVul
       capability validation and decision policy
       shared run / budget / evidence / replay runtime
       analyzer adapters

Each capability owns its hypothesis shape, observations and success predicate. Capabilities share operation state, budgets, cancellation, artifacts, evidence and replay. The shared layer does not force authorization, lifecycle or state-machine problems into Source/Sink semantics.

The planned Flow Model describes Source, Sink, explicit propagation steps and barriers. Target references, analyzer selection and budgets belong to the execution request. CodeQL becomes an optional execution backend rather than part of the Flow Model.

The aggregate capability API (autovul_research and autovul_run) is implemented alongside the existing CodeQL compatibility surface. Flow v1 is verified across the accepted 20-case Python, JavaScript, Java and C/C++ real-CodeQL matrix, including vulnerable/fixed differential analysis and relocated model-free replay. The narrowly frozen MissingCheck v1 is verified for its declared JavaScript single-file CFG boundary. Typestate, Delta and Variant are not implemented support claims.

See:

• Research capability architecture
• Flow capability design
• Flow capability implementation plan
• Flow capability v1 change SPEC
• MissingCheck capability v1 SPEC
• Typestate candidate capability SPEC
• Delta role-classification SPEC
• Variant role-classification SPEC
• Product specification
• Project charter
• SPEC workflow

Architecture

Host Agent / Harness
  -> Integration Adapter
       -> Application API
            -> Core domain policy and workflow
                 -> Analyzer Ports
                      -> CodeQL Runner

Production dependencies follow:

contracts <- core <- codeql-runner <- pi-extension / cli

The packages are:

• @autovul/contracts: versioned TypeBox schemas and stable protocols;
• @autovul/core: deterministic policy, workflow state, budgets and verification decisions;
• @autovul/codeql-runner: CodeQL CLI/LSP, filesystem, process and SARIF adapters;
• @autovul/pi-extension: thin Pi registration, commands, lifecycle and presentation;
• @autovul/cli: deterministic debugging, CI and model-free replay interface.

Development

Requirements:

• Node.js and npm;
• CodeQL CLI for real analyzer tests and normal CodeQL execution;
• prebuilt vulnerable/fixed CodeQL databases for differential research.

Install and run the standard checks:

npm install
npm run build
npm run typecheck
npm test
npm run lint
npm run pack:check

npm run check runs the main build, architecture, specification, unit/integration and package gates. Real CodeQL, real-model and Golden commands remain separate when they require external tools or credentials.

CLI workflow

Build the workspace, inspect the environment and execute a query workflow:

npm run build
npx --workspace @autovul/cli autovul doctor --json
npx --workspace @autovul/cli autovul workflow start --spec ./spec.json --json
npx --workspace @autovul/cli autovul query draft <run-id> --candidate ./candidate.json --json
npx --workspace @autovul/cli autovul query verify <run-id> --candidate ./candidate.json --json
npx --workspace @autovul/cli autovul workflow finalize <run-id> --json

Replay a finalized Query Pack without a model or the originating run:

npx --workspace @autovul/cli autovul query-pack verify ./query-pack \
  --vulnerable-db /path/to/vulnerable \
  --fixed-db /path/to/fixed \
  --workspace-root /trusted/root \
  --json

For the aggregate research interface, pass a structured request file; the CLI only validates, executes, inspects, cancels, or replays a bounded operation. It does not accept an open-ended research goal:

npx --workspace @autovul/cli autovul research validate --request ./flow-request.json --json
npx --workspace @autovul/cli autovul research execute --request ./flow-request.json --json
npx --workspace @autovul/cli autovul run status <run-id> --json
npx --workspace @autovul/cli autovul run cancel <run-id> --json
npx --workspace @autovul/cli autovul run replay <run-id> --json

The finalized pack contains the rendered query, candidate, vulnerability specification, qlpack metadata, verification result, evidence, reproduction instructions and a digest manifest.

Pi integration

Load the built extension from the project workspace:

npm run build
pi -e ./packages/pi-extension/dist/index.js

The extension exposes the target aggregate research interface:

• autovul_research — validate or execute a structured Flow hypothesis;
• autovul_run — inspect, cancel, or replay a bounded operation.

The following remain compatibility tools for the existing CodeQL workflow:

• codeql_database
• codeql_workflow
• codeql_query

Pi remains the Agent. The extension converts tool input, calls the shared Application API, handles cancellation and presents compact results. It does not start another Agent Loop.

Useful commands include /codeql, /codeql doctor, /codeql-status [run-id] and /codeql-generate.

Real verification gates

Run these gates only when their external dependencies are available:

npm run test:m3-compile
npm run test:m3-golden-real
npm run test:l0-lsp:snapshot
npm run test:lsp:conformance
npm run test:m4-pi-e2e
npm run test:m4-golden-real

The M3 real Golden gate covers 20 Python, JavaScript, Java and C/C++ vulnerable/fixed cases, strict endpoint checks and relocated replay. LSP results remain advisory; authoritative acceptance comes from CodeQL CLI execution. Model-backed evaluation must use an approved wrapper and reports blocked when the model, credentials or analyzer environment is unavailable. It never falls back to fake success.

Safety and platform status

• Paths are canonicalized and constrained to configured workspace or trusted roots.
• Long-running operations support timeout and cancellation and clean up subprocess trees.
• Critical state and artifacts use atomic persistence and deterministic recovery.
• Output, execution time, candidate count, revisions and concurrency are bounded.
• Target build and install scripts are not executed automatically.
• Recognized secrets and unrestricted environment data must not be persisted.

The accepted platform path is POSIX/macOS. Windows process-tree cleanup and Windows CI have not been implemented or verified, so Windows support is not currently claimed.

Compatibility

During the pre-1.0 migration window, pure-auto-codeql-v2 remains a CLI alias. Deprecated PURE_AUTO_CODEQL_* environment variables continue to work where documented, while matching AUTOVUL_* values take precedence. Stable historical CodeQL rule IDs retain their former namespace where changing them would break replay or compatibility.

The Python V1 runtime is outside this isolated TypeScript workspace and is not imported as a V2 dependency.