Skip to content

Signature Database

github-actions[bot] edited this page Sep 3, 2026 · 1 revision

Generated from 'wiki-signature-database.ts' on 2026-09-01, 11:46:30 UTC (v2.15.8, R v4.6.1), please do not edit directly.

Signature Database

flowR ships a database of the complete history of all exports in every version of all CRAN packages so it can resolve calls into the packages you load. After library(ggplot2), a call to ggplot() resolves to ggplot2::ggplot. The same database qualifies bare names and backs various components like the dependencies and call-context queries as well as the undefined symbol and unused import rules.

You can search what it knows at flowr-analysis.github.io/flowr/wiki/sigdb, a static page listing every exported name, generated from this database by npm run gen:landing.

What is stored

Every function is a DecodedFunction:

field holds
DecodedFunction::exported whether the name is a package export
DecodedFunction::signature the parameters, with their defaults and their ArgProp mask
DecodedFunction::callees the function's own local calls
DecodedFunction::topic the Rd help topic when it differs from the name
DecodedFunction::file, DecodedFunction::line source location
DecodedFunction::props flags like higher-order, recursive, deprecated

The parameter mask is the one flowR states its own built-ins with, so a parameter carries ArgProp::NoDefault when it has no default, ArgProp::Forced when the function always evaluates it, and whichever roles the extractor could infer (ArgProp::Alias, ArgProp::Presence, ArgProp::Callee, ...). Every bit it cannot see stays unset, so an unset bit reads as "unknown" rather than "no"; fnInfoFromSignature hands the mask on unchanged, which is what lets a package function answer the same questions a built-in does.

Per version the source also answers declared dependencies (ResolvedDependency), release dates, the plain export view (LibraryExports), the versions it carries (AvailableVersion), and its class relations (SigClassInfo, via SigDatabase::classes).

A class record states what a declaration does: its direct superclasses, its slots with the types they were declared with, whether it is virtual, and whether it is a setClassUnion (whose supers are the members it unites). SigClassInfo::package names the package defining a class the record only relates to, which is what tells a class the package owns from one it inherits -- something the flat name list of LibraryExports::s4Classes has nowhere to hang. The same shape carries Reference classes, S7 and R6, since all four declare a name, a parent and a set of members. On the analysis side declaredClasses reads these off setClass/setClassUnion/setIs/setRefClass/new_class/R6Class calls and toSigClasses hands them over in this form.

Beyond the flags above, DecodedFunction::props also carry FnProp::NoDoc (a documented package has no help page for this name), FnProp::S3Method (a registered S3 method, from the package NAMESPACE or base R's method table), and FnProp::S3Owner (an exported constructor for an S3 class this package OWNS: it also registers at least one S3 method for that class). The owned classes of a version are LibraryExports::s3Classes, and SigDatabase::classOwner answers, for a class name, which package owns it (backed by a reverse index built once). This lets version guessing mark a package used when the analyzed project's own NAMESPACE registers an S3 method for a class it owns, even with no direct call, e.g. tseries's S3method("as.irts","zoo") marks zoo used.

The S4 side has FnProp::S4Owner for an exported class and FnProp::S4Method for a name a package exports because it answered a generic for one of its classes (setMethod("sin", "float32", ...) plus exportMethods(sin)), rather than because it defines a function of its own. Such a name is often documented only under its sin,float32-method Rd alias, so it also carries FnProp::NoDoc. Because setMethod("Math", ...) answers every member of a group at once, SignatureDb::functionOf falls back to the group entry for a member it finds nothing for: pkg::sin is served by pkg's Math, which is what the call would dispatch to. groupGenericOf maps a member to its group.

FnProp::Generic says the definition is one others dispatch on: an S3 generic whose body calls UseMethod, an S4 one from setGeneric, or an S7 new_generic. The call graph shows the same for the S3 case, but only while a bundle carries one, and never for a generic built without an R body -- which is why the bit exists next to it. fnInfoFromSignature reads it, falling back to the dispatching callee for a bundle written before it.

FnProp::Value says the export binds a value rather than a function (pi, LETTERS, ggplot2's class_gg). Only the extractor can tell: an entry without a definition location is as likely to be a function nothing wrote down, an S4 generic setGeneric builds or a Vectorize result, so a reader that has only the location to go on can say no more than that there is none.

These are derived on demand by the signature query, not stored:

Reading It From an Analyzer

FlowrAnalyzerDependenciesContext::signatures is the entry point, and it is the one you want.

function fromTheAnalyzer(analyzer: FlowrAnalyzer) {
	const db = analyzer.inspectContext().deps.signatures();
	const lead = Identifier.make('lead', 'dplyr');
	return {
		version:    db.versionOf('dplyr'),      // the version this analysis assumes
		fn:         db.functionOf(lead),        // its entry, decoding only this one function
		parameters: db.parametersOf(lead),      // its formals, ready for MatchArgs.toNames
		exports:    db.exportsOf('dplyr')?.exported
	};
}

Defined at src/documentation/wiki-signature-database.ts#L29

The SignatureDb it hands back is every loaded source as one database, answering for the version the analyzed project assumes for each package, which is the version solver.sigdb.versionOverrides, solver.sigdb.versionSelection and solver.sigdb.assumedRVersion produced. That matters, because a PackageSignatureSource asked without a version answers for whatever it happens to hold as newest, which is not what the analysis assumes. When the assumed version is one the database does not carry, the answer falls back to the newest it has and says so in the log rather than quietly answering for another version.

SignatureDb::sources is the escape hatch to the raw sources for what the interface above does not cover, and reaches the same functions directly.

	const fn = source.functionByName('dplyr', 'lead', '1.1.4');
	return {
		exported:   fn?.exported,
		signature:  fn?.signature.map(p => p.name),
		localCalls: fn?.callees,
		topic:      fn?.topic,
		location:   [fn?.file, fn?.line],
		transitive: source.transitiveCallees('dplyr', 'lead', '1.1.4'),
		deps:       source.dependencies('dplyr', '1.1.4'),
		exports:    source.lookup('dplyr')?.exported,
		s3Classes:  source.lookup('zoo')?.s3Classes,
		classOwner: source.classOwner('zoo')
	};
}

Defined at src/documentation/wiki-signature-database.ts#L40

To check what a project can resolve against without touching the raw sources, a context exposes FlowrAnalyzerDependenciesContext::hasSignatureDatabase (a cheap presence check) and FlowrAnalyzerDependenciesContext::availableSignatureDatabases (the identifying names of the loaded databases), alongside the richer FlowrAnalyzerDependenciesContext::loadedSignatureDatabases metadata.

Configuration

The exports come from versions:sigdb, which reads bundled databases. It is enabled by default (see configuring flowR).

Which version's exports get resolved is decided by the version-reading plugins that pin the packages a project uses.

function usePackageDatabase(parser: KnownParser) {
	const sigdb = new FlowrAnalyzerPackageVersionsSigDbPlugin('/path/to/sigs.manifest.json.br');
	return new FlowrAnalyzerBuilder().setParser(parser).registerPlugins(sigdb).build();
}

Defined at src/documentation/wiki-signature-database.ts#L23

File sources load lazily on the first package load, so a script with no library() or use() calls never pays to parse them. Set solver.sigdb.eagerlyLoad to mount the database up front instead, or solver.sigdb.enabled to false to switch it off entirely. For a compressed (.br) or manifest source, preload it before analysis to mount it.

The base-R packages (base, stats, graphics, ...) resolve against an assumed R version, which defaults to 4.5.3 (solver.sigdb.assumedRVersion, or "auto" to detect the local R). So library(stats) attaches that release's exports, and a bare sd() qualifies to stats::sd even without attaching the base namespaces to the graph. Set solver.sigdb.linkBaseR to also link them as dataflow edges.

Signature shards are not committed to the repository because of their size (the current.* and history.* scopes span tens of megabytes): the base.* floor (self-contained base-R signatures, a few hundred KB), the current.* scope (every package's latest version) and history.* (every older version) all live as assets on the free solver.sigdb.downloadRepo GitHub release. The only committed file is a tiny link file, src/data/sigdb/sigdb.remote.json, which records the release tag and each shard's sha256 and size, so :signature download builds the direct release-CDN URL, verifies every shard by content hash, and skips any already cached. Because the link file is versioned, a git pull that updates it re-syncs only the shards whose hash changed — and with solver.sigdb.autoSync that check runs on startup and re-downloads in the background; npm run build bakes the shards in as well. The richest downloaded scope is used (order full > current > base), so once fetched library(stats) resolves. Any path in solver.sigdb.additionalPaths (or $FLOWR_SIGDB_DIR) is searched alongside the default, so a downloaded bundle stays mounted on every start.

Bundled Databases

The default bundle is not a single file but a set of shards that a manifest routes between (see SigDatabaseSet). Nothing is read when the manifest opens. The first lookup of a package decompresses only the one shard that holds it, plus the shared dictionary once. The following ship with this build; the load column is the decompression time measured at generation time.

Shard Contents Versions kept Packages Versions Size (.br) Load (first touch)
base-current base-R packages (base, stats, graphics, ...) latest only 23 23 104 KB ≈ 1.4 ms
base-full base-R packages (base, stats, graphics, ...) full history 23 1,626 468 KB ≈ 14 ms
current-top the 1,000 most-downloaded CRAN packages latest only 1,000 1,000 2.1 MB ≈ 43 ms
current-rest the remaining CRAN packages latest only 22,742 22,742 15.1 MB ≈ 420 ms
history-rest the remaining CRAN packages full history 18,466 140,128 30.7 MB ≈ 1100 ms

Which shard answers a lookup follows from the package and the version asked for. A base-R package comes from base-current, one of the 1,000 most-downloaded CRAN packages from current-top, and anything else from current-rest. The *-full and history-* shards hold every historical version and are only touched when an older, pinned version is requested, so a normal analysis never decompresses them. Each scope carries its own shared dictionary that its shards depend on, so it is decompressed the first time any of its packages is looked up and then reused. The flowR Docker images ship this dictionary already decompressed, so a container reads it in place and skips that step (the load column above is the cost a plain npm install pays).

Every shard, dictionary, and manifest is published in both brotli (.br) and zstd (.zst, faster to decompress) compression, and flowR uses whichever the runtime supports: .zst when the Node version exposes zstd (Node ≥ 22.15), otherwise .br. :signature download fetches only that one variant per file, and :version reports the format each loaded database resolved to.

Format

The on-disk format is flowr-sigdb (schema 5). Beyond each version's exports it records, per version, every function's signature (the parameters, each with its default and its ArgProp mask) and call graph, together with that version's declared dependencies (Depends, Imports, ... with their version qualifiers). The layout is NDJSON: a header, then a shared string dictionary, then one self-contained blob per package, next to a sidecar .idx. A reader (SigDatabase) therefore loads the dictionary once and then seeks straight to the packages it needs, never reading the rest. The bundle is written by SigDbBuilder and can be split into several small shards (current-only versus full history, top-N versus the rest) that a flowr-sigdb-manifest routes transparently (SigDatabaseSet), and which information gets stored is selectable (SigDbFeatures). The extractor produces the bundle from its analysis of CRAN.

Performance

The dictionary is read once, the reader then seeks straight to each requested package, and consumers cache what they derive (the base-package list is precomputed when flowR is bundled, so it costs nothing at analysis time). After the one-time load a per-package lookup is O(1), so each library() or :: a script uses is a single cached lookup.

Clone this wiki locally