From 9d4cdb2a3549fb3ccd5298fc3cb987bcb0b2633a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Efe=20G=C3=B6kdemir?= Date: Tue, 22 Sep 2026 07:39:56 +0300 Subject: [PATCH 1/2] docs: document JSON output format MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Codex Signed-off-by: Efe Gökdemir --- docs/cli_help.md | 5 ++++ docs/json_output.md | 73 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100644 docs/json_output.md diff --git a/docs/cli_help.md b/docs/cli_help.md index 59de7b47..45765d99 100644 --- a/docs/cli_help.md +++ b/docs/cli_help.md @@ -46,6 +46,11 @@ * `--json ` * Write JSON output to a file, or "stdout" / "stderr". + * See the [JSON output format](./json_output.md) for the document structure. + +* `--jsonbin ` + * Write JSON output and raw sample data to a file and binary sidecar directories. + * See the [JSON output format](./json_output.md) for sidecar details. * `--markdown `, `--md ` * Write markdown output to a file, or "stdout" / "stderr". diff --git a/docs/json_output.md b/docs/json_output.md new file mode 100644 index 00000000..fd85acb6 --- /dev/null +++ b/docs/json_output.md @@ -0,0 +1,73 @@ +# JSON Output + +NVBench writes benchmark results as a JSON document when you use the +`--json` option. Use `--jsonbin` when you also need the raw sample times and +sample frequencies for post-processing. + +```console +./benchmark --json results.json +./benchmark --jsonbin results-with-samples.json +``` + +Both options write the same JSON structure. The `--jsonbin` option additionally +writes binary sidecar files next to the JSON document. + +## Document Structure + +The top-level JSON object contains these keys: + +| Key | Description | +| --- | --- | +| `meta` | Command-line arguments and NVBench/JSON format versions. | +| `devices` | Device properties used by the benchmark run. | +| `benchmarks` | Benchmark definitions and their measured states. | + +The JSON format is versioned at `meta.version.json`. Consumers should inspect +this value before parsing the document. The current format version is `1.0.0`. + +Each item in `benchmarks` contains the benchmark `name`, numeric `index`, +execution settings, selected `devices`, `axes`, and `states`. Each state +contains its axis values, summaries, and an `is_skipped` flag. Skipped states +also include `skip_reason`. + +Summary values are represented by a `tag` and optional `name`, `description`, +`hint`, and `data` fields. Entries in `data` use this structure: + +```json +{ + "name": "mean", + "type": "float64", + "value": "1.234" +} +``` + +Integer and floating-point summary values are encoded as strings so consumers +do not lose precision when parsing large values. + +## Binary Sidecar Files + +With `--jsonbin`, NVBench creates these directories next to the JSON file: + +* `-bin/` contains sample-time files. +* `-freqs-bin/` contains sample-frequency files. + +For example, `results-with-samples.json` produces: + +```text +results-with-samples.json +results-with-samples.json-bin/0.bin +results-with-samples.json-freqs-bin/0.bin +``` + +The JSON summary that produced a sidecar contains a `filename` relative to the +JSON file's directory and a `size` with the number of stored samples. Resolve +the filename relative to the JSON file rather than the current working +directory. + +Each binary file stores `size` little-endian IEEE 754 32-bit floating-point +values. Use the summary's `filename` and `size` to associate a file with its +state; do not infer that association from the numeric filename alone. + +If a sidecar cannot be written, NVBench keeps the summary metadata but omits +`filename` and `size` and reports a warning. Consumers should treat such bulk +data as unavailable instead of assuming that an absent file is empty. From 89b973172ceab4b01c6c98b1e9ec2c13a0b1ff50 Mon Sep 17 00:00:00 2001 From: Codex Date: Tue, 27 Oct 2026 04:37:16 +0300 Subject: [PATCH 2/2] docs: clarify JSON device list scope --- docs/json_output.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/json_output.md b/docs/json_output.md index fd85acb6..58ec5229 100644 --- a/docs/json_output.md +++ b/docs/json_output.md @@ -19,7 +19,7 @@ The top-level JSON object contains these keys: | Key | Description | | --- | --- | | `meta` | Command-line arguments and NVBench/JSON format versions. | -| `devices` | Device properties used by the benchmark run. | +| `devices` | Properties for all detected CUDA devices. Each benchmark lists its selected device IDs in `benchmarks[].devices`. | | `benchmarks` | Benchmark definitions and their measured states. | The JSON format is versioned at `meta.version.json`. Consumers should inspect