Skip to content

perf(build): don't build the doc-generator binaries by default - #24338

Merged
Dandandan merged 1 commit into
apache:mainfrom
Dandandan:perf/gate-doc-bins
Aug 13, 2026
Merged

Dandandan merged 1 commit into
apache:mainfrom
Dandandan:perf/gate-doc-bins

Conversation

@Dandandan

@Dandandan Dandandan commented Aug 13, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Adresses: #13814

Found while profiling compile times for #24325 / #24326 / #24329 / #24330.

Rationale for this change

datafusion/core/src/bin/ holds three binaries that regenerate the docs under
docs/source/user-guide: print_config_docs, print_runtime_config_docs and
print_functions_docs. Cargo auto-discovers them and they have no
required-features, so every cargo build links all three — each one
~174MB, since each links the whole datafusion rlib.

Nothing in normal development uses them. They are run by
dev/update_config_docs.sh and dev/update_function_docs.sh, and by the CI job
that checks the committed docs are up to date.

Two places where this shows up:

Cold builds. The three binaries link after every other unit has finished,
so they sit on the critical path with nothing to overlap with.
cargo build --timings shows them occupying the last 3.5s of a
cargo build -p datafusion (~8.8s of CPU), after the last library unit
completes.

The tightest inner loop — touch a file in core, rebuild. All three are
relinked every time:

before: 3.0s  2.4s
after:  1.3s  1.1s

What changes are included in this PR?

The three binaries move behind a new non-default docs_generation feature, and
the two dev/ scripts pass --features docs_generation.

Using required-features means declaring the [[bin]] targets explicitly, since
auto-discovered targets cannot carry it.

Are these changes tested?

  • cargo build -p datafusion no longer produces the three binaries
  • cargo build -p datafusion --features docs_generation does
  • ./dev/update_config_docs.sh still regenerates
    docs/source/user-guide/configs.md byte-identically (empty git diff
    afterwards), which is what the CI doc check compares

dev/update_function_docs.sh uses the same invocation pattern and all three of
its call sites were updated; CI exercises both scripts.

Are there any user-facing changes?

The three binaries are no longer built by a default cargo build. Anyone who
ran them directly needs --features docs_generation — same as the dev/ scripts
now do. No library API changes.

If you would rather these lived outside the published crate altogether, moving
them to a small non-published dev/ crate would have the same effect on build
times; I went with the smaller change.

🤖 Generated with Claude Code

`datafusion/core/src/bin/` holds three binaries that regenerate the docs under
`docs/source/user-guide`: `print_config_docs`, `print_runtime_config_docs` and
`print_functions_docs`. They are auto-discovered by cargo with no
`required-features`, so every `cargo build` links all three -- each ~174MB,
against a crate whose own rlib they link in full.

They are only ever run by `dev/update_config_docs.sh` and
`dev/update_function_docs.sh` (and by the CI job that checks the committed docs
are up to date), so they now sit behind a non-default `docs_generation` feature,
and those scripts pass `--features docs_generation`.

Effect on the two builds that matter:

- Cold `cargo build -p datafusion`: the three binaries link *after* everything
  else has finished, so they sit on the critical path with nothing to overlap
  with. `cargo build --timings` shows them occupying the last 3.5s of the build
  (~8.8s of CPU).
- Edit-a-file-in-core and rebuild, the tightest inner loop -- they are relinked
  every time:

      before: 3.0s  2.4s
      after:  1.3s  1.1s

Verified that a default `cargo build -p datafusion` no longer produces them,
that `--features docs_generation` does, and that `./dev/update_config_docs.sh`
still regenerates `docs/source/user-guide/configs.md` byte-identically.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions github-actions Bot added development-process Related to development process of DataFusion core Core DataFusion crate labels Aug 13, 2026
@Dandandan Dandandan changed the title build: don't build the doc-generator binaries by default perf(build): don't build the doc-generator binaries by default Aug 13, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.17%. Comparing base (ab12f5e) to head (17ab914).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24338      +/-   ##
==========================================
+ Coverage   81.14%   81.17%   +0.03%     
==========================================
  Files        1112     1109       -3     
  Lines      386933   386746     -187     
  Branches   386933   386746     -187     
==========================================
- Hits       313967   313957      -10     
+ Misses      54476    54295     -181     
- Partials    18490    18494       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Dandandan good catch

@Dandandan
Dandandan added this pull request to the merge queue Aug 13, 2026
Merged via the queue into apache:main with commit f171587 Aug 13, 2026
40 checks passed
@Dandandan
Dandandan deleted the perf/gate-doc-bins branch August 13, 2026 18:16
imtherealnaska pushed a commit to imtherealnaska/datafusion that referenced this pull request Aug 16, 2026
…e#24338)

## Which issue does this PR close?

Adresses: apache#13814

Found while profiling compile times for apache#24325 / apache#24326 / apache#24329 /
apache#24330.

## Rationale for this change

`datafusion/core/src/bin/` holds three binaries that regenerate the docs
under
`docs/source/user-guide`: `print_config_docs`,
`print_runtime_config_docs` and
`print_functions_docs`. Cargo auto-discovers them and they have no
`required-features`, so **every `cargo build` links all three** — each
one
~174MB, since each links the whole `datafusion` rlib.

Nothing in normal development uses them. They are run by
`dev/update_config_docs.sh` and `dev/update_function_docs.sh`, and by
the CI job
that checks the committed docs are up to date.

Two places where this shows up:

**Cold builds.** The three binaries link *after* every other unit has
finished,
so they sit on the critical path with nothing to overlap with.
`cargo build --timings` shows them occupying the last **3.5s** of a
`cargo build -p datafusion` (~8.8s of CPU), after the last library unit
completes.

**The tightest inner loop** — touch a file in core, rebuild. All three
are
relinked every time:

```
before: 3.0s  2.4s
after:  1.3s  1.1s
```

## What changes are included in this PR?

The three binaries move behind a new non-default `docs_generation`
feature, and
the two `dev/` scripts pass `--features docs_generation`.

Using `required-features` means declaring the `[[bin]]` targets
explicitly, since
auto-discovered targets cannot carry it.

## Are these changes tested?

- `cargo build -p datafusion` no longer produces the three binaries
- `cargo build -p datafusion --features docs_generation` does
- `./dev/update_config_docs.sh` still regenerates
  `docs/source/user-guide/configs.md` byte-identically (empty `git diff`
  afterwards), which is what the CI doc check compares

`dev/update_function_docs.sh` uses the same invocation pattern and all
three of
its call sites were updated; CI exercises both scripts.

## Are there any user-facing changes?

The three binaries are no longer built by a default `cargo build`.
Anyone who
ran them directly needs `--features docs_generation` — same as the
`dev/` scripts
now do. No library API changes.

If you would rather these lived outside the published crate altogether,
moving
them to a small non-published `dev/` crate would have the same effect on
build
times; I went with the smaller change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Dandandan added a commit to emilk/datafusion that referenced this pull request Aug 21, 2026
…nto 9 (apache#24342)

## Which issue does this PR close?

None. Found while profiling compile times (apache#24325, apache#24326, apache#24329,
apache#24330,
apache#24338, apache#24339) — this is the largest remaining item I measured.

## Rationale for this change

`datafusion-functions` declares **63 `[[bench]]` targets**. Each one is
a
separate binary that statically links this crate, arrow and criterion,
so
building the crate's benchmarks means 63 link steps.

Linking, not compiling, is the bulk of it: rebuilding a **single** bench
target
takes **10.6s**, almost all of it link, while the 63 benchmark files
together are
only ~10.5k lines of source.

This is paid by `cargo bench`, by `cargo build --all-targets`, and by
CI's
`clippy --all-targets`.

## What changes are included in this PR?

The targets are grouped by their `required-features`, which preserves
that
semantics exactly — a group can only be built when its feature is on,
just as
each individual bench could before:

| target | benches | required-features |
|---|---|---|
| `string_expressions` | 16 | `["string_expressions"]` |
| `math_expressions` | 16 | `["math_expressions"]` |
| `unicode_expressions` | 12 | `["unicode_expressions"]` |
| `datetime_expressions` | 8 | `["datetime_expressions"]` |
| `regex_expressions` | 4 | `["regex_expressions"]` |
| `misc` | 4 | none |

`crypto`, `encoding` and `dictionary_encoding` stay standalone — they
are the
only members of their feature groups, so grouping them would buy
nothing.

Each group lives in its own directory, `benches/<group>/`, with
`main.rs`
declaring the individual benchmarks as modules and listing their
criterion
groups in a single `criterion_main!`. Cargo does not auto-discover
targets
inside those directories, so no `autobenches = false` is needed — which
also
keeps `cargo machete` working (it stops scanning `benches/` when
autodiscovery
is disabled). `benches/helper.rs` stays shared at the top level, reached
with
`#[path = "../helper.rs"]`.

**The benchmark code itself is untouched.** There is not a single change
to a
`bench_function`, `benchmark_group` or `bench_with_input` line anywhere
in the
diff — git reports every moved file as a 94–99% similarity rename. Per
file the
change is only:

- drop `criterion_main!` — a module cannot define `main`
- drop the now-unused `criterion_main` import
- drop `extern crate criterion` (a no-op since edition 2018, present in
4 files)
- point the four users of `benches/helper.rs` at `crate::helper`

## Are these changes tested?

Interleaved with `main` so machine drift cancels out:

| `cargo clean -p datafusion-functions` then… | main | this PR |
|---|---|---|
| `cargo build --benches` | 43.9s / 42.9s | **28.4s / 28.3s** |
| `cargo check --benches` | 15.0s / 14.7s | **12.9s / 12.8s** |

Every benchmark still runs, and the counts show nothing was dropped:

| target | benchmarks (`-- --test`) |
|---|---|
| `string_expressions` | 252 |
| `unicode_expressions` | 251 |
| `datetime_expressions` | 103 |
| `dictionary_encoding` | 36 |
| `regex_expressions` | 27 |
| `misc` | 21 |
| `encoding` | 9 |
| `crypto` | 10 (with `--features crypto_expressions`) |

Also clean: `cargo clippy -p datafusion-functions --all-targets`,
`cargo fmt --check`, and `cargo machete --with-metadata`.

Filtering a single benchmark still works — `cargo bench --bench
math_expressions -- power` selects exactly one.

`cargo metadata` reports 10 bench targets, against 64 on `main`; both
counts
include the auto-discovered `helper`, which is unchanged by this PR.

## Are there any user-facing changes?

Only for running an individual benchmark. Where you would previously
write

```
cargo bench -p datafusion-functions --bench lower
```

you now select the group and filter by name:

```
cargo bench -p datafusion-functions --bench string_expressions -- lower
```

The filter is criterion's own, matching on benchmark id, so it also
narrows to a
single case within a benchmark. No library changes.

If this approach looks right, roughly 100 more bench targets across the
other
crates could get the same treatment — I did this one crate first so the
tradeoff
is visible on a real diff before it spreads.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate development-process Related to development process of DataFusion v56.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants