Skip to content

docs(guide): how to load many sheets without paying for it (HOT-15559) - #1759

Open
marcin-kordas-hoc wants to merge 4 commits into
developfrom
docs/perf-guide-bulk-load
Open

docs(guide): how to load many sheets without paying for it (HOT-15559)#1759
marcin-kordas-hoc wants to merge 4 commits into
developfrom
docs/perf-guide-bulk-load

Conversation

@marcin-kordas-hoc

@marcin-kordas-hoc marcin-kordas-hoc commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Context

A customer case (ClickUp HOT-15559) turned on a property of the engine that the guide never mentioned: with a dense cross-sheet dependency graph, loading sheets one at a time is quadratic. Measured on the released 3.4.0 with 500 cross-referencing sheets, the per-sheet loop takes over a hundred times as long as a single buildFromSheets call, and the customer had walked straight into it. Nothing in docs/guide/performance.md warned about loading at all — the page covers useColumnIndex, address-mapping policies, lazy-transformation cleanup and suspending recalculation, and stops there.

This adds a "Loading multiple sheets" section and two subsections for the findings that came out of the same investigation:

  • Build once. buildFromSheets against a per-sheet addSheet + setSheetContent loop, with the mechanism: every setSheetContent recalculates the loaded cells that depend on the sheet it just filled, so the work grows with each step. addSheet adds to this whenever the loaded formulas already point at the sheet being added, which is why pre-registering the sheet names removes about half the cost and not the growth.
  • Load order does not matter. A formula referencing a sheet that does not exist yet evaluates to #REF! and is repaired when that sheet arrives, with no re-parse. This shipped in 3.1.1 and is documented nowhere. It also means ordering the inserts by dependency is not a fix for the cost above — it holds only while every reference points one way.
  • Pass an instance, not the class. An integration handed the HyperFormula class calls buildEmpty and then adds sheets one at a time, which is the slow path by construction.

The section also states that reads throw while the evaluation is suspended, and links to batch operations — that is what turns a batch wrapped around a load loop into a crash in a host application that renders from the engine.

How did you test your changes?

Documentation only, no production code touched.

  • Both code samples were executed against the current develop source rather than written from memory. Reading Sheet1!B1 after the first sample gives 10; reading Hub!A1 through the second gives #REF!, then 1 after addSheet('Later'), then 42 after setSheetContent — the values the sample's comments claim. The samples themselves print nothing; the reads were added to run them.
  • The #REF!-repair claim was checked twice: the behaviour on the released 3.4.0, and the mechanism still present on develop (SheetMapping placeholder handling, isPlaceholder).
  • The performance claim comes from a benchmark of 500 sheets x 60 x 10 with 49,450 cross-sheet references, all pointing forward, medians of five interleaved runs per mode. Every run is gated on a correctness check that resolves one reference row per sheet — 4,945 of them, a tenth of the total — and no run was counted unless the gate passed. The naive loop measures 168,085 ms against 1,183 ms for one buildFromSheets.
  • grep over docs/guide/ confirmed the placeholder-sheet behaviour appears in no other page, so this does not contradict or duplicate an existing statement.
  • The frontmatter gains buildFromSheets and addSheet as search tags. The page keeps ten tags, which is the maximum every other guide page observes.

Review pass: the findings from an automated review are applied in 5f4513b — the tag count, the duplication with the page's own suspension section, the 3.1.1 version note on the placeholder behaviour, a noun-phrase heading, one cell-value convention across the samples, and dropping a general claim about third-party integrations in favour of the condition that actually matters.

Types of changes

  • Breaking change (a fix or a feature because of which an existing functionality doesn't work as expected anymore)
  • New feature or improvement (a non-breaking change that adds functionality)
  • Bug fix (a non-breaking change that fixes an issue)
  • Additional language file, or a change to an existing language file (translations)
  • Change to the documentation

Related issues:

  1. ClickUp HOT-15559 (customer case that surfaced the gap)

Checklist:

  • I have reviewed the guidelines about Contributing to HyperFormula and I confirm that my code follows the code style of this project.
  • I have signed the Contributor License Agreement.
  • My change is compliant with the OpenDocument standard.
  • My change is compatible with Microsoft Excel.
  • My change is compatible with Google Sheets.
  • I described my changes in the CHANGELOG.md file.
  • My changes require a documentation update.
  • My changes require a migration guide.

🤖 Generated with Claude Code


Note

Low Risk
Documentation-only change with no runtime or API behavior modifications.

Overview
Expands the performance guide with a new Loading multiple sheets section (plus frontmatter tags buildFromSheets and addSheet) so integrators avoid the slow incremental load path.

It recommends buildFromSheets over per-sheet addSheet + setSheetContent, explains why cross-sheet dependencies make incremental loads increasingly expensive (including addSheet when formulas already reference the new sheet), and points to batching and not reading cell values while evaluation is suspended when sheets must be added at runtime.

Subsections document that load order does not matter since 3.1.1 (missing sheets yield #REF! with live repair on add), that dependency-ordered inserts are not a reliable fix, and that passing a pre-built instance instead of the HyperFormula class avoids integrations that add sheets one-by-one.

Reviewed by Cursor Bugbot for commit da86b6a. Bugbot is set up for automated code reviews on this repo. Configure here.

marcin-kordas-hoc and others added 2 commits September 3, 2026 09:09
…-15559)

Loading sheets one by one is the slowest way to get data into the
engine, and the guide never said so. Every setSheetContent call
recalculates the loaded cells that depend on the sheet it just filled,
so with cross-sheet references the cost of a per-sheet loop grows with
every step; a customer case measured it at over a hundred times the
cost of a single buildFromSheets call for 500 cross-referencing sheets.

The section states the rule, explains the mechanism, notes that
pre-registering sheet names only removes the extra recalculation
addSheet triggers and not the growth itself, and points to batch
operations for the case where the data is not known upfront.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…OT-15559)

Three more findings from the same customer case, all of them things the
guide never said:

- A formula that references a sheet added later is repaired when that
  sheet arrives, without re-parsing, so load order is free. It also
  means ordering the inserts by dependency is not a fix for the cost of
  incremental loading: it holds only while every reference points one
  way.
- Reads throw while the evaluation is suspended, which is what turns a
  batch around a load loop into a crash in a host application that
  renders from the engine.
- Integrations that accept the HyperFormula class build an empty engine
  and add sheets one at a time; hand them an instance instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4dd93e4. Configure here.

Comment thread docs/guide/performance.md
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 3, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
hyperformula-docs da86b6a Commit Preview URL

Branch Preview URL
Sep 03 2026, 06:10 PM

- Drop the `loading` tag: every other guide page stops at ten tags and
  this page was the only one at eleven. The word is already in the
  heading it would index.
- Don't restate what "Suspending automatic recalculations" further down
  the same page already says; link to it instead.
- Attribute the doubling to the measurement it comes from rather than
  stating it as a property of addSheet.
- "Order of loading" as a noun phrase, matching every other heading on
  the page, and it no longer reads as an absolute the next paragraph
  walks back.
- Say that the placeholder-repair behaviour arrived in 3.1.1; without
  it the advice is false on earlier releases.
- One cell-value convention across both samples.
- Don't assert what third-party integrations do in general; state the
  condition and what to do about it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

Performance comparison of head (da86b6a) vs base (c920375)

                                     testName |    base |    head | change
--------------------------------------------------------------------------
                                      Sheet A |  509.42 |  512.74 | +0.65%
                                      Sheet B |  166.92 |  168.14 | +0.73%
                                      Sheet T |  149.57 |  146.54 | -2.03%
                                Column ranges |  492.18 |  487.42 | -0.97%
                                Sorted lookup | 15766.1 | 15118.7 | -4.11%
Sheet A:  change value, add/remove row/column |   17.03 |   16.86 | -1.00%
 Sheet B: change value, add/remove row/column |  149.77 |  147.69 | -1.39%
                   Column ranges - add column |  160.81 |   167.6 | +4.22%
                Column ranges - without batch |  497.12 |  503.82 | +1.35%
                        Column ranges - batch |  127.13 |  124.29 | -2.23%

"it has to call buildEmpty" asserts a necessity the API does not
impose — a library handed the class could collect the data and call
buildFromSheets itself. What is true is narrower: the engine gets built
on the library's terms, and one that receives sheets one at a time adds
them one at a time. Passing a prepared instance is what removes the
choice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.32%. Comparing base (c920375) to head (da86b6a).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##           develop    #1759   +/-   ##
========================================
  Coverage    97.32%   97.32%           
========================================
  Files          195      195           
  Lines        15739    15739           
  Branches      3390     3390           
========================================
  Hits         15318    15318           
  Misses         421      421           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant