You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
docs: version each product's documentation independently - #12592
Splits the documentation into independently versioned products, published as one site. Previously one Sphinx project built all of docs/, so Python's pages were published beneath the C++ version, and each deploy kept only the build that had just run.
Major design components:
Three Sphinx projects, assembled into one site:
docs/ C++ -> /cccl/cpp/<version>/
docs/python/ Python -> /cccl/python/<version>/
docs/site/ no version -> /cccl/
A build carries exactly one version, so independently versioned products need separate builds. The root project is unversioned and holds the repository-level docs (contributors, maintainers, infrastructure), which move there from docs/.
docs/registry.yml lists the products. The directories each build skips, the cross-reference mappings, release-tag routing, the landing index and each product's build command all read it, so a new product is one registry entry plus its own build script and Sphinx project. Registry order is build order.
Cross-references between projects resolve through intersphinx: Python against C++, the root against both, and the C++ build maps nothing. A project maps only the products listed above it, since those are built first, and reads their inventories from the site being assembled (or this run's build output) before the published site, which is a publish behind.
Releases are driven by tags: the prefix selects the product (v for C++, python- for Python) and the remainder is the version. Only an exact MAJOR.MINOR.PATCH publishes; pre-release tags are refused. source-ref lets a tag cut before this change be built from another ref.
One deploy with force_orphan. The assembly seeds itself from the published branch before adding the version just built, so the artifact is the whole site and recreating the branch loses nothing. On a fork, one snapshot measured 10 MB while 56 deploys with force_orphan off accumulated 90 MB; seeded, the branch stays one commit whose size follows the content.
Each product's switcher manifest is generated from the published branch's directory listing, and check_manifests() fails the run if it disagrees with the assembled site.
Seeding is guarded: runs publishing one branch are serialized, and a failed fetch of the published branch stops the run instead of being treated as a first publication.
Pull-request CI runs the same assembly without seeding or publishing, so every pull request builds all three projects with warnings as errors, including the contributor and maintainer docs that a products-only build would stop checking.
Minor changes and comments:
cuda.coop stays in the Python product until cuda-coop versions separately. Its Sphinx settings moved to docs/python/conf.py, and its visualization extension resolves pages from the Python project's root.
Seven labels were defined both in a docs page and in a C++ header's Doxygen comment. Sphinx's parallel build reports a duplicate label only when both copies are read in the same chunk, so these surfaced or stayed hidden depending on the document set and CPU count; the C++-only build exposed one. None was referenced, so the header copies are removed.
Both workflows create the docs virtual environment before anything reads the registry, so nothing is installed into the runner's system Python; PyYAML is declared in docs/requirements.txt.
Removed: docs-deploy.yml; the custom 404 machinery (404.html, its two helpers and scrape_docs.bash); the hand-written docs/index.html chooser; and docs/cccl/index.rst, whose content moved into the C++ landing page.
The moved docs link into the C++ pages through cpp: cross-references, and paths and URLs in READMEs, AGENTS.md, .coderabbit.yaml, CI messages and iterator docstrings follow the new layout.
Unstable pages carry a development-documentation banner.
ci/project_files_and_dependencies.yaml ignores build-site.yml in place of docs-deploy.yml.
docs/test_docs_build.py covers the failures that would otherwise be silent: tag routing, a manifest disagreeing with the site, the inventory source, seeding before force_orphan, and per-version replacement. A pre-commit hook runs it.
Modelled on rapidsai/docs. Verified end to end on a fork: three C++ and two Python versions published, one commit on the branch, and every version reachable from the switcher.
Description
closes
Checklist
New or existing tests cover these changes.
The documentation is up to date with these changes.
This pull request changes a CodeRabbit configuration file. Because it comes from a fork or its author is not a repository collaborator, reviews use only the configuration from the target branch. The proposed configuration will take effect after it is merged.
Summary
Summary by CodeRabbit
New Features
Documentation is organized into product-specific C++ and Python sections, with version-aware navigation and a shared landing page.
Documentation can be published as a complete site or for a selected product and release, while retaining other published versions.
Unstable C++ documentation displays a development notice.
Documentation
Updated cross-references and API links to match the new structure.
Replaced the former CCCL overview with a C++ libraries landing page.
Behavior Changes
The custom not-found page, similar-page search, and redirect behavior have been removed.
The pull request replaces the documentation build and deployment flow with per-product builds and combined versioned-site publication. It adds registry-driven configuration, assembly scripts, and tests, and updates documentation navigation and links to the new layout.
The registry defines C++ and Python products and their version rules. Product configurations and build scripts create versioned artifacts and manifests. Tests cover version selection, artifact layout, manifests, redirects, and Intersphinx mappings.
Site assembly and publication docs/build_site.py, docs/site/conf.py, docs/site/.gitignore, docs/.gitignore, .github/workflows/build-site.yml, .github/workflows/docs-deploy.yml, .github/actions/docs-build/action.yml, .pre-commit-config.yaml, ci/project_files_and_dependencies.yaml
The site builder seeds published content, places product artifacts, builds the root documentation, and checks manifests. The new workflow resolves build inputs and publishes the assembled site; the prior deployment workflow is removed. A pre-commit hook runs build-contract tests.
The root documentation becomes the C++ libraries landing page, and the site project generates product navigation. Documentation references and package links use the product namespaces and versioned paths.
The previous 404 pages, page scraper, and root redirect are removed. The change also removes the prior CCCL overview page and selected RST anchors from API documentation.
Suggested labels:docs, infra
Suggested reviewers:gonidelis
Change: Feature
Merge Risk: Low · up to 53554
The docs build and publish flow now validates its inputs and its tests are portable. A queued release-docs publish can still be dropped silently when a newer run replaces it, and old development URLs can serve frozen content. Both are recoverable and low impact, so the change is mergeable with owner awareness.
Security architecture risk: Low · up to 53554
The inspected changes do not establish a new unauthenticated attack path or broader publishing permissions. The main design risk is conditional loss of retained documentation: remote-read failures are not consistently distinguished from an empty publication branch before whole-site replacement.
Retained concerns
Medium · reliability · inferred: The new preservation lifecycle can enter its first-publication state without proving that the target branch is absent. After a failed fetch, the workflow accepts any nonzero ls-remote result, including remote-access errors. If no usable tracking ref exists, assembly seeds nothing; the manifest fallback probes the supplied origin/branch name, and final validation does not detect omitted products or versions absent from the assembled tree. Whole-site replacement could therefore discard retained documentation. Existing tracking refs and successful remote reads substantially constrain this failure path. Force-orphan publication predates this PR; the concern is the incomplete failure containment in the new shared-product preservation contract.
Security review details
Security Blast Radius
observed — All product versions share one publication branch and one replacement operation, so a preservation failure can affect documentation beyond the selected release. The publishing token has repository contents-write authority; the target-branch allowlist constrains workflow routing rather than the token's underlying permission.
Trust Boundaries and Controls
observed — Manual source-ref selects the checkout whose requirements, registry, and build code execute in the publishing job. Tag and branch inputs are passed through environment variables or arguments, and validated version labels are the only labels interpolated into registry build commands. The registry remains executable build configuration, not an untrusted-data sandbox.
Resilience and Maintainability Implications
observed — Build failures stop execution before the publish step, and an existing seed commit whose docs directory cannot be archived aborts assembly. Tests cover selected-version replacement and unreadable seed content, but the workflow fetch-failure test checks source text rather than exercising remote-error exit statuses.
Hardening Proposals
proposed — Make publication initialization fail closed: accept only the explicit remote-ref-not-found status as absence, keep remote head names distinct from local tracking refs, and carry a verified seed snapshot or explicit empty-site decision through assembly and validation.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
suggestion: run() passes an unquoted string to shell=True. The callers build it from python_bin(), VENV, HERE and root_source (Lines 68-71 and 134-138). If the checkout path contains a space, those commands split at the space. build_root and ensure_dependencies then fail with a confusing error. The gen_*.bash scripts already quote their paths.
Quote each interpolated path with shlex.quote. Alternatively, take a list for the internal commands. Use shlex.split only for the registry build string.
suggestion: The files filter omits inputs that test_docs_build.py reads. A change to any of these does not trigger the hook:
docs/cpp_site/index.html and docs/python_site/index.html. test_checked_in_redirects_still_name_production and test_no_legacy_combined_paths_remain read them.
docs/site/conf.py. test_no_legacy_combined_paths_remain loads it.
docs/conf.py. test_loose_markdown_is_excluded_by_name reads it.
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
Major · Stability & Availability · Heavy lift
important: cancel-in-progress: false does not queue every run. GitHub keeps one running and one pending run per concurrency group. A newer run replaces any older pending run in the group.
If a v3.5.0 dispatch and a python-1.2.0 dispatch are both queued behind a running publish, a later push to main can cancel the older pending release dispatch. That release is then never published, and the workflow gives no failure signal. The test at test_publishing_one_branch_at_a_time assumes no run is dropped.
Fix this one of two ways:
Give release dispatches a unique concurrency group per tag. Serialize the push-to-main runs separately.
Make the publish step safe to run from a single latest run, and document that dispatches must be sequential.
The reason will be displayed to describe this comment to others. Learn more.
Agreed. I believe the fix is queue: max, which keeps runs serialized without letting a newer run replace a waiting one, but the repo's pinned actionlint doesn't support that key yet, so I've left it out of this PR. I don't think per-tag groups are an option. Two runs could then publish gh-pages at once, and one would drop the other's version.
The reason will be displayed to describe this comment to others. Learn more.
Analysis chain
web_search executed:
Check official GitHub Actions documentation for workflow concurrency queue: max: supported syntax, maximum pending runs, behavior at the limit, and compatibility with cancel-in-progress: false.
@sri-koundinyan important: My per-tag group suggestion was incorrect. Concurrent publishes could overwrite each other’s versions.
GitHub documents queue: max as allowing up to 100 pending runs. It cancels additional runs when the queue is full.
Until the fix is available, check that each release dispatch succeeds and rerun it if canceled. Would you like me to open a follow-up issue for actionlint support and queue: max?
Splits the documentation into independently versioned products, published as
one site. Previously one Sphinx project built all of docs/, so Python's pages
were published beneath the C++ version, and each deploy kept only the build
that had just run.
Major design components:
- Three Sphinx projects, assembled into one site:
docs/ C++ -> /cccl/cpp/<version>/
docs/python/ Python -> /cccl/python/<version>/
docs/site/ no version -> /cccl/
A build carries exactly one version, so independently versioned products
need separate builds. The root project is unversioned and holds the
repository-level docs (contributors, maintainers, infrastructure), which
move there from docs/.
- docs/registry.yml lists the products. The directories each build skips, the
cross-reference mappings, release-tag routing, the landing index and each
product's build command all read it, so a new product is one registry entry
plus its own build script and Sphinx project. Registry order is build order.
- Cross-references between projects resolve through intersphinx: Python
against C++, the root against both, and the C++ build maps nothing. A
project maps only the products listed above it, since those are built
first, and reads their inventories from the site being assembled (or this
run's build output) before the published site, which is a publish behind.
- Releases are driven by tags: the prefix selects the product (v for C++,
python- for Python) and the remainder is the version. Only an exact
MAJOR.MINOR.PATCH publishes; pre-release tags are refused. source-ref lets a
tag cut before this change be built from another ref.
- One deploy with force_orphan. The assembly seeds itself from the published
branch before adding the version just built, so the artifact is the whole
site and recreating the branch loses nothing. On a fork, one snapshot
measured 10 MB while 56 deploys with force_orphan off accumulated 90 MB;
seeded, the branch stays one commit whose size follows the content.
- Each product's switcher manifest is generated from the published branch's
directory listing, and check_manifests() fails the run if it disagrees with
the assembled site.
- Seeding is guarded: runs publishing one branch are serialized, only
gh-pages or a gh-pages-<name> rehearsal branch can be targeted, and a failed
fetch of the published branch stops the run instead of being treated as a
first publication.
- Pull-request CI runs the same assembly without seeding or publishing, so
every pull request builds all three projects with warnings as errors,
including the contributor and maintainer docs that a products-only build
would stop checking.
Minor changes and comments:
- cuda.coop stays in the Python product until cuda-coop versions separately.
Its Sphinx settings moved to docs/python/conf.py, and its visualization
extension resolves pages from the Python project's root.
- Seven labels were defined both in a docs page and in a C++ header's Doxygen
comment. Sphinx's parallel build reports a duplicate label only when both
copies are read in the same chunk, so these surfaced or stayed hidden
depending on the document set and CPU count; the C++-only build exposed
one. None was referenced, so the header copies are removed.
- Both workflows create the docs virtual environment before anything reads
the registry, so nothing is installed into the runner's system Python;
PyYAML is declared in docs/requirements.txt.
- The site URL is derived from the repository, with the owner lowercased as
Pages serves it, so production and every fork need no configuration.
- Removed: docs-deploy.yml; the custom 404 machinery (404.html, its two
helpers and scrape_docs.bash); the hand-written docs/index.html chooser; and
docs/cccl/index.rst, whose content moved into the C++ landing page.
- The moved docs link into the C++ pages through cpp: cross-references, and
paths and URLs in READMEs, AGENTS.md, .coderabbit.yaml, CI messages and
iterator docstrings follow the new layout.
- Unstable pages carry a development-documentation banner.
- ci/project_files_and_dependencies.yaml ignores build-site.yml in place of
docs-deploy.yml.
- docs/test_docs_build.py covers the failures that would otherwise be silent:
tag routing, a manifest disagreeing with the site, the inventory source,
seeding before force_orphan, and per-version replacement. A pre-commit hook
runs it.
- Modelled on rapidsai/docs. Verified end to end on a fork: three C++ and two
Python versions published, one commit on the branch, and every version
reachable from the switcher.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
suggestion: Stream the seed archive to disk instead of holding it in memory.
git archive writes the whole published docs/ tree, which is every C++ and Python version. Right now capture_output=True keeps that whole tar in memory. The code then writes a second full copy to .seed.tar before it unpacks. The branch keeps every release forever. So memory use and temporary disk use both grow with every version published, on every publish run.
Send git archive output straight to a file with stdout=, or pipe it into tarfile.open(mode="r|"). Then delete the tar as before.
- listing = subprocess.run(- ["git", "archive", f"{branch}:{published_prefix}"],- cwd=repo,- capture_output=True,- check=False,- )- if listing.returncode != 0:+ tar = pathlib.Path(site_dir) / ".seed.tar"+ with tar.open("wb") as out:+ listing = subprocess.run(+ ["git", "archive", f"{branch}:{published_prefix}"],+ cwd=repo,+ stdout=out,+ stderr=subprocess.PIPE,+ check=False,+ )+ if listing.returncode != 0:+ tar.unlink(missing_ok=True)
raise SystemExit(
@@
- tar = pathlib.Path(site_dir) / ".seed.tar"- tar.write_bytes(listing.stdout)
shutil.unpack_archive(str(tar), str(site_dir), format="tar")
tar.unlink()
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.
#!/bin/bashprintf'%s\n''--- published root directories ---'
git ls-tree -d --name-only FETCH_HEAD:docs/
printf'%s\n''--- published legacy page metadata ---'
git show FETCH_HEAD:docs/unstable/python/index.html | grep -n -E 'theme_switcher_json_url|theme_switcher_version_match|show_version_warning_banner|false;'printf'%s\n''--- workflow assembly and publish ---'
cat -n .github/workflows/build-site.yml | sed -n '113,158p'printf'%s\n''--- root project sources ---'
find docs/site -maxdepth 2 -type f | sort
Repository: NVIDIA/cccl
Length of output: 3475
suggestion: Retire the legacy development URLs
Seeding retains the old unstable/ tree, but the new builds update cpp/unstable/ and python/unstable/ instead. Old links can therefore serve frozen development documentation without a warning.
Redirect the legacy URLs to the appropriate new pages, or otherwise retire them.
Tools
ast-grep (0.51.1)
[error] 91-97: Command coming from incoming request
Context: subprocess.run(
["git", "rev-parse", "--verify", "--quiet", f"{branch}^{{commit}}"],
cwd=repo,
capture_output=True,
text=True,
check=False,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
(subprocess-from-request)
[error] 102-107: Command coming from incoming request
Context: subprocess.run(
["git", "archive", f"{branch}:{published_prefix}"],
cwd=repo,
capture_output=True,
check=False,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').
The reason will be displayed to describe this comment to others. Learn more.
Agreed. It'd be good to wipe gh-pages before this merges, so the first deploy publishes only the new layout and nothing from the old unstable/ tree survives.
The reason will be displayed to describe this comment to others. Learn more.
@sri-koundinyan important: Before wiping gh-pages, back it up and confirm that removing all published versions and breaking legacy links is acceptable. Coordinate the wipe with the first deploy to limit downtime. After deployment, verify that the legacy unstable/ tree is absent.
The reason will be displayed to describe this comment to others. Learn more.
A few points before I look deeper into the PR:
The writing in the comments and source files is very AI and quite difficult to read. Can you please instruct your agent to take a pass over it and make it less AI?
I'm a little confused over the structure. The PR description states that we are splitting C++ and Python docs builds into pieces (which is a good thing), but the directory structure here leaves docs/cccl, docs/cpp_site, docs/python, docs/python_site, docs/site (also _ext). Which is the python build and which is the C++ one? Naively I would have assumed there is docs/cpp and docs/python (and possibly docs/common).
The reason will be displayed to describe this comment to others. Learn more.
Do we really need tests for our docs builds? These seems like an AI-ism that has crept in.
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Jacobfaib
left a comment
Splits the documentation into independently versioned products, published as one site. Previously one Sphinx project built all of docs/, so Python's pages were published beneath the C++ version, and each deploy kept only the build that had just run.
Major design components:
Three Sphinx projects, assembled into one site:
A build carries exactly one version, so independently versioned products need separate builds. The root project is unversioned and holds the repository-level docs (contributors, maintainers, infrastructure), which move there from docs/.
docs/registry.yml lists the products. The directories each build skips, the cross-reference mappings, release-tag routing, the landing index and each product's build command all read it, so a new product is one registry entry plus its own build script and Sphinx project. Registry order is build order.
Cross-references between projects resolve through intersphinx: Python against C++, the root against both, and the C++ build maps nothing. A project maps only the products listed above it, since those are built first, and reads their inventories from the site being assembled (or this run's build output) before the published site, which is a publish behind.
Releases are driven by tags: the prefix selects the product (v for C++, python- for Python) and the remainder is the version. Only an exact MAJOR.MINOR.PATCH publishes; pre-release tags are refused. source-ref lets a tag cut before this change be built from another ref.
One deploy with force_orphan. The assembly seeds itself from the published branch before adding the version just built, so the artifact is the whole site and recreating the branch loses nothing. On a fork, one snapshot measured 10 MB while 56 deploys with force_orphan off accumulated 90 MB; seeded, the branch stays one commit whose size follows the content.
Each product's switcher manifest is generated from the published branch's directory listing, and check_manifests() fails the run if it disagrees with the assembled site.
Seeding is guarded: runs publishing one branch are serialized, and a failed fetch of the published branch stops the run instead of being treated as a first publication.
Pull-request CI runs the same assembly without seeding or publishing, so every pull request builds all three projects with warnings as errors, including the contributor and maintainer docs that a products-only build would stop checking.
Minor changes and comments:
cuda.coop stays in the Python product until cuda-coop versions separately. Its Sphinx settings moved to docs/python/conf.py, and its visualization extension resolves pages from the Python project's root.
Seven labels were defined both in a docs page and in a C++ header's Doxygen comment. Sphinx's parallel build reports a duplicate label only when both copies are read in the same chunk, so these surfaced or stayed hidden depending on the document set and CPU count; the C++-only build exposed one. None was referenced, so the header copies are removed.
Both workflows create the docs virtual environment before anything reads the registry, so nothing is installed into the runner's system Python; PyYAML is declared in docs/requirements.txt.
Removed: docs-deploy.yml; the custom 404 machinery (404.html, its two helpers and scrape_docs.bash); the hand-written docs/index.html chooser; and docs/cccl/index.rst, whose content moved into the C++ landing page.
The moved docs link into the C++ pages through cpp: cross-references, and paths and URLs in READMEs, AGENTS.md, .coderabbit.yaml, CI messages and iterator docstrings follow the new layout.
Unstable pages carry a development-documentation banner.
ci/project_files_and_dependencies.yaml ignores build-site.yml in place of docs-deploy.yml.
docs/test_docs_build.py covers the failures that would otherwise be silent: tag routing, a manifest disagreeing with the site, the inventory source, seeding before force_orphan, and per-version replacement. A pre-commit hook runs it.
Modelled on rapidsai/docs. Verified end to end on a fork: three C++ and two Python versions published, one commit on the branch, and every version reachable from the switcher.
Description
closes
Checklist