Skip to content

build: ground sub-package packaging in current uv/hatch/setuptools-scm guidance - #4407

Merged
d-v-b merged 8 commits into
zarr-developers:mainfrom
d-v-b:chore/subpackage-packaging-practices
Sep 25, 2026
Merged

d-v-b merged 8 commits into
zarr-developers:mainfrom
d-v-b:chore/subpackage-packaging-practices

Conversation

@d-v-b

@d-v-b d-v-b commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

this is a refresh of our subpackaging structure. The biggest fact on display here is that we don't want to use hatch / uv monorepo features (yet) because our projects don't have the exact same python support range.

🤖 AI text below 🤖

Depends on #4406 and includes its commits; merge that first, after which this diff shrinks to the last five commits. (GitHub can't natively stack PRs whose head branches live on a fork.)

Summary

I checked our monorepo packaging against the current docs for uv (0.12.18), Hatch (1.18.1) / hatchling (1.32.4), setuptools-scm (10.3.4), hatch-vcs (0.5.0), and the PyPA sdist spec. This PR fixes the three gaps that were worth fixing.

1. Test each sdist on its own before release (ci:)

hatch build builds the wheel from the source tree, so nothing ever used the sdist. The first version of #4229 produced sdists that pip refuses to unpack (a symlink pointing outside the archive), and the release workflows would have uploaded them. Each release workflow's test_artifacts job now:

  • builds a wheel from the sdist with pip (pip wheel), which unpacks under the sdist format rules: no links out of the archive;
  • runs just test and just docs-check from the unpacked sdist, with no repo checkout.

The job has no checkout, so zarr-http-server resolves zarr from PyPI there. That also covers the zarr>=3.1 floor, which repo runs never check: uv run --with-editable versions are "respected regardless of the project's requirements".

2. Replace deprecated git_describe_command (build:)

setuptools-scm deprecates git_describe_command in favour of scm.git.describe_command. The DeprecationWarning is swallowed during builds. This moves the root and all three sub-packages to the new key and adds setuptools-scm>=9 to build-system.requires. hatch-vcs passes raw-options to setuptools_scm.get_version(), which only accepts scm from 9.0; 8.4, the version the docs name, was never released.

  • tag.prefix is setuptools-scm's documented option for monorepos, but hatch-vcs 0.5.0 can't reach it (get_version() takes no tag argument).
  • The remaining tag_regex deprecation warning comes from hatch-vcs's own translation of tag-pattern, so we can't fix it from our config.
  • ⚠ Anyone building zarr from source now needs setuptools-scm>=9; e.g. the conda-forge recipe's host requirements.

3. Commit zarr-metadata's uv.lock

.gitignore excluded it ("a library, not an app"). uv's docs say uv.lock "should be checked into version control", with no exception for libraries, and the other two sub-packages already commit theirs.

Checked, no change needed

Verification

Check Result
hatch version for root / zarr-metadata / zarr-indexing / zarr-http-server, before vs after on main (37706fe): 3.4.1.dev27, 0.5.1.dev79, 0.2.2.dev90, 0.1.1.dev94; on this branch, 6 commits ahead: dev33, dev85, dev96, dev100, exactly +6 each
setuptools-scm 8.2.0 / 8.3.1 with scm= TypeError, hence >=9
new pip step on the old broken #4229 sdist fails: "would link to … outside the destination"
new pip step on sdists from this branch (hatch 1.16.5, as in CI) all three build
just test from unpacked sdists metadata 595 passed; indexing 1547 passed / 207 skipped (no zarr); http-server 212 passed / 9 skipped
just docs-check from unpacked sdists all three build under --strict
zizmor, GitHub workflow schema clean

I haven't run the release workflows: on the fork, workflow_dispatch would also trigger the TestPyPI upload job.

The zarr-metadata lock is generated against the docs pins #4229 aligned.

🤖 Generated with Claude Code

…rr from just

The packages reached in-repo zarr two different ways: zarr-http-server
pinned it with `[tool.uv.sources] zarr = { path = "../.." }` in
pyproject.toml, and zarr-indexing ran its tests in the repo root's
environment (`uv run --project ../..`). Both reach outside the package, so
neither package's unpacked sdist could run its own `just test`.

Now every package's pyproject.toml stands alone, and the justfiles add
the in-repo zarr as an editable overlay (`--with-editable ../..`) only
when ../.. is the repository. From an sdist there is no overlay:
zarr-http-server resolves zarr from PyPI, and zarr-indexing's zarr suites
skip. zarr-metadata needs no zarr and is unchanged.

zarr-http-server's uv.lock now resolves zarr from PyPI, and zarr-indexing
CI syncs the package environment instead of the root one.

Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
…uv.lock

setuptools-scm deprecates the top-level `git_describe_command` in favour of
`scm.git.describe_command` (a DeprecationWarning builds swallow). Move the
root and all three sub-packages to the new key; hatch-vcs passes raw-options
to setuptools_scm.get_version, which accepts `scm` from setuptools-scm 9.0
(8.4, which the docs name, was never released), so require
setuptools-scm>=9 at build time. Computed versions are unchanged.

`tag.prefix`, setuptools-scm's documented monorepo option, is not reachable
through hatch-vcs 0.5.0: get_version() takes no `tag` argument.

uv's docs say uv.lock "should be checked into version control", with no
library exception, and the other two sub-packages already commit theirs.
Stop ignoring zarr-metadata's.

Assisted-by: ClaudeCode:claude-opus-5-5
`hatch build` builds the wheel from the source tree, so nothing exercised
the sdist: zarr-developers#4229's first cut shipped sdists whose symlinks pip refuses to
unpack, and the release workflows would have uploaded them. The
test_artifacts job now builds a wheel from the sdist with pip, then runs
the package's `just test` and `just docs-check` from the unpacked sdist.
The job has no checkout, so zarr (for zarr-http-server) resolves from
PyPI, which also covers the `zarr>=3.1` floor that repo runs, overlaying
in-repo zarr, never check.

Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
@d-v-b
d-v-b merged commit f58644a into zarr-developers:main Sep 25, 2026
67 of 71 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant