Extending the reusable test-suite workflow in heavily-customized downstreams (e.g. setuptools) #211

Background

#199 factored the shared test/collateral setup into a reusable test-suite.yml (reusable workflows having been chosen over composite actions per the discussion in #162 / #163). For most consumers this is a clear win: main.yml shrinks to thin uses: calls, and the shared steps + env live in one place.

For heavily-customized downstreams it's a harder fit. setuptools is the motivating case. Its test job interleaves a lot of project-specific work with the shared steps:

• pre-building the sdist/wheel and exporting PRE_BUILT_SETUPTOOLS_*
• a cached download of setupcfg examples
• SETUPTOOLS_USE_DISTUTILS and a pypy continue-on-error
• publishing coverage

Plus two jobs that can't use the reusable workflow at all: test_cygwin (every step runs under a Cygwin bash shell) and integration (separate, tag-gated).

The friction

1. You can't inject steps into a reusable workflow. A calling job may only set uses/with/secrets/needs/if/strategy/permissions — never steps. So setuptools can't "call the shared workflow and add its pre-build steps"; the custom steps have nowhere to live in the caller.

2. So the only way to keep the pattern is to fork test-suite.yml and add the project steps there, gated by an input (setuptools uses a full-test flag). But that customizes the skeleton's own shared file — the one the skeleton will keep iterating on — so every future skeleton update to test-suite.yml now conflicts. It inverts the usual "customize main.yml, inherit test-suite.yml" contract.

3. Env can't be shared with decoupled jobs. A called reusable workflow doesn't inherit the caller's top-level env:, and test_cygwin/integration can't call it, so the shared env block (FORCE_COLOR, PIP_*, TOX_OVERRIDE) ends up duplicated across test-suite.yml and each decoupled job.

setuptools' current resolution (in the pending skeleton merge) is Option-D-style: a forked test-suite.yml with a full-test input gating the setuptools-specific steps, and test_cygwin/integration decoupled with duplicated env. It works, but it accepts (2) and (3).

Question

Could the skeleton offer extension points so a downstream can customize without forking test-suite.yml? A couple of directions:

• Convention hooks — test-suite.yml runs a downstream-provided script at a well-known path if it exists (e.g. a pre-tox ./.github/pre-test and/or a post-tox hook). Degenerate/no-op by default; setuptools would move its pre-build / download-cache logic into such a script, keeping test-suite.yml upstream-pristine.
• Composite actions for the genuinely shared concerns (setup-python + tox; publish-coverage) that both test-suite.yml and decoupled jobs like test_cygwin could call — which would also address the env/coverage duplication in (3). (Acknowledging the composite-action tradeoffs already raised in tests and collateral share some behavior implicitly #162 — collapsed/ugly log output in particular.)

Curious whether hooks, composites, or "downstreams just fork test-suite.yml and own it" is the intended story for cases like setuptools.