Skip to content

[infra] Guard uv in e2e.sh and fix the pytest log level in ut.sh - #986

Merged
wenjin272 merged 1 commit into
apache:mainfrom
weiqingy:982-build-script-fallbacks
Aug 10, 2026
Merged

wenjin272 merged 1 commit into
apache:mainfrom
weiqingy:982-build-script-fallbacks

Conversation

@weiqingy

@weiqingy weiqingy commented Aug 8, 2026 •

Copy link
Copy Markdown
Collaborator

Linked issue: #982

Purpose of change

Two defects in fallback paths that CI never exercises, which is why neither has surfaced before. Both were found while working on #979.

tools/e2e.sh breaks without uv on PATH

It invokes uv by bare name seven times and has no command -v uv guard anywhere, so a machine without uv on PATH gets seven uv: command not found lines and no hint about what to install.

It is the odd one out among the build scripts. tools/lint.sh and tools/ut.sh both guard their uv usage and fall back to pip, and tools/build.sh installs uv before using it. tools/e2e.sh does neither. It only delegates to tools/build.sh conditionally, and that delegation is skipped once e2e-test/target and python/uv.lock exist.

So it now probes once, after the build delegation so that a build performed in the same run counts:

if command -v uv >/dev/null 2>&1; then
  UV=(uv)
elif python3 -m uv --version >/dev/null 2>&1; then
  UV=(python3 -m uv)
else
  echo "Error: uv not found. Run tools/build.sh first, or install uv." >&2
  exit 1
fi

The binary on PATH has to win over the module. A standalone uv, from the curl installer or homebrew, gives a working binary and no importable uv module, so preferring the module would break exactly those installs.

The module branch tests uv by running it rather than by importing it. uv/__init__.py imports find_uv_bin without calling it, and the five-directory search for the binary happens at call time inside uv/__main__.py. So import uv succeeds even when the binary is missing, and a probe built on it would select a module that then fails at all seven sites, relocating the very failure this change removes.

The uv pip install site also names its target environment. uv pip honors the VIRTUAL_ENV that uv's module entry point derives from the calling interpreter, so without --python .venv the fallback could install apache-flink outside the venv the tests actually run in. tools/build.sh already does the same at its own wheel install.

tools/ut.sh passes a log level pytest rejects

The pip fallback passed -o log_cli_level=${LOG_LEVEL:-OFF} while the uv branch in the same function passed ${LOG_LEVEL:-CRITICAL}. OFF is not a Python logging level name, so pytest rejects it during startup:

ERROR: 'OFF' is not recognized as a logging level name for 'log_cli_level'. Please consider passing the logging level num instead.

That is exit 4, raised before any test is collected, which ut.sh then reports through its "unknown error" arm. Aligning the fallback with the uv branch lets the path run.

Tests

CI behavior is unchanged. LOG_LEVEL: INFO is set at four workflow sites, and the two Python jobs that fall back to the default take the uv branch, which already used CRITICAL. The runners also provide uv on PATH, so the probe selects the same binary the script used before.

The probe was exercised across every combination of binary and module availability:

uv on PATH module importable binary findable Selected Exit
yes yes yes uv 0
yes no n/a uv 0
no yes yes python3 -m uv 0
no no n/a error 1
no yes no error 1

The last row is the case that motivated testing the module by running it. With an import-based probe it selected python3 -m uv and exited 0, then failed at the first real call site.

Argument passing was checked at all seven sites against the previous version with a stub on PATH, including the one inside a command substitution and the ones in cd ... && chains. Byte-identical under both probe branches.

For tools/ut.sh, the fallback path previously exited 4 with nothing collected. It now collects 375 tests and 293 pass locally. The rest are apache-flink~=2.3.0 being uninstallable on macOS arm64, since no apache-beam wheel at the required version exists for that platform. That is environmental and unrelated to this change, so this fixes the first blocker on that path rather than making it green everywhere.

bash -n passes on both scripts, and shellcheck reports the same finding set before and after, with nothing landing on the new block.

API

No API change.

Documentation

  • doc-needed
  • doc-not-needed
  • doc-included

Was this patch authored or co-authored using generative AI tooling?

  • Yes
  • No

@github-actions github-actions Bot added doc-not-needed Your PR changes do not impact docs fixVersion/0.4.0 priority/major Default priority of the PR or issue. labels Aug 8, 2026

@wenjin272 wenjin272 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on. LGTM. Merging after resolving conflicts.

Two defects in fallback paths that CI never exercises.

tools/e2e.sh invoked uv by bare name seven times with no guard anywhere,
so a machine without uv on PATH got seven "command not found" lines and
no indication of what to install. Unlike the sibling scripts it neither
guards its uv usage the way lint.sh and ut.sh do, nor installs uv the
way build.sh does. It only delegates to build.sh conditionally, and that
delegation is skipped once e2e-test/target and uv.lock exist.

Probe once instead, after the build delegation so that a build performed
in this run counts, and prefer the binary on PATH. A standalone uv (curl
installer, homebrew) provides a working binary and no importable module,
so the PATH form has to win; the module is the fallback, and neither
available is now a clear error rather than seven obscure ones. The probe
tests the module by running it rather than by importing it, because
uv/__init__.py imports find_uv_bin without calling it, so an import
succeeds even when the binary it locates is absent.

The pip install site also names its target environment. uv pip honors
the VIRTUAL_ENV that uv's module entry point derives from the calling
interpreter, so without --python .venv the fallback could install
apache-flink outside the venv the tests actually run in. build.sh
already does the same at its own wheel install.

tools/ut.sh passed -o log_cli_level=OFF in its pip fallback while the uv
branch passed CRITICAL. OFF is not a Python logging level name, so
pytest rejected it and exited 4 during startup, before collecting
anything. Aligning the two branches lets that path run: it now collects
375 tests where it previously collected none.

Generated-by: Claude Code 2.1.226
@weiqingy
weiqingy force-pushed the 982-build-script-fallbacks branch from 6fef839 to dae8a7c Compare August 9, 2026 05:52
@github-actions github-actions Bot added doc-not-needed Your PR changes do not impact docs and removed doc-not-needed Your PR changes do not impact docs labels Aug 9, 2026
@weiqingy

weiqingy commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator Author

@wenjin272 Thanks for the review. Rebased onto main now that #979 is in.

The conflict was the squash: #979's two commits landed as one commit with a new SHA, so this branch and main were changing the same three files from different ancestries. I dropped those two commits with git rebase --onto origin/main 16f01719 and confirmed first that the squashed commit is byte-identical to them, so nothing was lost in the drop.

The PR is down to the single new commit, touching only tools/e2e.sh and tools/ut.sh. No source change from what you reviewed. I also dropped the "stacked on #979" note from the description since it no longer applies.

@wenjin272
wenjin272 merged commit 0334f43 into apache:main Aug 10, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs fixVersion/0.4.0 priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants