Support creating a venv for a foreign platform.

[jasonwbarnett]

Closes #3278.

A venv's layout is fixed by the Python implementation, version and OS; the
machine architecture plays no part. So a local interpreter can lay a venv out
for a foreign platform it differs from in architecture alone, leaving just the
venv Python to point at wherever the foreign interpreter will live.

The new pex3 venv create --link-python does that pointing. The path need not
exist when the venv is created, which is what makes the foreign case work: the
venv, its scripts and the foreign wheels all land locally and the Python link
only resolves once the venv reaches its final resting place. That is also
useful on its own for a venv built somewhere other than where it will run.

When the target is foreign, --link-python is required and a stand-in
interpreter is searched for on --python-path ($PATH by default), matching
the target's interpreter tag, ABI and platform_system. Nothing matching means
a fail fast naming what was needed. --compile now uses the venv's own Python
rather than the one running Pex, so a foreign venv gets bytecode for its own
version.

Built and verified on linux-aarch64 for linux_x86_64-cp-311-cp311:

bin/python -> /usr/local/bin/python3.11
home = /usr/local/bin
lib/python3.11/site-packages/psutil/_psutil_linux.abi3.so: ELF 64-bit LSB shared object, x86-64

N.B.: tests/integration/cli/commands/test_venv_create.py grew a cross-arch
case that leans on ensure_python_interpreter(PY310); it could not run where I
wrote it, since pyenv install 3.10.7 will not build on Ubuntu 24.04. Its
assertions were checked against a real foreign venv built with 3.11 instead.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JXoAkfaDfspdEqDzgHoTzG

[jsirois]

@jasonwbarnett can you allow maintainers to edit PR? I have scoping (no 2.7 support, no Windows support) and test (pyvenv.cfg executable is not a thing until 3.11) fixes I want to push:

:; git push altana-ai HEAD
remote: Permission to altana-ai/pex.git denied to jsirois.
fatal: unable to access 'https://github.com/altana-ai/pex/': The requested URL returned error: 403

Or, apply this please (also attached patch.zip):

From 76d1e7e9be96c61c474bc98473d7c45137a852dc Mon Sep 17 00:00:00 2001
From: John Sirois <john.sirois@gmail.com>
Date: Tue, 15 Sep 2026 12:07:03 -0700
Subject: [PATCH] Fix scope (modern Unix venvs) & tests.

---
 pex/cli/commands/venv.py                      |  17 ++-
 pex/sysconfig.py                              |  10 +-
 .../cli/commands/test_venv_create.py          | 113 +++++++++++++-----
 3 files changed, 105 insertions(+), 35 deletions(-)

diff --git a/pex/cli/commands/venv.py b/pex/cli/commands/venv.py
index a794e873..8bb0785b 100644
--- a/pex/cli/commands/venv.py
+++ b/pex/cli/commands/venv.py
@@ -20,6 +20,7 @@ from pex.executor import Executor
 from pex.fingerprinted_distribution import FingerprintedDistribution
 from pex.interpreter import PythonInterpreter
 from pex.orderedset import OrderedSet
+from pex.os import Os
 from pex.pex import PEX
 from pex.pex_bootstrapper import normalize_path
 from pex.pex_info import PexInfo
@@ -292,12 +293,26 @@ class Venv(OutputMixin, JsonMixin, BuildTimeCommand):
             if layout is InstallLayout.VENV:
                 venv_interpreter = target.get_interpreter()
                 if target.is_foreign:
-                    if not self.options.link_python:
+                    if Os.CURRENT is Os.WINDOWS:
+                        return Error(
+                            "Cannot create a local venv for foreign platform {platform}.\n"
+                            "This is only supported for Unix environments.".format(
+                                platform=target.platform
+                            )
+                        )
+                    elif not self.options.link_python:
                         return Error(
                             "Cannot create a local venv for foreign platform {platform}.\n"
                             "Specify --link-python to say where the venv's Python will live in "
                             "the foreign environment.".format(platform=target.platform)
                         )
+                    elif not target.python_version or target.python_version < (3, 3):
+                        return Error(
+                            "Cannot create a local venv for foreign platform {platform}.\n"
+                            "This is only supported for Python 3.3 and newer.".format(
+                                platform=target.platform
+                            )
+                        )
                     venv_interpreter = try_(_find_stand_in_interpreter(target, self.options))

                 venv = Virtualenv.create(
diff --git a/pex/sysconfig.py b/pex/sysconfig.py
index d21257dc..880edf5f 100644
--- a/pex/sysconfig.py
+++ b/pex/sysconfig.py
@@ -17,7 +17,7 @@ from pex.os import Os
 from pex.typing import TYPE_CHECKING

 if TYPE_CHECKING:
-    from typing import Optional, Text, TypeVar
+    from typing import Optional, Text, Tuple, TypeVar, Union

 EXE_EXTENSION = get_config_var("EXE") or ""
 EXE_EXTENSIONS = (
@@ -159,6 +159,14 @@ class _PlatformValue(Enum.Value):
         # type: () -> str
         return "Scripts" if self.os is Os.WINDOWS else "bin"

+    def venv_lib_dir(self, version):
+        # type: (Union[Tuple[int, int], Tuple[int, int, int]]) -> str
+        return (
+            "Lib"
+            if self.os is Os.WINDOWS
+            else "lib/python{major}.{minor}".format(major=version[0], minor=version[1])
+        )
+
     def binary_name(self, binary_name):
         # type: (_Text) -> _Text
         return "{binary_name}{extension}".format(
diff --git a/tests/integration/cli/commands/test_venv_create.py b/tests/integration/cli/commands/test_venv_create.py
index c142ab16..f515893c 100644
--- a/tests/integration/cli/commands/test_venv_create.py
+++ b/tests/integration/cli/commands/test_venv_create.py
@@ -5,6 +5,7 @@ from __future__ import absolute_import

 import glob
 import os.path
+import re
 import shutil
 import sys
 from subprocess import CalledProcessError
@@ -24,14 +25,17 @@ from pex.pep_440 import Version
 from pex.pep_503 import ProjectName
 from pex.pex import PEX
 from pex.resolve import abbreviated_platforms
+from pex.sysconfig import SysPlatform
 from pex.typing import TYPE_CHECKING
 from pex.venv.bin_path import BinPath
 from pex.venv.virtualenv import Virtualenv
 from testing import (
     IS_ARM_64,
     IS_MAC,
+    IS_WINDOWS,
     PY39,
     PY310,
+    IntegResults,
     ensure_python_interpreter,
     make_env,
     run_pex_command,
@@ -545,51 +549,94 @@ def test_foreign_target(
     assert Version("5.9.5") == dist.metadata.version


-@pytest.fixture
-def cross_arch_platform():
-    # type: () -> str
+def cross_arch_platform(
+    major,  # type: int
+    minor,  # type: int
+    abiflags="",  # type: str
+):
+    # type: (...) -> str
     """A foreign platform differing from the local one in machine architecture alone."""
+    pyver = "{major}{minor}".format(major=major, minor=minor)
     if IS_MAC:
-        return "macosx_11_0_{machine}-cp-310-cp310".format(
-            machine="x86_64" if IS_ARM_64 else "arm64"
+        return "macosx_11_0_{machine}-cp-{pyver}-cp{pyver}{abiflags}".format(
+            machine="x86_64" if IS_ARM_64 else "arm64", pyver=pyver, abiflags=abiflags
         )
-    return "linux_{machine}-cp-310-cp310".format(machine="x86_64" if IS_ARM_64 else "aarch64")
+    return "linux_{machine}-cp-{pyver}-cp{pyver}{abiflags}".format(
+        machine="x86_64" if IS_ARM_64 else "aarch64", pyver=pyver, abiflags=abiflags
+    )


+@pytest.mark.skipif(IS_WINDOWS, reason="This feature is not supported on Windows.")
 def test_foreign_target_link_python(
-    tmpdir,  # type: Any
-    cross_arch_platform,  # type: str
+    tmpdir,  # type: Tempdir
+    py310,  # type: PythonInterpreter
+    py311,  # type: PythonInterpreter
 ):
     # type: (...) -> None

-    dest = os.path.join(str(tmpdir), "dest")
-    link_python = "/opt/python/bin/python3.10"
-    run_pex3(
-        "venv",
-        "create",
-        "psutil==5.9.5",
-        "-d",
-        dest,
-        "--platform",
-        cross_arch_platform,
-        "--python-path",
-        ensure_python_interpreter(PY310),
-        "--link-python",
-        link_python,
-    ).assert_success()
+    venv_dir = tmpdir.join("venv")
+    link_python = "/opt/python/bin/bob"

-    assert {link_python} == {
-        os.readlink(python) for python in glob.glob(os.path.join(dest, "bin", "python*"))
-    }
+    def create_venv(foreign_platform):
+        # type: (str) -> IntegResults
+        return run_pex3(
+            "venv",
+            "create",
+            "psutil==5.9.5",
+            "-d",
+            venv_dir,
+            "--platform",
+            foreign_platform,
+            "--python-path",
+            os.pathsep.join(interpreter.binary for interpreter in (py310, py311)),
+            "--link-python",
+            link_python,
+            "--force",
+        )

-    pyvenv_cfg = PyVenvCfg.parse(os.path.join(dest, "pyvenv.cfg"))
-    assert os.path.dirname(link_python) == pyvenv_cfg.home
-    assert link_python == pyvenv_cfg.config("executable")
+    def assert_foreign_venv(
+        major,  # type: int
+        minor,  # type: int
+        expect_pyvenv_cfg_executable,  # type: Optional[str]
+    ):
+        # type: (...) -> None
+        assert {link_python} == {
+            os.readlink(python) for python in glob.glob(os.path.join(venv_dir, "bin", "python*"))
+        }
+
+        pyvenv_cfg = PyVenvCfg.parse(os.path.join(venv_dir, "pyvenv.cfg"))
+        assert os.path.dirname(link_python) == pyvenv_cfg.home
+        pyvenv_cfg_executable = pyvenv_cfg.config("executable")
+        if expect_pyvenv_cfg_executable:
+            assert pyvenv_cfg_executable == expect_pyvenv_cfg_executable
+        else:
+            assert pyvenv_cfg_executable is None
+
+        site_packages = os.path.join(
+            venv_dir, SysPlatform.CURRENT.venv_lib_dir(version=(major, minor)), "site-packages"
+        )
+        distributions = list(dist_metadata.find_distributions(search_path=[site_packages]))
+        assert 1 == len(distributions)
+        assert ProjectName("psutil") == distributions[0].metadata.project_name
+
+    foreign_platform = cross_arch_platform(2, 7, "mu")
+    create_venv(foreign_platform=cross_arch_platform(2, 7, "mu")).assert_failure(
+        expected_error_re=r".*{msg}$".format(
+            msg=re.escape(
+                "Cannot create a local venv for foreign platform {foreign_platform}.\n"
+                "This is only supported for Python 3.3 and newer.".format(
+                    foreign_platform=foreign_platform
+                )
+            )
+        ),
+        re_flags=re.DOTALL,
+    )

-    site_packages = os.path.join(dest, "lib", "python3.10", "site-packages")
-    distributions = list(dist_metadata.find_distributions(search_path=[site_packages]))
-    assert 1 == len(distributions)
-    assert ProjectName("psutil") == distributions[0].metadata.project_name
+    create_venv(foreign_platform=cross_arch_platform(3, 10)).assert_success()
+    assert_foreign_venv(3, 10, expect_pyvenv_cfg_executable=None)
+
+    create_venv(foreign_platform=cross_arch_platform(3, 11)).assert_success()
+    assert_foreign_venv(3, 11, expect_pyvenv_cfg_executable=link_python)


 def test_venv_update_target_mismatch(
--
2.55.0

[jsirois]

Not sure what's up with GH. I may merge from the command line.

[jsirois]

Merged the old fashioned way: e06795b