[BUG] CAGRA hang on h100 oldest-deps CI (cuvs test_cagra, then cuvs_bench test_run_command_creates_results) #2772

tests/test_cli.py::test_run_command_creates_results in cuvs_bench failed in 4 pr.yaml runs since 2026-10-06, all on the 12.2.2, 3.11, amd64, rockylinux8, h100, earliest-driver, oldest-deps matrix entry. test_plot_command_creates_png_files fails in the same jobs because it reads the results the run test did not produce.

The Aborted! message is not a crash. It is click's response to SIGINT: ci/test_python.sh runs pytest under timeout 40m --signal=SIGINT, and the benchmark was hung until timeout fired.

In all 4 jobs:

1. The pytest cuvs step hits its 40 minute timeout and is SIGKILLed. It stalls inside tests/test_cagra.py at the same point (after roughly 52 tests, likely in test_cagra_dataset_dtype_host_device).
2. The pytest cuvs-bench step then starts, and CUVS_CAGRA_HNSWLIB_ANN_BENCH hangs about 3 seconds in, right after "We have sufficient memory to proceed with in memory build", until its own 40 minute timeout.

A passing run of the same job finishes the cuvs suite in 11 minutes and the bench suite in 72 seconds, with the same driver (580.178.04), CUDA 12.2 and rmm version, so the hang is intermittent. Two of the failing commits already contain #2729. The root cause is not yet identified; the logs do not show where the process is stuck.

Failing jobs:

A fifth job, https://github.com/NVIDIA/cuvs/actions/runs/34719781952/job/104185737619 (PR #2600, CUDA 13.3.0, rtxpro6000), failed the same two tests in 5.8 seconds with symbol lookup error on the bench binaries. That is a separate build/ABI problem, not this hang.

Method: failed pr.yaml runs created on or after 2026-09-08 (230 of 555), then failed python-tests jobs, then FAILED lines in their logs. The count may be incomplete if some job listings could not be fetched.