Repository navigation
build(dynacell): bump cubic to 0.9.0a2, keep 0.9.0a1 metric caches valid - #503
Merged
Merged
Conversation
cubic 0.9.0a2 chunks the MicroSSIM RI-factor objective and pools the per-slice SSIM elements into preallocated buffers, dropping the fit's peak from 40.5 to 12.9 GB on 576x640x960 float32. Nothing else in the release touches numeric code (git diff v0.9.0a1..v0.9.0a2 changes only cubic/metrics/microssim/ri_factor.py plus its tests). The uv.lock change is the cubic entry and its two requires-dist refs only; `uv lock --check` passes on it. REQUIRED_CUBIC_VERSION and the run_eval_direct.slurm default venv (cpdino-eval-cubic090a2, same freeze as cpdino-eval-cubic090a1 except cubic) move with the pin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015kuht3npKtb2o5E6h19xbA
….9.0a2 metrics_provenance_matches refuses any cache stamped with another cubic, so the 0.9.0a2 pin alone would force every leaf to recompute. The bump does not move a value: scoring one FOV x 7 timepoints of the A549-trained fnet3d_paper ER a549__denv predictions under both eval venvs gives bit-identical pixel_metrics and mask_metrics columns (MicroMS3IM included) and the same fitted alpha, 20.037598609924316. CP feature columns were not scored in that run; the release diff touches no CP code. CUBIC_VERSIONS_EQUIVALENT_TO_REQUIRED lists the prior versions the declared pin reproduces. They match only while the declared pin is installed, and the CP-space binding check is unchanged. 0.8.0a2 caches are still refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015kuht3npKtb2o5E6h19xbA
The test inlined the same write/load/overwrite/save sequence the new helper provides. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015kuht3npKtb2o5E6h19xbA
Contributor
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The dependency and cache-provenance changes are consistent and tested; only minor documentation corrections remain.
Review effort: Balanced
Findings: 2
Open (2)
What changed in this PR
Updates Dynacell’s evaluation stack to cubic 0.9.0a2 while retaining compatible 0.9.0a1 metric caches.
Changes:
- Bumps cubic pins, lockfile entry, and evaluation environment.
- Adds guarded cache compatibility for cubic 0.9.0a1.
- Tests and documents the updated provenance contract.
| File | Description |
|---|---|
uv.lock |
Locks cubic 0.9.0a2. |
applications/dynacell/tools/run_eval_direct.slurm |
Uses the new evaluation environment. |
applications/dynacell/src/dynacell/evaluation/provenance.py |
Adds version-equivalent cache handling. |
applications/dynacell/src/dynacell/evaluation/provenance_test.py |
Tests compatibility and rejection rules. |
applications/dynacell/src/dynacell/evaluation/CLAUDE.md |
Updates evaluation guidance. |
applications/dynacell/pyproject.toml |
Pins cubic 0.9.0a2. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…version A flat set of equivalent versions would keep accepting 0.9.0a1 caches after the next pin bump, although no one measured 0.9.0a1 against the new version. CUBIC_VERSIONS_EQUIVALENT_TO maps each declared version to the versions measured to reproduce it, and the gate looks up REQUIRED_CUBIC_VERSION, so a bump starts with no equivalents. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015kuht3npKtb2o5E6h19xbA
Reusing 0.9.0a1 caches under 0.9.0a2 keeps whatever they hold. Where the 0.9.0a1 MicroSSIM calibration ran out of memory, the cache holds MicroMS3IM=NaN for every FOV, and 0.9.0a2's bounded fit would have scored it. The scan lists such caches so they can be recomputed. Mito A549 mock, which is all-NaN for an unrelated data_range reason, is reported as known. Run on /hpc/projects/virtual_staining/training/dynacell on 2026-09-28: 926 sidecars, 35 all-NaN stamped 0.9.0a1, all mito A549 mock, 0 unexplained. Read-only; the eval gate is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015kuht3npKtb2o5E6h19xbA
The first measurement fitted alpha on 4 calibration pairs. Production fits 12 (576 slices), and 0.9.0a2 also changes the objective's accumulation from float32 to chunked float64, so bit-identity at 4 pairs did not carry over. Re-measured on the A549-trained fnet3d_paper ER a549__denv leaf with max_pairs=12, seed 42: alpha is 17.883591651916504 under both versions and MicroMS3IM over all 108 (FOV, t) differs by 0. The objective itself moves ~7e-8 relative, which the root finder absorbs. The equivalence is now stated as within a tolerance (MicroMS3IM relative delta <= 1e-6) instead of bit-identical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015kuht3npKtb2o5E6h19xbA
"Only bounds memory" undersold the release. 0.9.0a2 also accumulates the RI-factor objective in float64 instead of float32 and raises ValueError on an empty pool. The pyproject note that a pin bump "is paired with a full re-eval" contradicted this bump, which instead reuses caches through a measured equivalence entry. It now names both routes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015kuht3npKtb2o5E6h19xbA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Moves the dynacell eval stack from cubic 0.9.0a1 to 0.9.0a2. Existing 0.9.0a1 metric caches stay valid.
uv.lock(the cubic entry only;uv lock --checkpasses),REQUIRED_CUBIC_VERSION, and therun_eval_direct.slurmdefault venv, which is nowVisCy-venvs/cpdino-eval-cubic090a2.CUBIC_VERSIONS_EQUIVALENT_TO = {"0.9.0a2": frozenset({"0.9.0a1"})}. It is keyed by the declared version, so the next bump starts with no equivalents.metrics_provenance_matchesaccepts caches stamped with the versions listed forREQUIRED_CUBIC_VERSION, but only while that pin is installed. The CP-space binding rules are unchanged, and 0.8.0a2 caches are still refused.tools/scan_nan_microssim_caches.py, a read-only scan. See the caveat below.Release scope
git diff v0.9.0a1..v0.9.0a2touchescubic/__init__.py(the version),cubic/metrics/microssim/ri_factor.pyand that file's tests. Nothing else changes. The RI-factor fit now reduces its objective in bounded chunks, accumulating in float64 where 0.9.0a1 took a float32.mean(), and pools the per-slice elements into preallocated buffers. Peak memory drops from 40.5 to 12.9 GB.Two edge cases now raise
ValueError:get_global_ri_factoron 0 slices andget_ri_factoron empty elements. In dynacell,_calibrate_microssimreturns early on 0 positions or pairs. The one remaining empty case (D=0) already raisedValueErrorunder 0.9.0a1 (np.concatenate([])), and the caller catchesValueError, so behavior is unchanged.Equivalence (measured)
cpdino-eval-cubic090a2was built from thecpdino-eval-cubic090a1uv pip freezewith--no-deps, changing only the cubic line.uv pip freezediff: cubic only.Acceptance tolerance: MicroMS3IM relative delta ≤ 1e-6, far below the tables' 2-decimal rounding.
MicroMS3IM at production calibration settings. Leaf: A549-trained
fnet3d_paperERa549__denv, 12 FOVs. Calibration used the defaultmax_pairs=12and seed 42, giving a 576×640×960 float32 pool. MicroMS3IM was then scored on every (FOV, t). No torch model was loaded; the run was on an A40.The fit objective itself does change: on a fixed 640×960 float32 pair it moves about 7e-8 relative between the versions (
S(α)0.97342079878 vs 0.97342073207). The root finder lands on the same alpha anyway, so the comparison can detect a difference in the objective even though the result is identical.Other columns. A full eval of 1 FOV × 7 timepoints of the same leaf ran with
max_pairs=4. It produced bit-identicalpixel_metrics(PCC, SSIM, NRMSE, PSNR, SI_*, Spectral_PCC, XY/Z_FSC) andmask_metricscolumns. These columns do not call the RI-factor code. CP feature columns were not scored in that run; the release diff contains no CP code.Caveat: all-NaN MicroMS3IM in old caches
Where the 0.9.0a1 calibration ran out of memory (pre-#502 code), a cache holds
MicroMS3IM = NaNfor every FOV, and 0.9.0a2 would have produced a value. Reusing such a cache keeps the NaN.tools/scan_nan_microssim_caches.py --root /hpc/projects/virtual_staining/training/dynacelllists them.Scan after the re-eval jobs 36590711–18 finished (2026-09-29 09:36 PDT, rc=0): 930 sidecars, all stamped 0.9.0a1. 35 are all-NaN, and 0 are unexplained. All 35 are the mito A549 mock
data_rangecase, which predates this bump:mito/<model>/<train>/a549__mockdirs: 11 models × {a549, ipsc, joint} train sets, exceptcelldiff_r2(a549 and joint) andcelldiff_r2_iterative(ipsc only);a549/evaluations_with_embeddings/eval_{phase,vscyto3d_randinit}_mitochondria_mockdirs.No cache holds an OOM-caused NaN. The gate itself gets no special case.
Tests
Suite:
provenance_test.py,tests/test_evaluation_*.py,test_pipeline_cache.py,test_pixel_metrics_parity.py,tools/scan_nan_microssim_caches_test.py.origin/dynacell-models: 264 passed, 1 failed.The failure in both is the known order-dependent
test_load_eval_models_pins_morphem_to_the_configured_revision.Not in this PR
run_eval_direct.slurmGPU--constraintis untouched; fix(dynacell/eval): fail the job on MicroMS3IM calibration OOM instead of scoring NaN #502 raises it. This PR's hunk in that file is only the venv default and its comment.examples/hf_demo/requirements.txtstill pinscubic==0.9.0a1. That pin is deliberate (684f768), and this PR leaves it unchanged.🤖 Generated with Claude Code
https://claude.ai/code/session_015kuht3npKtb2o5E6h19xbA