feat(dynacell/eval): 2D pixel metrics (FRC) + exclude_fov_names for 2D evaluation by alxndrkalinin · Pull Request #484 · Biohub/VisCy

alxndrkalinin · 2026-07-21T22:34:00Z

Adds the eval-side plumbing to evaluate virtual-staining predictions in 2D — the precursor infrastructure for the planned 2D in-focus VS model track.

2D pixel metrics (FRC) — metrics.py gains 2D Fourier Ring Correlation alongside the existing 3D FSC, so per-z / 2D predictions can be scored on a like-for-like resolution metric (test_evaluation_metrics.py extended).
io.exclude_fov_names — evaluate_predictions can drop specific FOVs (matched by full name 0/0/fov0011 or leaf fov0011) uniformly across pred/gt/seg, so you can evaluate the finished FOVs of a partially-run predict without tripping the strict pred/gt count-alignment. Unlike limit_positions it does not set partial_walk, so deep-feature caches self-heal normally. Eval-side analog of the predict-side exclude_fov_names used by --resume-predict.
Test — end-to-end coverage of the FOV-exclusion filter (both match modes) driving evaluate_predictions on the CPU fixture.

Base: dynacell-v1.1.

Tests: test_evaluation_metrics.py (29) and the new test_exclude_fov_names_drops_positions pass. Note the eval integration tests must run in the cpdino-eval venv (cubic 0.8.0a2) — the shared .venv (cubic 0.7.0) breaks the segment_cpdino import for the whole eval-pipeline test module.

🤖 Generated with Claude Code

Make the pixel-metric path dimension-flexible so the same code scores a single in-focus 2D projection as well as the full 3D volume, following cubic's convention of dispatching on input rank: - ssim() accepts 2-D (H, W) or 3-D (D, H, W) input and derives spatial_dims from ndim, instead of hard-rejecting non-3D. - compute_pixel_metrics() uses ring-based frc_resolution (FRC_Resolution) for 2D and keeps shell-based fsc_resolution (*_FSC_Resolution) for 3D, and slices spacing to the trailing spatial axes. PCC/NRMSE/PSNR/ Spectral_PCC are shape-agnostic. The 3D path is byte-identical. This is the metric-side enabler for adding in-focus 2D columns to the benchmark without replacing the existing full-3D pixel metrics. The test change also fixes a pre-existing stale stub harness: the cubic stubs lacked glcm_features / cubic.scipy / cubic.skimage, so the guarded top-level import fell back to all-None and every stubbed test failed at HEAD. Adds those stubs plus 2D/3D dispatch, FRC-vs-FSC, and real-cubic integration tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qMvW4Um7irENLqZ1XnZY2

…iction FOVs evaluate_predictions can now drop specific FOVs (matched by full name 0/0/fov0011 or leaf fov0011) via io.exclude_fov_names, applied uniformly to pred/gt/seg positions so the strict position-count alignment still holds. Unlike limit_positions it does not set partial_walk, so deep-feature caches self-heal normally. Eval-side analog of the predict-side exclude_fov_names (--resume-predict): evaluate the finished FOVs of a partially-run predict. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg

End-to-end test driving evaluate_predictions on the 3-position CPU fixture: excluding one position by its full name (A/1/1) or its leaf (1) leaves N_POSITIONS-1 positions' worth of rows and keeps the strict pred/gt count-alignment. Mirrors the limit_positions integration test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg

* feat(viscy-models): expose downsample_z on Unet3d for Z-preserving 2D use Unet3d hardcoded downsample_z=True into UNet3DBase, so it could not run Z-preserving at Z=1 (the base D%2**depth divisor check trips on D=1) and rejected a downsample_z kwarg with TypeError. Expose downsample_z (default True, preserving FNet3D) and forward it to the base. With downsample_z=False the encoder uses stride (1,2,2) / Z-preserving ConvTranspose3d and the base skips the D divisor check, enabling the FNet2D track to run the FNet recursive U-Net at Z=1. Regression tests: default keeps downsamples_z True; downsample_z=False runs a (B,1,1,64,64) forward preserving Z; and skips the Z divisor check for a non-divisible Z. 11/11 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg * feat(dynacell): add offline focus-slab store extractor for the 2D track Pre-extract <name>_focus.zarr for 2D in-focus VS training: per position/T, estimate the in-focus plane on the phase channel by reusing the eval estimator (dynacell.evaluation.focus.estimate_focus_plane + the MIDBAND_FRACTIONS constant, no reimplementation, so train and eval land on the same plane), then cut a 2*hw+1-plane slab (hw=2 -> 5 planes) of all channels. A new clamp_shift_slab shifts the whole window inward at edge-focus FOVs (vs the eval's clipping focus_slab_from_plane) so every position yields a uniform rectangular depth for SlidingWindowDataset. omero/normalization and custom zattrs are preserved; a focus_slab_extraction provenance zattr records the params. pixel_size defaults to the store lateral spacing (--pixel-size override). zarr v3 / OME-Zarr v0.5, context-managed I/O. 9 integration tests (synthetic OME-Zarr with injected focus planes: shape, channel/zattr preservation, plane centering, clamp-shift edges, limit/overwrite, missing-channel raise, CLI) pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg * feat(dynacell): add 2D-track config overlays (fcmae_vscyto2d + fnet2d) Overlays for the 2D in-focus VS model track (fit + predict), mirroring the 3D FCMAE/FNet overlays with the 2D deltas: - recipes/models/fnet2d.yml: architecture FNet3D + model_config.downsample_z: false, in_stack_depth 1 (Z-preserving FNet run at Z=1 via the new Unet3d downsample_z passthrough). - data_overlays fcmae_vscyto2d_fit / fnet2d_fit: z_window_size 1; flatten Z augmentations (drop Z rotate/scale/sigma_z + the [15,...]/[32,...] z-crop; keep XY); RandWeightedCrop / center-crop roi Z -> 1. - model_overlays fcmae_vscyto2d_fit / fnet2d_fit: 2D model_config (FCMAE stem [1,2,2], in_stack_depth 1; FNet2D downsample_z false), single_gpu topology (drop the DDP 3h-timeout block), FCMAE lr 4e-4/200 epochs/16-mixed, FNet2D 200k steps/32-true. - model_overlays fcmae_vscyto2d_predict / fnet2d_predict: re-declare the full 2D model_config + data.z_window_size 1 so predict rebuilds a matching model before load_state_dict (engine.py:308). Render-verified via submit_benchmark_job.py --print-resolved-config. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg * feat(dynacell): add 2D nucleus/a549 smoke fit+predict leaves (fcmae_vscyto2d) Phase-B smoke pair for the 2D track: fcmae_vscyto2d_scratch, nucleus, a549. Fit reads the offline in-focus slab store dual_nucl_memb_all_focus.zarr inline (bypasses dataset_ref, as A549 leaves do), z_window_size 1, target_channel Nuclei, RandWeightedCrop re-keyed to Nuclei. Predict runs the 2D model over the unchanged 3D test store at z_window_size 1 -> full-Z prediction.zarr under the canonical directory grammar. Predict target_channel Nuclei satisfies the Nuclei_prediction eval channel contract. ckpt_path/warmup_steps are Phase-B/F TODOs (render clean; not loaded at compose time). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg * fix(dynacell): use identity Z scale in FCMAE-2D affine augmentation The FCMAE-2D fit data overlay set scale_range Z to [0, 0], intending "no Z scaling". kornia's RandomAffine3D treats scale as an absolute factor (1.0 = identity), so 0 zeroes diagonal element 3 of the 4x4 affine matrix; warp_affine3d's torch.linalg.inv then raises a singular- matrix _LinAlgError on the first GPU-augmented training batch. Use the identity factor [1, 1] and correct the mapping comment. Affects both FCMAE-2D leaves (scratch + pretrained), which compose this overlay. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg * fix(dynacell): size FCMAE-2D single-GPU batch to fit the 48 GB pool The FCMAE-2D fit overlay ran single-GPU (2D nets are light) but kept the 3D overlay's batch_size: 32, which there was split across ddp_4gpu (8/GPU). On one device 32 samples/step OOMs a 48 GB a40/a6000/l40s at ~47 GB, and the long GPU profile sets constraint: null, so any FCMAE-2D fit could land there and die on the first step. Lower to 16 (a verified bs=8 uses ~half the 48 GB, so 16 keeps ~2x headroom) so the fit stays on the cheap pool instead of being forced onto scarce 80 GB Hopper GPUs across all 36 planned fits. Effective batch is intentionally not matched to the 3D 4-GPU global batch; the 2D nets are tuned on their own. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg * feat(dynacell): add fcmae_vscyto2d_scratch fit leaves for the 2D track Author the remaining 11 UNeXt2-2D (FCMAE, pretraining=False, random init) fit leaves — 4 organelles x 3 train_sets, minus the pre-existing nucleus/a549_mantis smoke-validated exemplar. Each mirrors the 3D fcmae_vscyto3d_scratch leaf with 2D geometry: composes the fcmae_vscyto2d fit overlays (in_stack_depth=1, stem [1,2,2], z_window_size=1), single-GPU topology, and reads the offline 5-plane in-focus _focus.zarr slab stores. iPSC single-set leaves set benchmark.dataset_ref: null and inline source_channel/target_channel/data_path, because the _compose_hook resolver raises when a leaf co-declares data_path alongside a full inherited dataset_ref. Joint leaves inline a BatchedConcatDataModule over the ipsc + a549 focus stores with batch_size 4 (joint mode does not divide by num_samples, so 4 x 4 = 16 samples/step matches the single-set 16) and the Z-flattened augmentation policy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg * feat(dynacell): add fcmae_vscyto2d_pretrained fit leaves for the 2D track Author the 12 VSCyto2D (FCMAE, pretraining=False, FCMAE-pretrained encoder init) fit leaves — 4 organelles x 3 train_sets. Identical to the fcmae_vscyto2d_scratch leaves except the model block loads the 2D FCMAE encoder (encoder_only: true, ckpt from the fcmae-2d hek-a549-bj5a-livecell-ctmc-400ep pretrain) and finetunes at lr 0.0002, mirroring the cytoland VSCyto2D recipe. Same 2D geometry, focus stores, iPSC dataset_ref: null handling, and joint BatchedConcatDataModule as the scratch counterpart. er/mito keep the 3D source's _ws8500 wandb-name suffix so 2D and 3D pretrained runs stay greppable together. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg * feat(dynacell): add fnet2d fit leaves for the 2D track Author the 12 FNet2D fit leaves — 4 organelles x 3 train_sets. Each mirrors the 3D fnet3d_paper leaf but composes the fnet2d fit overlays (FNet3D architecture with downsample_z: false + in_stack_depth 1 = Z-preserving 2D net, z_window_size 1, [1,64,64] crops), keeps the single-GPU topology, and reads the offline _focus.zarr slab stores. Retains FNet's mean/std Structure normalization and 8-crops-per-FOV sampling. iPSC single-set leaves use dataset_ref: null + inlined channels/path (resolver co-declaration guard); joint leaves inline a BatchedConcatDataModule over the ipsc + a549 focus stores. sbatch mem drops from the 3D leaves' 512G to 128G since the 5-plane focus stores are far smaller than the full-Z sources. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg * docs: correct Unet3d Z-divisibility docstring for downsample_z The class docstring unconditionally claimed the model downsamples all three spatial dimensions and that Z must be divisible by 2**depth. With the downsample_z parameter, Z is preserved (stride (1,2,2)) when downsample_z=False, so the Z-divisibility constraint applies only when downsample_z=True. Also replace the hardcoded "D % 16" (correct only for depth=4) with the depth-general "D % 2**depth". Docstring only; no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg * fix(dynacell): pin OME-Zarr version=0.5 on focus-slab output store The module docstring states the output is zarr v3 / OME-Zarr v0.5, but the open_ome_zarr write call relied on iohub's default version. Pin version="0.5" explicitly so the output format is deterministic regardless of the iohub default, matching build_cpdino_seg_cleaned.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg * docs: fix stale bs=32 comments in fcmae_vscyto2d model overlay The batch size in the sibling data_overlays/fcmae_vscyto2d_fit.yml was lowered 32->16 for single-GPU OOM headroom, but two comments in the model overlay still referenced bs=32. Update both to bs=16 to match the actual batch_size the composed leaves use. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg * fix(viscy-models): restore 5D input guard in UNet3DBase.forward An explicit `if x.ndim != 5` guard once lived in Unet3d.forward but was dropped in refactor 3bad6cd when that method was removed in favor of the shared UNet3DBase.forward, which never re-added it. A 4D input (e.g. a squeezed Z=1 dim) then hits Conv3d and raises a confusing "expected N channels" error instead of a clear message. The 2D track's Z=1 / downsample_z=False path newly exercises this, so restore the guard and document both raises. Add a regression test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg --------- Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Adds the eval-side + model/config plumbing needed to support a 2D “in-focus” virtual-staining track in DynaCell, including 2D frequency-domain pixel metrics (FRC), evaluation-time FOV exclusion for partially-complete predictions, and the supporting Z-preserving U-Net variant + training/predict config leaves.

Changes:

Add 2D-capable pixel metrics: SSIM now dispatches by input rank, and compute_pixel_metrics emits FRC resolution for 2D inputs while keeping FSC for 3D.
Add io.exclude_fov_names filtering in evaluate_predictions to uniformly drop FOVs across pred/gt/seg before strict alignment checks.
Introduce Z-preserving “FNet2D” track infrastructure: Unet3d(downsample_z=...), a focus-slab extraction tool + tests, and a full set of 2D benchmark config overlays/leaves.

Reviewed changes

Copilot reviewed 54 out of 54 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
packages/viscy-models/tests/test_unet/test_unet3d.py	Adds regression tests for `downsample_z` default and Z-preserving forward behavior.
packages/viscy-models/tests/test_unet/test_unet3d_base.py	Adds test ensuring non-5D inputs raise a clear `ValueError`.
packages/viscy-models/src/viscy_models/unet/unet3d.py	Adds `downsample_z` parameter to support Z-preserving “2D-style” operation.
packages/viscy-models/src/viscy_models/unet/unet3d_base.py	Adds explicit 5D input validation and clarifies divisibility error contract.
applications/dynacell/tools/extract_focus_slab_store.py	New CLI/tool to derive `_focus.zarr` stores by extracting an in-focus Z slab.
applications/dynacell/tools/extract_focus_slab_store_test.py	Integration tests for focus-slab extraction, including CLI smoke and metadata preservation.
applications/dynacell/tests/test_evaluation_pipeline_parallel_cpu.py	Adds end-to-end test covering `io.exclude_fov_names` behavior.
applications/dynacell/tests/test_evaluation_metrics.py	Extends tests for 2D/3D SSIM dispatch + FRC vs FSC selection; expands cubic stubs.
applications/dynacell/src/dynacell/evaluation/pipeline.py	Adds uniform FOV exclusion filtering before strict pred/gt/seg alignment.
applications/dynacell/src/dynacell/evaluation/metrics.py	Adds FRC support and 2D/3D SSIM dispatch; adapts spacing for rank-specific frequency metrics.
applications/dynacell/configs/recipes/models/fnet2d.yml	Introduces FNet2D recipe (FNet3D architecture with `downsample_z: false`, `in_stack_depth: 1`).
applications/dynacell/configs/benchmarks/virtual_staining/nucleus/fnet2d/joint_ipsc_confocal_a549_mantis/train.yml	New joint 2D nucleus fit leaf using in-focus slab stores.
applications/dynacell/configs/benchmarks/virtual_staining/nucleus/fnet2d/ipsc_confocal/train.yml	New single-set 2D nucleus fit leaf for iPSC confocal focus store.
applications/dynacell/configs/benchmarks/virtual_staining/nucleus/fnet2d/a549_mantis/train.yml	New single-set 2D nucleus fit leaf for A549 focus slab store.
applications/dynacell/configs/benchmarks/virtual_staining/nucleus/fcmae_vscyto2d_scratch/joint_ipsc_confocal_a549_mantis/train.yml	New joint 2D FCMAE scratch nucleus leaf (in-focus slab geometry).
applications/dynacell/configs/benchmarks/virtual_staining/nucleus/fcmae_vscyto2d_scratch/ipsc_confocal/train.yml	New iPSC 2D FCMAE scratch nucleus leaf with overlay key overrides.
applications/dynacell/configs/benchmarks/virtual_staining/nucleus/fcmae_vscyto2d_scratch/a549_mantis/train.yml	New A549 2D FCMAE scratch nucleus leaf with overlay key overrides.
applications/dynacell/configs/benchmarks/virtual_staining/nucleus/fcmae_vscyto2d_scratch/a549_mantis/predict__a549_mantis_mock.yml	Adds 2D predict leaf (nucleus) targeting A549 mock test set.
applications/dynacell/configs/benchmarks/virtual_staining/nucleus/fcmae_vscyto2d_pretrained/joint_ipsc_confocal_a549_mantis/train.yml	New joint 2D FCMAE pretrained nucleus leaf (encoder warm-start + lr override).
applications/dynacell/configs/benchmarks/virtual_staining/nucleus/fcmae_vscyto2d_pretrained/ipsc_confocal/train.yml	New iPSC 2D FCMAE pretrained nucleus leaf.
applications/dynacell/configs/benchmarks/virtual_staining/nucleus/fcmae_vscyto2d_pretrained/a549_mantis/train.yml	New A549 2D FCMAE pretrained nucleus leaf.
applications/dynacell/configs/benchmarks/virtual_staining/mito/fnet2d/joint_ipsc_confocal_a549_mantis/train.yml	New joint 2D mito FNet2D fit leaf.
applications/dynacell/configs/benchmarks/virtual_staining/mito/fnet2d/ipsc_confocal/train.yml	New iPSC 2D mito FNet2D fit leaf.
applications/dynacell/configs/benchmarks/virtual_staining/mito/fnet2d/a549_mantis/train.yml	New A549 2D mito FNet2D fit leaf.
applications/dynacell/configs/benchmarks/virtual_staining/mito/fcmae_vscyto2d_scratch/joint_ipsc_confocal_a549_mantis/train.yml	New joint 2D FCMAE scratch mito leaf.
applications/dynacell/configs/benchmarks/virtual_staining/mito/fcmae_vscyto2d_scratch/ipsc_confocal/train.yml	New iPSC 2D FCMAE scratch mito leaf.
applications/dynacell/configs/benchmarks/virtual_staining/mito/fcmae_vscyto2d_scratch/a549_mantis/train.yml	New A549 2D FCMAE scratch mito leaf.
applications/dynacell/configs/benchmarks/virtual_staining/mito/fcmae_vscyto2d_pretrained/joint_ipsc_confocal_a549_mantis/train.yml	New joint 2D FCMAE pretrained mito leaf.
applications/dynacell/configs/benchmarks/virtual_staining/mito/fcmae_vscyto2d_pretrained/ipsc_confocal/train.yml	New iPSC 2D FCMAE pretrained mito leaf.
applications/dynacell/configs/benchmarks/virtual_staining/mito/fcmae_vscyto2d_pretrained/a549_mantis/train.yml	New A549 2D FCMAE pretrained mito leaf.
applications/dynacell/configs/benchmarks/virtual_staining/membrane/fnet2d/joint_ipsc_confocal_a549_mantis/train.yml	New joint 2D membrane FNet2D fit leaf.
applications/dynacell/configs/benchmarks/virtual_staining/membrane/fnet2d/ipsc_confocal/train.yml	New iPSC 2D membrane FNet2D fit leaf.
applications/dynacell/configs/benchmarks/virtual_staining/membrane/fnet2d/a549_mantis/train.yml	New A549 2D membrane FNet2D fit leaf.
applications/dynacell/configs/benchmarks/virtual_staining/membrane/fcmae_vscyto2d_scratch/joint_ipsc_confocal_a549_mantis/train.yml	New joint 2D FCMAE scratch membrane leaf.
applications/dynacell/configs/benchmarks/virtual_staining/membrane/fcmae_vscyto2d_scratch/ipsc_confocal/train.yml	New iPSC 2D FCMAE scratch membrane leaf.
applications/dynacell/configs/benchmarks/virtual_staining/membrane/fcmae_vscyto2d_scratch/a549_mantis/train.yml	New A549 2D FCMAE scratch membrane leaf.
applications/dynacell/configs/benchmarks/virtual_staining/membrane/fcmae_vscyto2d_pretrained/joint_ipsc_confocal_a549_mantis/train.yml	New joint 2D FCMAE pretrained membrane leaf.
applications/dynacell/configs/benchmarks/virtual_staining/membrane/fcmae_vscyto2d_pretrained/ipsc_confocal/train.yml	New iPSC 2D FCMAE pretrained membrane leaf.
applications/dynacell/configs/benchmarks/virtual_staining/membrane/fcmae_vscyto2d_pretrained/a549_mantis/train.yml	New A549 2D FCMAE pretrained membrane leaf.
applications/dynacell/configs/benchmarks/virtual_staining/er/fnet2d/joint_ipsc_confocal_a549_mantis/train.yml	New joint 2D ER FNet2D fit leaf.
applications/dynacell/configs/benchmarks/virtual_staining/er/fnet2d/ipsc_confocal/train.yml	New iPSC 2D ER FNet2D fit leaf.
applications/dynacell/configs/benchmarks/virtual_staining/er/fnet2d/a549_mantis/train.yml	New A549 2D ER FNet2D fit leaf.
applications/dynacell/configs/benchmarks/virtual_staining/er/fcmae_vscyto2d_scratch/joint_ipsc_confocal_a549_mantis/train.yml	New joint 2D FCMAE scratch ER leaf.
applications/dynacell/configs/benchmarks/virtual_staining/er/fcmae_vscyto2d_scratch/ipsc_confocal/train.yml	New iPSC 2D FCMAE scratch ER leaf.
applications/dynacell/configs/benchmarks/virtual_staining/er/fcmae_vscyto2d_scratch/a549_mantis/train.yml	New A549 2D FCMAE scratch ER leaf.
applications/dynacell/configs/benchmarks/virtual_staining/er/fcmae_vscyto2d_pretrained/joint_ipsc_confocal_a549_mantis/train.yml	New joint 2D FCMAE pretrained ER leaf.
applications/dynacell/configs/benchmarks/virtual_staining/er/fcmae_vscyto2d_pretrained/ipsc_confocal/train.yml	New iPSC 2D FCMAE pretrained ER leaf.
applications/dynacell/configs/benchmarks/virtual_staining/er/fcmae_vscyto2d_pretrained/a549_mantis/train.yml	New A549 2D FCMAE pretrained ER leaf.
applications/dynacell/configs/benchmarks/virtual_staining/_internal/shared/model/model_overlays/fnet2d_predict.yml	Adds shared FNet2D predict overlay (enforces Z=1 windowing).
applications/dynacell/configs/benchmarks/virtual_staining/_internal/shared/model/model_overlays/fnet2d_fit.yml	Adds shared FNet2D fit overlay (model+trainer binding).
applications/dynacell/configs/benchmarks/virtual_staining/_internal/shared/model/model_overlays/fcmae_vscyto2d_predict.yml	Adds shared FCMAE-2D predict overlay (2D geometry, Z=1 windowing).
applications/dynacell/configs/benchmarks/virtual_staining/_internal/shared/model/model_overlays/fcmae_vscyto2d_fit.yml	Adds shared FCMAE-2D fit overlay (2D geometry, single-GPU).
applications/dynacell/configs/benchmarks/virtual_staining/_internal/shared/model/data_overlays/fnet2d_fit.yml	Adds shared FNet2D data overlay (Z-flattened augmentations).
applications/dynacell/configs/benchmarks/virtual_staining/_internal/shared/model/data_overlays/fcmae_vscyto2d_fit.yml	Adds shared FCMAE-2D data overlay (Z-flattened augmentations).

Comments suppressed due to low confidence (1)

applications/dynacell/src/dynacell/evaluation/metrics.py:196

compute_pixel_metrics() unconditionally calls frc_resolution(...) for 2D inputs, but frc_resolution may be None if the installed cubic version doesn't provide it. That would currently crash with a confusing 'NoneType' object is not callable error; it should raise an explicit ImportError telling the user which cubic capability/version is required for 2D evaluation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…stants Hoist the mantis acquisition defaults (na_det=1.35, lambda_ill=0.450) into module constants DEFAULT_NA_DET / DEFAULT_LAMBDA_ILL in evaluation/focus.py, use them as the read_focus_compute_config defaults, and import them into the focus-slab extractor instead of re-declaring the literals. Single source of truth so the training slab and the eval slab share the same in-focus plane (train/eval cannot silently diverge). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg

Tidy ExtractionSummary and the extraction loop without behavior change: derive n_positions as a property of focus_planes (drop the redundant field and its manual counter), build the summary once the pixel size is resolved (drop the float("nan") placeholder + reassignment), inline the single-use _split_position_name helper into name.split("/"), and unpack the leading shape dims via data.shape[:3]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg

compute_pixel_metrics always receives a (Z, Y, X) spacing tuple, so reduce it to the input rank unconditionally with list(spacing)[-ndim:] and drop the dead scalar-spacing guard. Also drop the redundant int() around pred_xp.ndim (already int). Byte-identical for the 3D path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg

All 24 FCMAE_VSCyto2D fits (scratch + pretrained, every train_set × organelle) diverged to NaN loss under precision: 16-mixed — top-k checkpoints frozen at epoch 0-15, final loss/validate=NaN despite reaching epoch 199/200. FNet2D (32-true) and the 3D FCMAE (same 16-mixed + MixedLoss) both train fine, so the failure is specific to the 2D FCMAE under fp16. Verified empirically (scratchpad/repro_fcmae2d_nan.py): MixedLoss / ms_ssim_25d on a real median/iqr-normalized target with near-untrained predictions is finite in both fp32 and fp16 autocast, so the NaN is not the loss forward (nor zero-divisor normalization) but fp16 training dynamics — activation/gradient overflow the GradScaler misses. 32-true removes fp16 entirely; 2D nets are light so the cost is negligible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aLSCR6RRyUwjHXzF2RAKg

_compute_ssim_and_cs_bf16 runs its mean / mean-of-squares convolutions in bf16 regardless of trainer precision. On large-magnitude z-scored FCMAE targets (a channel with a tiny iqr, e.g. nucleus iqr~1.8, normalizes to values in the hundreds) the variance estimate mu_xx - mu_x**2 catastrophically cancels and rounds *negative*. At depth=1 (2D FCMAE) each SSIM window has only 1*H_k*W_k samples, so the cancellation is far more likely than at depth=15 (3D, where the extra Z samples keep the estimate stable) — a near-flat window then collapses the SSIM denominator to ~0 and yields intermittent NaN loss that trains fine for a while then diverges. clamp=True in MixedLoss cannot repair it (clamp bounds magnitude, not NaN), so the poisoned gradient corrupts every weight. Fix, in the shared helper: - clamp both diagonal variances to >=0 (a variance is non-negative by definition), flooring the CS denominator at c2; - bound the covariance by Cauchy-Schwarz (|sigma_xy| <= sqrt(sigma_x*sigma_y)). Exact fp32 recovers cs=1 on a flat window because the equally-negative sigma_x/sigma_y/sigma_xy cancel in the ratio; clamping only the diagonal terms leaves an unbalanced negative sigma_xy and inflates |cs| past 10 on background windows (which dominate a VS image). The bound restores cs=1; - eps-floor both denominators so a zero data_range (c1=c2=0) cannot yield 0/0. On well-conditioned windows the bound is slack and the clamps are no-ops: 3D correlated-pair aggregate SSIM and gradient-flow cosine vs MONAI fp32 are unchanged within existing tolerances. Two deterministic, hardware-independent regression tests are added: zero data_range stays finite (baseline NaN on every device), and near-flat large-magnitude windows keep cs~=1 (the unbounded-covariance variant fails with |cs-1| > 10). The negative-variance NaN itself is a bf16 rounding knife-edge not unit-reproducible across GPU architectures; it is validated end-to-end (the exact failing fit ran clean after the clamp).

…is fixed The 32-true switch (93990d1) was a wrong hypothesis: the 2D FCMAE NaN was the bf16 SSIM denominator collapse in ms_ssim_25d, not fp16 training dynamics. _compute_ssim_and_cs_bf16 runs bf16 convolutions regardless of trainer precision, so 32-true never addressed the real cause. The preceding metrics fix clamps the variances and bounds the covariance at the source, so 16-mixed is safe again — restoring the lighter, faster precision the 3D FCMAE also uses.

* feat(viscy-utils): add the segmentation-aware and background losses Port the Spotlight-motivated losses from dynacell-models so they live in the shared package before any application consumes them: SegAuxDice (seg_aux.py) with the SAUNA-map and soft-clDice weight maps (seg_aux_maps.py), the segmenter-consistency loss (seg_consistency.py), the Track H background low-pass target (background_lowpass.py), and SpotlightLoss(return_components=True). Defaults keep SpotlightLoss's existing behavior; no package imports an application. Source on dynacell-models: 40de793 (#498), 6c28a31 (#521), 0cbeda1 (#524), 050f553 (#530). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-utils): build foreground masks with CLAHE, median and Otsu Port the Spotlight fg-mask recipe from dynacell-models. generate_fg_masks now binarizes a CLAHE-equalized, 5^3-median-filtered volume at an Otsu threshold picked on that volume after a sigma=5 Gaussian; compute_otsu uses the same full-resolution smoothing, so the otsu_grid_spacing parameter is gone from generate_normalization_metadata and VisCyTrainer.preprocess. The median and Gaussian run through cubic on the GPU when cubic and a device are present, else SciPy; cubic stays an optional import. torch is imported lazily so that light callers of write_meta_field do not pay for it. Behavior change: Spotlight masks and Otsu thresholds written by `viscy preprocess` differ from main's. Source on dynacell-models: 40de793 (#498), d087d96 (#510). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-utils): make prediction writes resumable, identified and write-once Port the HCSPredictionWriter and prediction_metadata work from dynacell-models: - z_reduction="center" and z_planes (slab-only predicts write only the planes they predicted, and record them); - per-FOV completion markers and the checkpoint/settings identity that predict resume and the eval caches key on (prediction_metadata.py); - refuse writes into an existing store when a FOV outside the run has unmarked prediction channels, or when the array would grow; - assemble each volume in memory and write it once on a background thread (bytes unchanged); - raise NotImplementedError when trainer.world_size > 1, instead of letting ranks race on one store. Source on dynacell-models: 2b848a6, 1be4466, 74fa18a (#497), 8e41498 (#511), 996adb4 (#526), 6c83fff (#531), 8e74a7e (#532). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-utils): guard every fit with checkpoint and health callbacks Port the training guardrails from dynacell-models. VisCyCLI now injects, on every `fit`: - a latest-weights ModelCheckpoint that owns last.ckpt (the monitored checkpoint gets save_last=False), so last.ckpt always means the latest epoch, inheriting the leaf's write cadence; - MonitorHealthCheck, which raises when the monitored metric is non-finite or bit-identical for 10 validation epochs; - OptimizerHealthCheck, which raises after 5 consecutive non-finite losses or on GradScaler collapse. Also adds the opt-in LinearLRCooldown callback and raises the CLI subprocess test timeout to 120 s (it timed out on main). Behavior change: this applies to cytoland, dynaclr and dynacell fits. Source on dynacell-models: 20e54db, daa7477, bc0f444, 74fa18a (#497), 92693ae, 2f9ef31 (#528). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): keep SSIM finite under mixed precision; lazy exports Port two viscy-utils changes from dynacell-models: - ssim_25d / ms_ssim_25d: clamp the variances, bound the covariance by Cauchy-Schwarz through a safe sqrt, floor the stability constants symmetrically, and run MS-SSIM outside autocast, so 16-mixed training cannot NaN the gradient. SSIM/MS-SSIM loss values change slightly for MixedLoss users. - viscy_utils/__init__ resolves its exports lazily (PEP 562), keeping torch and lightning off light import paths. Source on dynacell-models: a556bc9 (#484), 130e034, cee4e41, 272051c, 59343ad, 8b2413f (#509). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-data): slab-only and read-once predict, uint8 foreground masks Port the viscy-data side of the dynacell-models predict and Spotlight work: - HCSDataModule.predict_z_planes and SlidingWindowDataset z_planes / read_z_planes, so a 2D model predicts only its focus slab; - cache_volume: with num_workers == 0 predict reads each (position, t) volume once and, when every transform is pointwise, normalizes it once; - foreground masks are read in their stored dtype (uint8) and cast to float on the device in on_after_batch_transfer. Behavior change: every app's predict at num_workers=0 takes the read-once path. Source on dynacell-models: 40de793 (#498), 996adb4 (#526), 6c83fff (#531), 8e74a7e (#532). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-models): per-axis CellDiff patches and mask-conditioning channels Port from dynacell-models: - patch_size may be a per-axis (z, y, x) tuple through the patch embedding, transformer unpatchify and ViT bottleneck, which lets CELLDiffNet run Z-preserving at Z=1 for the 2D track (an int is still accepted); - CELLDiffNet(cond_channels=1) adds extra conditioning input channels (default keeps the old single-channel net). Source on dynacell-models: 0126fe6, 0617ed7, 40de793 (#498). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(viscy-models): non-uniform ODE time grids for CellDiff sampling Port the time_schedule option of the CellDiff ODE integrators from dynacell-models: uniform (default, unchanged), cosine, or a Beta(p, q) grid packed toward the data end, which is what lets the DynaCell-lite recipe sample in 14 velocity evaluations. scipy joins the celldiff extra for betaincinv. Source on dynacell-models: 19765e7 (#520). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-models): add the MorphEm feature extractor Port MorphEmModel from dynacell-models, the fifth deep-feature extractor of the DynaCell evaluation. transformers is imported lazily and the trust_remote_code model is pinned to a Hub commit. Source on dynacell-models: d3f6422 (#479), 4918d2e, 13cb336, c9c6914. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-models): let the PatchGAN discriminator preserve the Z axis Port preserve_z from dynacell-models, so the 3D PatchGAN runs on Z=1 inputs for pix2pix2d (default off, unchanged). Source on dynacell-models: c2d0481. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-models): UNet3D downsample_z for FNet-2D; reject non-5D input Port from dynacell-models: - UNet3D(downsample_z=True): False keeps Z through the encoder, which is how FNet2D runs on single planes (default unchanged); - UNet3DBase.forward raises on non-5D input, and a comment flags the ConvTranspose checkerboard artifact. Source on dynacell-models: a556bc9 (#484), 83a1d69, 16b1d31. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-transforms): stop BatchedRandAffined blanking singleton axes Port 9640537 from dynacell-models: a size-1 spatial axis (Z=1 for 2D models) gets an identity row in the affine, instead of being resampled off the volume and zeroed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(viscy-transforms): mark the sampled normalizers pointwise NormalizeSampled and MinMaxSampled set pointwise = True, which lets the read-once predict path in viscy-data normalize a whole volume once instead of once per window. Source on dynacell-models: 8e74a7e (#532). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(viscy-transforms): make the Gaussian-noise statistics test deterministic Port 3409560 (#513) from dynacell-models. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build: parallel test runs with pytest-xdist Port dfcf9a5 (#512) from dynacell-models: pytest-xdist joins the test group, and a root conftest.py pins OMP/MKL/OpenBLAS to one thread under each xdist worker. Without the pin every worker's torch sizes its pool to all cores, which made `-n 11` 10x slower than it should be and timed out subprocess tests. CLAUDE.md documents the usage. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: register pytest markers at the root and make them strict Port the marker half of c50300f (#489) from dynacell-models. strict_markers = true turns an unregistered marker into a collection error, so a typo'd marker can no longer silently decorate nothing. hpc_integration moves from the cytoland and dynaclr conftests to the root: an addinivalue_line registration only holds when that conftest is in the invocation's ancestry. dynaclr drops its own testpaths so the root ini applies. requires_data is registered too: main's applications/dynacell/tools/generate_grouped_eval_configs_test.py uses it, and strict markers would otherwise drop that module at collection. (dynacell-models no longer uses the marker; the dynacell PR that follows removes the registration with it.) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build: put the dynacell hf_demo on the pytest pythonpath Port the root half of 7b7cb43 (#523) from dynacell-models so the HF demo's tests import its app module. The directory already exists on main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(dynaclr): join annotations on the full fov_name Port d1ea492 from dynacell-models. The orchestrated linear-classifier fixture joined on the first fov_name component, which failed 7 tests on main once pytest collects dynaclr's colocated *_test.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(cytoland): correct the Spotlight recipe comments Port the cytoland recipe comment fixes from 40de793 (#498): the preprocess flags take explicit values (`--compute_otsu true --compute_fg_masks true`), fg_threshold and nonzero_threshold are inert while fg_mask_key is set, and the Otsu-centred target is what makes 0 the FG/BG boundary. Comments only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(cytoland): install the VCP tutorials' unpublished deps from GitHub Port the tutorial fix from d3f6422 (#479): cytoland and viscy-utils are not on PyPI yet, so the tutorials install them from the GitHub monorepo; their siblings resolve from PyPI. Adds the jupytext-paired notebooks (no outputs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: cut CLAUDE.md content the code or global memory already states Port 93ac234 from dynacell-models: drop the repo-layout tree, setup and common-command blocks, lint/docstring rules (pyproject.toml is the source) and the generic coding-philosophy sections. The DynaCell job runbook from the same file arrives with the dynacell PR, since it documents tools that only exist there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run the viscy-utils tests in their own job viscy-utils had no CI job on main or on dynacell-models: four of its test files ran inside the dynacell job and the rest (losses, metrics, meta_utils, optimizer health, CLI, lazy imports) ran nowhere. Mirror the viscy-data job on ubuntu-latest for Python 3.12 and 3.13, and make the gate job wait for it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(dynacell): run the launcher, engine and prediction-writer suites Port the test-dynacell-configs job from dynacell-models, minus the eight test files that only arrive with the dynacell PR: sync viscy-utils' test group with dynacell, and add test_submit_benchmark_batch, test_engine, test_pipeline_cache, feature_metrics_test and the viscy-utils prediction-writer, prediction-metadata, monitor-health and LR-cooldown tests. The dynacell-models PR target is not carried over. Source on dynacell-models: 33d2fee, 74fa18a (#497), 2d1ed32 (#506), 4450efd (#518), 2798dee (#519), 2f9ef31 (#528). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build: re-lock against the ported packages and main's dynacell Start from dynacell-models' uv.lock and re-resolve with `uv lock` against this tree, whose applications/dynacell/pyproject.toml is still main's. Against dynacell-models' lock this drops the ten dynacell-only packages (dinov3, etils, ftfy, jsonpath-rw, mlcroissant, pandas-stubs, ply, rdflib, submitit, termcolor) and resolves cubic back to 0.7.0; the dynacell PR restores them. Against main's lock it adds pytest-xdist and moves boto3/botocore 1.43.0 -> 1.43.20, cellpose 4.1.1 -> 4.2.1.1, pyarrow 19.0.1 -> 24.0.0, s3transfer and tracksdata; torch and lightning are unchanged. `uv lock --check` exits 0 (main's own lock does not). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): declare torchvision, which evaluation.metrics imports viscy_utils.evaluation.metrics imports torchvision.ops at module level, and losses.MixedLoss imports that module, but viscy-utils never listed torchvision. Installs only worked because another workspace member pulled it in. The new test-utils CI job syncs viscy-utils alone and failed to collect six test modules with ModuleNotFoundError. The lock gains only the dependency edge; no version moves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(dynacell): keep main's batch-launcher test and the writer tests out Two corrections to the test-dynacell-configs job: - main's tests/test_submit_benchmark_batch.py resolves leaves under /hpc and fails on a runner (5 PermissionErrors in CI). The dynacell-models version that runs there arrives with the DynaCell PR, which adds it back. - the four viscy-utils test files (prediction writer, prediction metadata, monitor health, LR cooldown) now run in the test-utils job on both Python versions, so the dynacell job no longer repeats them or syncs viscy-utils' test group for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(viscy-utils): share the fit-callbacks lookup between CLI injectors _inject_checkpoint_guardrails and _inject_optimizer_health_guard each repeated the same walk from the config to fit's trainer.callbacks plus the already-injected check; the `else config` branch of that walk could never run behind the `subcommand != "fit"` return. Both now call _fit_callbacks_without. _callback_field collapses its two identical branches, and the monitored checkpoint's init_args are kept from the search loop instead of being looked up twice. No behavior change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(viscy-utils): drop MonitorHealthCheck's unused check_finite flag Nothing sets check_finite (the CLI injects monitor and patience only, and no config or test passes it), so the callback always checks finiteness; it now does so with math.isfinite instead of building a tensor for one float. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(viscy-utils): stop describing fg masks as built from Otsu thresholds generate_fg_masks does not read the stored otsu_threshold; it binarizes a CLAHE-equalized, median-filtered volume. Its summary line and VisCyTrainer.preprocess's compute_fg_masks entry still said "from Otsu thresholds". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build: leave the hf_demo pytest path to the DynaCell PR Nothing on this branch imports from applications/dynacell/examples/ hf_demo: its only test, build_demo_data_test.py, arrives with the DynaCell PR, which adds the pythonpath entry back with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): guard a fit that sets no callbacks Trainer.callbacks defaults to None, and the CLI injectors returned early unless they found a list, so a fit that set no callbacks ran without OptimizerHealthCheck, the guard meant for every fit. The shared lookup now gives such a fit an empty list to append to (None and [] mean the same to Trainer). The checkpoint guardrails still need a monitored ModelCheckpoint and remain a no-op there. Reported by Copilot on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(viscy-utils): say what the AMP skip counter actually counts The counter resets only when GradScaler grows its scale, so successful optimizer steps can sit between the reductions it counts; the error message claimed every step was skipped "in a row". Describe it as reductions without growth in between, which is what it detects: the backward overflowing faster than growth_interval clean steps can restore the scale. Behavior unchanged. Reported by Copilot on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): resolve viscy_utils.callbacks names lazily VisCyCLI now injects MonitorHealthCheck and OptimizerHealthCheck into every fit by class path through viscy_utils.callbacks, whose __init__ eagerly imported OnlineEvalCallback and with it scikit-learn, which only the eval extra installs. A fit on a base install would therefore fail before training. The package now resolves its exports lazily (PEP 562), like viscy_utils itself. The new test blocks sklearn and imports the health callbacks: the eager __init__ fails it with ModuleNotFoundError, the lazy one passes. Reported by Copilot on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(cytoland): install viscy-data from GitHub in the VCP tutorials The tutorials install viscy-utils from the GitHub monorepo and let the siblings resolve from PyPI. The writer they predict with now reads SlidingWindowDataset.z_planes, which the PyPI viscy-data 0.1.0 does not have (checked in its wheel), so predict would fail with AttributeError in on_predict_start. Install viscy-data from GitHub too, in the .py sources and their paired notebooks. Found by /code-review on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): return an empty CLAHE result when the clip range is zero foreground_mask_from_volume guards a constant volume, but a blank volume with a few hot pixels (under 0.2% of voxels) has equal 0.1/99.9 percentiles too, and equalize_clahe then divided by zero and fed NaN to CLAHE (RuntimeWarning: divide by zero, reproduced on a (20, 64, 64) volume). equalize_clahe now returns zeros there, which the mask's existing flat-histogram guard turns into an empty mask, as for a constant volume. Volumes with any contrast are unchanged. Found by /code-review on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): keep a step-cadence leaf's last.ckpt on its step trigger The injected latest-weights ModelCheckpoint takes over save_last from the monitored one but copied only every_n_epochs (defaulting to 1). A leaf that checkpoints every N steps, typical for max_steps runs with long epochs, would get last.ckpt only at epoch boundaries, so a preempted run resumed from much older weights. The injected callback now copies the monitored callback's every_n_train_steps or train_time_interval when set; epoch-cadence leaves are unchanged. No current leaf combines a monitored checkpoint with a step trigger. Found by /code-review on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): take last.ckpt from every monitored checkpoint The guardrail turned save_last off only on the first monitored ModelCheckpoint. A leaf with two monitored checkpoints (say loss and a validation metric) kept a second last.ckpt writer next to the injected latest-weights callback, so last.ckpt would alternate between "latest" and "that monitor's cadence". All monitored callbacks now give it up; the first still names the monitor, directory and cadence. No current leaf has two monitored checkpoints. Found by /code-review on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): write AnnData zarr without switching pandas string inference off anndata 0.12.6 (capped below 0.12.9, which needs pandas<3) cannot write pandas 3's default Arrow-backed strings to zarr. The workaround was global: write_embedding_dataset set pd.options.future.infer_string = False in the caller's process, and the test conftest set it for the whole session. Tests therefore never ran under the configuration users have, and the other writers (EmbeddingSnapshot's rewrite, annotation convert) worked only after the embedding writer had flipped the option. write_anndata_zarr converts obs/var Arrow strings (columns, categories, index) to object just before writing, reusing the conversion append_to_anndata_zarr already had; all three writers and the tests use it, and both global switches are gone. New tests pin a round trip under the default, that plain write_zarr still fails (so the TODO is dropped deliberately once anndata 0.13 lands), and that the embedding writer leaves the option alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MzpgExfYMKCgogfnhcSgDc * fix(dynaclr): drop the session-wide pandas string-inference switch dynaclr's pytest_configure set pd.options.future.infer_string = False for the whole pytest session, so in a repo-wide run every package's tests ran with pandas 3's default off (it hid the viscy-utils guard added in the previous commit). split_embeddings flipped it in the caller's process too. Both go; the eleven test writes and the split writer use viscy_utils' write_anndata_zarr instead. The cellanome embedding scripts keep their own switch: each owns its process and has no test to check a change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MzpgExfYMKCgogfnhcSgDc --------- Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* feat(viscy-utils): add the segmentation-aware and background losses Port the Spotlight-motivated losses from dynacell-models so they live in the shared package before any application consumes them: SegAuxDice (seg_aux.py) with the SAUNA-map and soft-clDice weight maps (seg_aux_maps.py), the segmenter-consistency loss (seg_consistency.py), the Track H background low-pass target (background_lowpass.py), and SpotlightLoss(return_components=True). Defaults keep SpotlightLoss's existing behavior; no package imports an application. Source on dynacell-models: 40de793 (#498), 6c28a31 (#521), 0cbeda1 (#524), 050f553 (#530). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-utils): build foreground masks with CLAHE, median and Otsu Port the Spotlight fg-mask recipe from dynacell-models. generate_fg_masks now binarizes a CLAHE-equalized, 5^3-median-filtered volume at an Otsu threshold picked on that volume after a sigma=5 Gaussian; compute_otsu uses the same full-resolution smoothing, so the otsu_grid_spacing parameter is gone from generate_normalization_metadata and VisCyTrainer.preprocess. The median and Gaussian run through cubic on the GPU when cubic and a device are present, else SciPy; cubic stays an optional import. torch is imported lazily so that light callers of write_meta_field do not pay for it. Behavior change: Spotlight masks and Otsu thresholds written by `viscy preprocess` differ from main's. Source on dynacell-models: 40de793 (#498), d087d96 (#510). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-utils): make prediction writes resumable, identified and write-once Port the HCSPredictionWriter and prediction_metadata work from dynacell-models: - z_reduction="center" and z_planes (slab-only predicts write only the planes they predicted, and record them); - per-FOV completion markers and the checkpoint/settings identity that predict resume and the eval caches key on (prediction_metadata.py); - refuse writes into an existing store when a FOV outside the run has unmarked prediction channels, or when the array would grow; - assemble each volume in memory and write it once on a background thread (bytes unchanged); - raise NotImplementedError when trainer.world_size > 1, instead of letting ranks race on one store. Source on dynacell-models: 2b848a6, 1be4466, 74fa18a (#497), 8e41498 (#511), 996adb4 (#526), 6c83fff (#531), 8e74a7e (#532). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-utils): guard every fit with checkpoint and health callbacks Port the training guardrails from dynacell-models. VisCyCLI now injects, on every `fit`: - a latest-weights ModelCheckpoint that owns last.ckpt (the monitored checkpoint gets save_last=False), so last.ckpt always means the latest epoch, inheriting the leaf's write cadence; - MonitorHealthCheck, which raises when the monitored metric is non-finite or bit-identical for 10 validation epochs; - OptimizerHealthCheck, which raises after 5 consecutive non-finite losses or on GradScaler collapse. Also adds the opt-in LinearLRCooldown callback and raises the CLI subprocess test timeout to 120 s (it timed out on main). Behavior change: this applies to cytoland, dynaclr and dynacell fits. Source on dynacell-models: 20e54db, daa7477, bc0f444, 74fa18a (#497), 92693ae, 2f9ef31 (#528). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): keep SSIM finite under mixed precision; lazy exports Port two viscy-utils changes from dynacell-models: - ssim_25d / ms_ssim_25d: clamp the variances, bound the covariance by Cauchy-Schwarz through a safe sqrt, floor the stability constants symmetrically, and run MS-SSIM outside autocast, so 16-mixed training cannot NaN the gradient. SSIM/MS-SSIM loss values change slightly for MixedLoss users. - viscy_utils/__init__ resolves its exports lazily (PEP 562), keeping torch and lightning off light import paths. Source on dynacell-models: a556bc9 (#484), 130e034, cee4e41, 272051c, 59343ad, 8b2413f (#509). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-data): slab-only and read-once predict, uint8 foreground masks Port the viscy-data side of the dynacell-models predict and Spotlight work: - HCSDataModule.predict_z_planes and SlidingWindowDataset z_planes / read_z_planes, so a 2D model predicts only its focus slab; - cache_volume: with num_workers == 0 predict reads each (position, t) volume once and, when every transform is pointwise, normalizes it once; - foreground masks are read in their stored dtype (uint8) and cast to float on the device in on_after_batch_transfer. Behavior change: every app's predict at num_workers=0 takes the read-once path. Source on dynacell-models: 40de793 (#498), 996adb4 (#526), 6c83fff (#531), 8e74a7e (#532). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-models): per-axis CellDiff patches and mask-conditioning channels Port from dynacell-models: - patch_size may be a per-axis (z, y, x) tuple through the patch embedding, transformer unpatchify and ViT bottleneck, which lets CELLDiffNet run Z-preserving at Z=1 for the 2D track (an int is still accepted); - CELLDiffNet(cond_channels=1) adds extra conditioning input channels (default keeps the old single-channel net). Source on dynacell-models: 0126fe6, 0617ed7, 40de793 (#498). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(viscy-models): non-uniform ODE time grids for CellDiff sampling Port the time_schedule option of the CellDiff ODE integrators from dynacell-models: uniform (default, unchanged), cosine, or a Beta(p, q) grid packed toward the data end, which is what lets the DynaCell-lite recipe sample in 14 velocity evaluations. scipy joins the celldiff extra for betaincinv. Source on dynacell-models: 19765e7 (#520). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-models): add the MorphEm feature extractor Port MorphEmModel from dynacell-models, the fifth deep-feature extractor of the DynaCell evaluation. transformers is imported lazily and the trust_remote_code model is pinned to a Hub commit. Source on dynacell-models: d3f6422 (#479), 4918d2e, 13cb336, c9c6914. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-models): let the PatchGAN discriminator preserve the Z axis Port preserve_z from dynacell-models, so the 3D PatchGAN runs on Z=1 inputs for pix2pix2d (default off, unchanged). Source on dynacell-models: c2d0481. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(viscy-models): UNet3D downsample_z for FNet-2D; reject non-5D input Port from dynacell-models: - UNet3D(downsample_z=True): False keeps Z through the encoder, which is how FNet2D runs on single planes (default unchanged); - UNet3DBase.forward raises on non-5D input, and a comment flags the ConvTranspose checkerboard artifact. Source on dynacell-models: a556bc9 (#484), 83a1d69, 16b1d31. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-transforms): stop BatchedRandAffined blanking singleton axes Port 9640537 from dynacell-models: a size-1 spatial axis (Z=1 for 2D models) gets an identity row in the affine, instead of being resampled off the volume and zeroed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(viscy-transforms): mark the sampled normalizers pointwise NormalizeSampled and MinMaxSampled set pointwise = True, which lets the read-once predict path in viscy-data normalize a whole volume once instead of once per window. Source on dynacell-models: 8e74a7e (#532). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(viscy-transforms): make the Gaussian-noise statistics test deterministic Port 3409560 (#513) from dynacell-models. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build: parallel test runs with pytest-xdist Port dfcf9a5 (#512) from dynacell-models: pytest-xdist joins the test group, and a root conftest.py pins OMP/MKL/OpenBLAS to one thread under each xdist worker. Without the pin every worker's torch sizes its pool to all cores, which made `-n 11` 10x slower than it should be and timed out subprocess tests. CLAUDE.md documents the usage. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: register pytest markers at the root and make them strict Port the marker half of c50300f (#489) from dynacell-models. strict_markers = true turns an unregistered marker into a collection error, so a typo'd marker can no longer silently decorate nothing. hpc_integration moves from the cytoland and dynaclr conftests to the root: an addinivalue_line registration only holds when that conftest is in the invocation's ancestry. dynaclr drops its own testpaths so the root ini applies. requires_data is registered too: main's applications/dynacell/tools/generate_grouped_eval_configs_test.py uses it, and strict markers would otherwise drop that module at collection. (dynacell-models no longer uses the marker; the dynacell PR that follows removes the registration with it.) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build: put the dynacell hf_demo on the pytest pythonpath Port the root half of 7b7cb43 (#523) from dynacell-models so the HF demo's tests import its app module. The directory already exists on main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(dynaclr): join annotations on the full fov_name Port d1ea492 from dynacell-models. The orchestrated linear-classifier fixture joined on the first fov_name component, which failed 7 tests on main once pytest collects dynaclr's colocated *_test.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(cytoland): correct the Spotlight recipe comments Port the cytoland recipe comment fixes from 40de793 (#498): the preprocess flags take explicit values (`--compute_otsu true --compute_fg_masks true`), fg_threshold and nonzero_threshold are inert while fg_mask_key is set, and the Otsu-centred target is what makes 0 the FG/BG boundary. Comments only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(cytoland): install the VCP tutorials' unpublished deps from GitHub Port the tutorial fix from d3f6422 (#479): cytoland and viscy-utils are not on PyPI yet, so the tutorials install them from the GitHub monorepo; their siblings resolve from PyPI. Adds the jupytext-paired notebooks (no outputs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: cut CLAUDE.md content the code or global memory already states Port 93ac234 from dynacell-models: drop the repo-layout tree, setup and common-command blocks, lint/docstring rules (pyproject.toml is the source) and the generic coding-philosophy sections. The DynaCell job runbook from the same file arrives with the dynacell PR, since it documents tools that only exist there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run the viscy-utils tests in their own job viscy-utils had no CI job on main or on dynacell-models: four of its test files ran inside the dynacell job and the rest (losses, metrics, meta_utils, optimizer health, CLI, lazy imports) ran nowhere. Mirror the viscy-data job on ubuntu-latest for Python 3.12 and 3.13, and make the gate job wait for it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(dynacell): run the launcher, engine and prediction-writer suites Port the test-dynacell-configs job from dynacell-models, minus the eight test files that only arrive with the dynacell PR: sync viscy-utils' test group with dynacell, and add test_submit_benchmark_batch, test_engine, test_pipeline_cache, feature_metrics_test and the viscy-utils prediction-writer, prediction-metadata, monitor-health and LR-cooldown tests. The dynacell-models PR target is not carried over. Source on dynacell-models: 33d2fee, 74fa18a (#497), 2d1ed32 (#506), 4450efd (#518), 2798dee (#519), 2f9ef31 (#528). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build: re-lock against the ported packages and main's dynacell Start from dynacell-models' uv.lock and re-resolve with `uv lock` against this tree, whose applications/dynacell/pyproject.toml is still main's. Against dynacell-models' lock this drops the ten dynacell-only packages (dinov3, etils, ftfy, jsonpath-rw, mlcroissant, pandas-stubs, ply, rdflib, submitit, termcolor) and resolves cubic back to 0.7.0; the dynacell PR restores them. Against main's lock it adds pytest-xdist and moves boto3/botocore 1.43.0 -> 1.43.20, cellpose 4.1.1 -> 4.2.1.1, pyarrow 19.0.1 -> 24.0.0, s3transfer and tracksdata; torch and lightning are unchanged. `uv lock --check` exits 0 (main's own lock does not). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(dynacell): DynaCell v1.1 benchmark configs from dynacell-models Port every DynaCell config path of dynacell-models verbatim: the model leaves (baselines, A549/joint/HEK/brightfield, the 2D track with its seed replicates and warmup-matched refit, the temporal and patch-vs-augmentation ablations, CELL-Diff-R2 and the DynaCell-lite leaves), the grouped and instance-AP eval buckets of the paper models, the shared overlays, train/predict sets and launcher profiles, and the deletion of 155 stale config files. The leaves are generated; review the generators and guard tests in the code commits, not each leaf. Spotlight v2 arm leaves, their four fit overlays and their 231 spotlight_v2_* eval buckets are left out; they follow in their own PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(dynacell): DynaCell v1.1 application code from dynacell-models Port every DynaCell code path of dynacell-models verbatim (226 non-merge commits since 2026-06-16; PRs #479-#530): the evaluation pipeline (cpdino/MorphEm/focus-aware 2D, focus2d slab metrics, foreground-limited metrics, MicroMS3IM, scale-invariant pixel metrics, CP shared space, provenance and cache invalidation), the paths registry, preprocessing/registry/croissant/distribution, the launcher and stall watchdog, the lite benchmark and CELL-Diff fast inference, the 2D track, the HF demo, docs, and the engine, celldiff_wrapper and mask_conditioning modules exactly as on dynacell-models. The three LightningModules keep their Spotlight training hooks (seg_aux, seg_consistency, mask_mode, target_bg_lowpass). Every hook defaults off, the bit-identity tests guard that, and only Spotlight leaves turn them on; two DynaCell-paper 2D checkpoints store seg_aux_weight in their hparams, so a hook-free engine could not resume them. mask_conditioning.py comes along because engine.py and celldiff_wrapper.py import it. The files that import the Spotlight leaf generator are trimmed in the commits that follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(dynacell): lock the dynacell eval, croissant and preprocess deps Take dynacell-models' uv.lock verbatim, which this tree's pyprojects now match (`uv lock --check` exits 0). Against the shared-packages lock it moves cubic 0.7.0 -> 0.9.0a4 and adds dinov3 (pinned commit), etils, ftfy, jsonpath-rw, mlcroissant, pandas-stubs, ply, rdflib, submitit and termcolor. A fresh `uv lock` would instead float dinov3 to its current HEAD and bump mlcroissant/pandas-stubs, so the verbatim lock keeps the versions the paper evaluations ran with. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: drop the requires_data marker registration Its only user was main's generate_grouped_eval_configs_test.py, which the DynaCell port replaced: the live-data tests there are now marked slow. With strict markers, nothing needs the registration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(dynacell): run the release-path, provenance and watchdog suites Restore the eight dynacell test files the shared-packages PR left out of the test-dynacell-configs job, now that they exist: test_evaluation_microssim_calibration, segmentation_test, test_distribution, test_croissant, test_main_cli_routing, test_evaluation_fid_provenance, provenance_test and watch_stalled_jobs_test. The job now matches dynacell-models'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: add the runbook for jobs that hang while holding an allocation Port the DynaCell job runbook of the root CLAUDE.md from dynacell-models (6193b9c, bcbfbe6, c4c5ed0, 74fa18a #497, e1d75cf): how to tell a finished-but-hung job from healthy compute by the CPU-time derivative, the stall watchdog (tools/watch_stalled_jobs.py), per-family predict and fit wall limits, and the forensics to capture before cancelling. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(dynacell): scope the grouped eval walk to registered model dirs The grouped generator imported ARMS from the Spotlight-v2 leaf generator, which is not part of this PR, to skip the Spotlight arms that share the canonical prediction tree. Drop that import and the Spotlight model names from _CODE_TO_PAPER, _DETERMINISTIC_MODELS and _CELLDIFF_MODELS, keeping the four 2D seed replicates the 2D appendix scores. The shared tree still holds the arms' stores, and parse_zarr_name raises on an unknown model token (e.g. the CellDiff "variant" segaux_iterative). walk_predictions now skips any model dir outside _REGISTERED_MODEL_DIRS before parsing and prints how many stores it skipped. The test that pinned the Spotlight skip becomes one that pins the unregistered-dir skip and its count. evaluation/paths.py keeps its Spotlight names: without them canonical_model_name would silently resolve an arm's run dir onto its baseline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(dynacell): drop the Spotlight leaf entries from the config guards The Spotlight arm leaves are not part of this PR, so: - test_benchmark_config_composition: drop the eight v1 *_spotlight train leaves, their iPSC predict leaves and data-hparam entries, the _UNMIGRATED_MODELS filter that only they needed, and the Spotlight members of _WIDE_GPU_TRAIN_LEAVES and _LONG_WALL_TRAIN_LEAVES (the 2D pix2pix seed1 replicate keeps its wide-GPU opt-out); - test_paths: drop the two tests that import the Spotlight leaf generator or depend on the grouped generator registering the arms. The canonical_model_name / resolve_model tests of the arm names stay, since evaluation/paths.py keeps those names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(dynacell): describe the 2D seed1 leaves without the Spotlight generator The 40 leaves of the four 2D baseline seed replicates were emitted by the Spotlight-v2 leaf generator, which is not part of this PR, and their headers told the reader to edit that generator. Say instead what they are for: the second draw scored by the *__2d_focus2d_seed1 buckets. Comments only; the configs are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(dynacell): drop the Spotlight eval generator from the generator lists evaluation/README.md and the hand-written-bucket guard's comment in test_evaluate_compose.py named generate_spotlight_v2_eval_configs.py among the generators that write grouped buckets; it is not part of this PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): declare torchvision, which evaluation.metrics imports viscy_utils.evaluation.metrics imports torchvision.ops at module level, and losses.MixedLoss imports that module, but viscy-utils never listed torchvision. Installs only worked because another workspace member pulled it in. The new test-utils CI job syncs viscy-utils alone and failed to collect six test modules with ModuleNotFoundError. The lock gains only the dependency edge; no version moves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(dynacell): keep main's batch-launcher test and the writer tests out Two corrections to the test-dynacell-configs job: - main's tests/test_submit_benchmark_batch.py resolves leaves under /hpc and fails on a runner (5 PermissionErrors in CI). The dynacell-models version that runs there arrives with the DynaCell PR, which adds it back. - the four viscy-utils test files (prediction writer, prediction metadata, monitor health, LR cooldown) now run in the test-utils job on both Python versions, so the dynacell job no longer repeats them or syncs viscy-utils' test group for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(viscy-utils): share the fit-callbacks lookup between CLI injectors _inject_checkpoint_guardrails and _inject_optimizer_health_guard each repeated the same walk from the config to fit's trainer.callbacks plus the already-injected check; the `else config` branch of that walk could never run behind the `subcommand != "fit"` return. Both now call _fit_callbacks_without. _callback_field collapses its two identical branches, and the monitored checkpoint's init_args are kept from the search loop instead of being looked up twice. No behavior change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(viscy-utils): drop MonitorHealthCheck's unused check_finite flag Nothing sets check_finite (the CLI injects monitor and patience only, and no config or test passes it), so the callback always checks finiteness; it now does so with math.isfinite instead of building a tensor for one float. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(viscy-utils): stop describing fg masks as built from Otsu thresholds generate_fg_masks does not read the stored otsu_threshold; it binarizes a CLAHE-equalized, median-filtered volume. Its summary line and VisCyTrainer.preprocess's compute_fg_masks entry still said "from Otsu thresholds". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build: leave the hf_demo pytest path to the DynaCell PR Nothing on this branch imports from applications/dynacell/examples/ hf_demo: its only test, build_demo_data_test.py, arrives with the DynaCell PR, which adds the pythonpath entry back with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): guard a fit that sets no callbacks Trainer.callbacks defaults to None, and the CLI injectors returned early unless they found a list, so a fit that set no callbacks ran without OptimizerHealthCheck, the guard meant for every fit. The shared lookup now gives such a fit an empty list to append to (None and [] mean the same to Trainer). The checkpoint guardrails still need a monitored ModelCheckpoint and remain a no-op there. Reported by Copilot on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(viscy-utils): say what the AMP skip counter actually counts The counter resets only when GradScaler grows its scale, so successful optimizer steps can sit between the reductions it counts; the error message claimed every step was skipped "in a row". Describe it as reductions without growth in between, which is what it detects: the backward overflowing faster than growth_interval clean steps can restore the scale. Behavior unchanged. Reported by Copilot on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): resolve viscy_utils.callbacks names lazily VisCyCLI now injects MonitorHealthCheck and OptimizerHealthCheck into every fit by class path through viscy_utils.callbacks, whose __init__ eagerly imported OnlineEvalCallback and with it scikit-learn, which only the eval extra installs. A fit on a base install would therefore fail before training. The package now resolves its exports lazily (PEP 562), like viscy_utils itself. The new test blocks sklearn and imports the health callbacks: the eager __init__ fails it with ModuleNotFoundError, the lazy one passes. Reported by Copilot on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(cytoland): install viscy-data from GitHub in the VCP tutorials The tutorials install viscy-utils from the GitHub monorepo and let the siblings resolve from PyPI. The writer they predict with now reads SlidingWindowDataset.z_planes, which the PyPI viscy-data 0.1.0 does not have (checked in its wheel), so predict would fail with AttributeError in on_predict_start. Install viscy-data from GitHub too, in the .py sources and their paired notebooks. Found by /code-review on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): return an empty CLAHE result when the clip range is zero foreground_mask_from_volume guards a constant volume, but a blank volume with a few hot pixels (under 0.2% of voxels) has equal 0.1/99.9 percentiles too, and equalize_clahe then divided by zero and fed NaN to CLAHE (RuntimeWarning: divide by zero, reproduced on a (20, 64, 64) volume). equalize_clahe now returns zeros there, which the mask's existing flat-histogram guard turns into an empty mask, as for a constant volume. Volumes with any contrast are unchanged. Found by /code-review on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): keep a step-cadence leaf's last.ckpt on its step trigger The injected latest-weights ModelCheckpoint takes over save_last from the monitored one but copied only every_n_epochs (defaulting to 1). A leaf that checkpoints every N steps, typical for max_steps runs with long epochs, would get last.ckpt only at epoch boundaries, so a preempted run resumed from much older weights. The injected callback now copies the monitored callback's every_n_train_steps or train_time_interval when set; epoch-cadence leaves are unchanged. No current leaf combines a monitored checkpoint with a step trigger. Found by /code-review on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(dynacell): skip only PAPER_KEY-known arms in the grouped eval walk The scope filter skipped every model dir outside the generator's registry and only printed a count. That also silenced the loud "unknown model code-name" a new, never-registered model used to raise, so its stores would drop out of every paper bucket without an error. Now the walk skips only dirs that paths.PAPER_KEY knows but this generator does not register (the Spotlight arms on this branch); anything unknown to both still raises in parse_zarr_name, as on dynacell-models. Tests pin both sides. Found by /code-review on #534. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(dynacell): say the instance-AP audit does not share the walk's arm skip audit_prediction_coverage claimed to apply the same skip filters as walk_predictions. It applies the same test-set filter, but on purpose not the walk's skip of PAPER_KEY models the grouped generator leaves unregistered: listing every unregistered model is what it is for. Found by /code-review on #534. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(dynacell): carry the seg-consistency engine and focus-parity tests test_seg_consistency_training.py exercises the seg_consistency hook of DynacellUNet and DynacellGAN with a stand-in segmenter, and test_seg_consistency_focus.py pins the loss's focus-plane rule to dynacell.evaluation.focus. Both test code this PR ships and import nothing from the Spotlight PR, so they belong here rather than with the Spotlight arms. Verbatim from dynacell-models. Found by /code-review on #534. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(viscy-utils): take last.ckpt from every monitored checkpoint The guardrail turned save_last off only on the first monitored ModelCheckpoint. A leaf with two monitored checkpoints (say loss and a validation metric) kept a second last.ckpt writer next to the injected latest-weights callback, so last.ckpt would alternate between "latest" and "that monitor's cadence". All monitored callbacks now give it up; the first still names the monitor, directory and cadence. No current leaf has two monitored checkpoints. Found by /code-review on #533. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(dynacell): one model-dir acceptance rule for parse and walk filter _REGISTERED_MODEL_DIRS restated, as a separate set, the rule _split_model_variant uses to accept a dir name, so the two could drift. _match_model_variant is now that rule (None when the dir does not parse); _split_model_variant wraps it and raises the same errors as before, and the walk filter calls it. No behavior change. Found by /code-review on #534. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(dynacell): share the foreign-model skip between the walk and the audit walk_predictions skipped the Spotlight arms' dirs on the shared tree, but the instance-AP coverage audit still parsed them, so on this branch it reported 468 coverage errors on the live tree instead of dynacell-models' 102, and its docstring claimed it applied the walk's filters. Both now call one predicate, is_foreign_model_dir: a dir whose model paths.PAPER_KEY knows but the grouped generator does not parse. Measured on the live tree, the audit's error set is identical to dynacell-models' (102 entries; the remainder are pre-existing). The new audit test holds one registered, one foreign and one unknown store and expects exactly the unknown one. Found by /code-review on #534 (findings 1, 9, 10). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(dynacell): keep raising on campaign models the generator cannot parse pix2pix2d_unetvit_wmatch{,_seed1} are in _CODE_TO_PAPER (and in PAPER_KEY) but not in _DETERMINISTIC_MODELS, so their stores do not parse. On dynacell-models the walk raises on them and the audit lists them. The foreign-model predicate would have skipped them as another campaign's arms. A model the generator itself lists in _CODE_TO_PAPER is not foreign: the walk raises and the audit flags it, as on dynacell-models. With this, the audit's error set on the live tree is identical to dynacell-models' (102 entries, compared path by path). Registering wmatch properly is a separate decision (its comment says "evaluate in focused sibling buckets", and adding it to _DETERMINISTIC_MODELS would fold it into the canonical buckets), so it is left as on dynacell-models. Found by /code-review on #534 (findings 2, 7). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(dynacell): stop pointing at the Spotlight leaf generator from this branch evaluation/paths.py and tests/test_paths.py cited tools/generate_spotlight_v2_leaves.py, which this branch does not have, and test_paths claimed a pin to it that only exists with the Spotlight v2 configs. Say where the generator lives instead. The Spotlight model names stay: without them canonical_model_name would resolve an arm onto its baseline. Comments only. Found by /code-review on #534 (finding 8). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(dynacell): parse the warmup-matched pix2pix2d stores in the grouped walk _CODE_TO_PAPER declares pix2pix2d_unetvit_wmatch{,_seed1} a parse-only registration "same rule as fnet2d_voxelmatched", but unlike that model they were never added to _DETERMINISTIC_MODELS. Their 56 stores on the live tree therefore raised "unknown model code-name", which alone stops walk_predictions. Register them like fnet2d_voxelmatched: they now parse, and, as for every parse-only ablation, they are scored in their focused sibling buckets (the committed *__2d_focus2d_pix2pix2d_wmatch leaves), not by regenerating the canonical ones. Fixes part of FOLLOWUPS #1 (live-tree walk crash). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(dynacell): skip the listed checkpoint re-predicts in the grouped walk The live tree holds second-checkpoint predictions of registered models beside their canonical stores, under train-set dirs paths.py does not know: fnet3d_paper/ipsc_step{8400,11k}, fnet2d_voxelmatched/ipsc_step200k, fcmae_vscyto{2d_scratch_samplematched,3d_scratch}/ipsc_final and fcmae_vscyto3d_scratch/ipsc_repredict (34 stores). key_from_prediction_store raises on them ("unknown train_set token 'ipsc_final'"), which stopped walk_predictions and filled the instance-AP audit. They come from the voxel-matched FNet-2D and sample-matched UNeXt2-2D ablations, are scored only by hand-written experiment buckets, and no committed config references them, so they are not canonical inputs. Each (model dir, train-set dir) pair is now listed in _CHECKPOINT_REPREDICTS with its campaign; the walk skips and counts them and the audit skips them. A train-set dir that is neither canonical nor listed still raises. Fixes part of FOLLOWUPS #1 (live-tree walk crash). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(dynacell): name the P1 segcons replicate dirs as foreign in the walk nucleus/pix2pix2d_unetvit_segcons_{s1,ctrl_s1,x10} (12 stores, Spotlight P1 follow-ups of 2026-10-19) sit in the canonical tree but are in neither this generator's registry nor paths.PAPER_KEY, so the PAPER_KEY-based foreign test cannot see them and they raised "unknown model code-name". They are now listed by name in _FOREIGN_MODEL_DIRS and skipped like the PAPER_KEY arms; the skip message no longer says "PAPER_KEY". Any other unknown dir still raises. The new audit test holds one store of each kind on the live tree (registered, checkpoint re-predict, named foreign, wmatch, unknown train set) and expects only the unknown one to be flagged. Fixes the rest of FOLLOWUPS #1 (live-tree walk crash). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(dynacell): list checkpoint re-predicts per organelle _CHECKPOINT_REPREDICTS matched (model dir, train-set dir) in every organelle, so a future mito/fnet3d_paper/ipsc_step8400 store would have been skipped as a known re-predict instead of raising, though only the ER one exists. Entries are now (organelle, model, train set) triples for exactly the stores on the live tree; a listed token in another organelle raises "unknown train_set token" again (new test, fails before this change). The audit test now uses the nucleus step-11k store, since the step-8400 one is ER-only. Found by /code-review on the FOLLOWUPS #1 fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(dynacell): one skip rule for the grouped walk and the audit walk_predictions and audit_prediction_coverage each called is_foreign_model_dir and is_checkpoint_repredict inline, so a third skip category added to one could silently miss the other and split them again. Both now call skip_reason(zarr_path), which names the category; the walk counts per category. No behavior change. Found by /code-review on the FOLLOWUPS #1 fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(dynacell): guard the grouped registries against the walk crash class Two guards on the hand-kept registries behind the live-tree crash: - every _CODE_TO_PAPER model must parse. At a04b3eb this lists pix2pix2d_unetvit_wmatch{,_seed1}, the omission that stopped walk_predictions; - the name-based skip lists stay disjoint from what the campaign registers: no _FOREIGN_MODEL_DIRS entry in _CODE_TO_PAPER or the parse tuples (it would be dropped silently, since the foreign check runs first), and no re-predict token among the canonical train sets. Found by /code-review on the FOLLOWUPS #1 fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(dynacell): reflow the checkpoint re-predict comment The organelle-scoping edit left a broken line wrap in the _CHECKPOINT_REPREDICTS comment. Comment only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(dynacell): keep the wmatch refit out of canonical and instance-AP leaves Registering pix2pix2d_unetvit_wmatch{,_seed1} for parsing (97c30a1) fixed the live-tree walk, but a default regeneration would then also fold their 56 stores into the canonical {organelle}_{train_set} buckets and the instance-AP leaves, against DON'T-FOLD: the committed *__2d_focus2d_pix2pix2d_wmatch buckets are where they are scored. They are now listed in _SIBLING_ONLY_MODELS: walk_predictions still returns them, so sibling buckets can be built from it, but main() buckets canonical_predictions() (the walk minus that list) and the instance-AP in_scope() drops them. New tests: canonical_predictions leaves them out, in_scope rejects them. MEASURED on the live tree, both generators run the default way (no --out-root) emit 0 wmatch blocks, and the canonical pool (628 stores) is identical, store by store, to what the pre-fix rules parse, so a default regeneration writes exactly what a04b3eb would have written had its walk not crashed. Found by /code-review on the FOLLOWUPS #1 fix (finding 2). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(dynacell): retitle the paper and describe DynaCell as the broader framework The camera-ready title is now "Evaluating Dynamic 3D Virtual Staining of Live Cells with DynaCell": DynaCell names a broader framework for image-based profiling of cell-state dynamics, and this paper's evaluation framework is part of it. Update the title in CITATION.cff (software and preferred citation), the README and both HF card BibTeX entries, and the Croissant citeAs string; rewrite the CITATION.cff abstract and the README tagline to describe the evaluation framework as part of DynaCell. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015kuht3npKtb2o5E6h19xbA * docs(dynacell): call the contribution the DynaCell evaluation suite Mirror paper 1c85e84: "This evaluation framework is part of DynaCell, a broader framework" repeated the noun. The paper's contribution is now the DynaCell evaluation suite, and DynaCell stays the broader framework. The CITATION.cff abstract also says "a set of baseline models", so "suite" means one thing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015kuht3npKtb2o5E6h19xbA * docs(dynacell): retitle to The DynaCell Evaluation Suite The paper's title is now "The DynaCell Evaluation Suite for Dynamic 3D Virtual Staining of Live Cells" (Overleaf b73f25f). Updates CITATION.cff, the README and HF card BibTeX, and the Croissant citeAs. test_croissant.py: 17 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015kuht3npKtb2o5E6h19xbA * chore(dynacell): delete finished one-shot tools Each did a one-time job that is done, and nothing else imports them: - build_migration_manifest, run_migration, restandardize_config_paths (+ tests): the canonical-path migration was applied 2026-07-13; a rebuilt manifest on 07-24 had nothing left to move; - retokenize_deconv_predictions (+ test): the 24 a549__deconv / joint__legacy_deconvgt renames were applied 2026-07-22; paths.py still reads both tokens for the dirs kept on purpose; - scan_nan_microssim_caches (+ test): one audit for the cubic 0.9.0a1 NaN MicroMS3IM caches that #506 fixed; - add_instance_ap_to_ablation_leaves: one codemod, already applied; - smoke_joint_leaf: an April CPU smoke the config-composition tests now cover. The provenance builders (cpdino seg masks, HEK stores, A549 dual fuse, random-init checkpoints) stay. README entries and one bake_best_ckpt docstring line follow. Full suite: 5251 passed, 0 failed (the 18 fewer are the two deleted tests/ files; the 14 colocated tool tests were never collected by the root run). Raised in review by edyoshikun. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MzpgExfYMKCgogfnhcSgDc --------- Co-authored-by: Alexandr Kalinin <alxndrkalinin@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

alxndrkalinin and others added 3 commits July 21, 2026 15:23

alxndrkalinin mentioned this pull request Jul 22, 2026

feat(dynacell): 2D in-focus VS model track #487

Merged

alxndrkalinin requested a review from Copilot July 23, 2026 04:32

Copilot started reviewing on behalf of alxndrkalinin July 23, 2026 04:32

View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Comment thread applications/dynacell/src/dynacell/evaluation/metrics.py Resolved

alxndrkalinin and others added 6 commits July 22, 2026 21:47

alxndrkalinin merged commit a556bc9 into dynacell-v1.1 Jul 23, 2026

alxndrkalinin deleted the dynacell-2d branch July 23, 2026 21:24

alxndrkalinin mentioned this pull request Oct 26, 2026

Port shared VisCy package changes from DynaCell v1.1 #533

Merged

Navigation Menu

Repository navigation

feat(dynacell/eval): 2D pixel metrics (FRC) + exclude_fov_names for 2D evaluation - #484

feat(dynacell/eval): 2D pixel metrics (FRC) + exclude_fov_names for 2D evaluation#484
alxndrkalinin merged 10 commits into
dynacell-v1.1from
dynacell-2d

alxndrkalinin commented Jul 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

alxndrkalinin commented Jul 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants