Repository navigation
Port virtual_staining examples into applications/cytoland - #409
Merged
edyoshikun merged 47 commits intoMay 18, 2026
Merged
Conversation
Tutorial-style scripts under applications/*/examples use jupytext-style percent cells, markdown docstrings, and long URL references that trip D205/D400/D100/D103 and E501. Treat them like notebooks and tests.
Bring back the four VSCyto inference demos (VSCyto2D, VSCyto3D, VSNeuromast, and TTA-augmented) plus plot.py helper from examples/virtual_staining/VS_model_inference on main. Imports are updated to the modular package layout: - viscy.data.hcs -> viscy_data.hcs - viscy.transforms -> viscy_transforms - viscy.trainer -> viscy_utils.trainer - viscy.translation.engine -> cytoland.engine - viscy.translation.predict_writer -> viscy_utils.callbacks
Bring back the DL@MBL 2024 image-translation exercise (solution.py, README, setup.sh, prepare-exercise.sh) from examples/virtual_staining/dlmbl_exercise on main. Notebooks are not ported; regenerate from solution.py via jupytext if needed. Imports are updated to the modular package layout: - viscy.data.hcs -> viscy_data.hcs - viscy.transforms -> viscy_transforms - viscy.trainer -> viscy_utils.trainer - viscy.translation.engine.VSUNet -> cytoland.engine.VSUNet - viscy.translation.engine.MixedLoss -> viscy_utils.losses.MixedLoss - viscy.translation.evaluation_metrics.mean_average_precision -> viscy_utils.evaluation.metrics.mean_average_precision setup.sh now installs applications/cytoland[metrics] editable instead of the legacy top-level viscy[metrics,visual,examples]>=0.2 wheel, plus cellpose and torchview as extra tutorial dependencies.
Tutorial-style scripts routinely import late inside % cells (E402) and reference notebook builtins like get_ipython without importing them (F821). Extending the per-file-ignores so the vcp tutorial scripts lint cleanly alongside the existing D and E501 exemptions.
Bring back the Virtual Cell Platform tutorials (quick_start.py, hek293t.py, neuromast.py, README.md) from examples/virtual_staining/vcp_tutorials on main. Notebooks are not ported; regenerate from .py via jupytext if needed. quick_start.py imports are updated to the modular package layout: - viscy.data.hcs -> viscy_data.hcs - viscy.transforms -> viscy_transforms - viscy.trainer -> viscy_utils.trainer - viscy.translation.engine.FcmaeUNet -> cytoland.engine.FcmaeUNet - viscy.translation.predict_writer -> viscy_utils.callbacks hek293t.py and neuromast.py have no viscy Python imports (they demonstrate the viscy preprocess / viscy predict CLIs), so no code changes are required. The commented-out pip install "viscy[...]" hints are left as-is for historical reference.
Add a Tutorials and demos section listing VS_model_inference, vcp_tutorials, dlmbl_exercise, and configs, so the newly ported examples are discoverable from the cytoland landing page.
Bring back the phase-contrast virtual staining tutorial (solution.py, README, setup.sh, prepare-exercise.sh) from examples/virtual_staining/phase_contrast on main. Notebooks are not ported; regenerate from solution.py via jupytext if needed. Imports are updated to the modular package layout: - viscy.data.hcs -> viscy_data.hcs - viscy.transforms -> viscy_transforms - viscy.trainer -> viscy_utils.trainer - viscy.translation.engine.VSUNet -> cytoland.engine.VSUNet setup.sh now installs applications/cytoland[metrics] editable instead of the legacy top-level viscy[metrics,visual,examples]>=0.2 wheel.
Add a concise Lightning primer before Part 1 (three-object mental model: LightningDataModule, LightningModule, Trainer; what trainer.fit replaces) so learners unfamiliar with Lightning can follow the rest of the exercise. Expand inline explanations at the three points where Lightning concepts first appear in code: - HCSDataModule section: describe the DataModule role and the exact dict shape (source, target, index) yielded to training_step. - VSUNet instantiation: frame the class as the LightningModule that bundles network, loss, and per-batch logic; gloss lr, schedule, freeze_encoder, log_batches_per_epoch. - Trainer constructors: annotate fast_dev_run, accelerator, devices, precision="16-mixed", max_epochs, log_every_n_steps, and TensorBoardLogger; spell out what trainer.fit does internally so learners see it replacing a hand-written training loop. Markdown only, no code changes.
Six concise additions for learners new to deep learning and Lightning: 1. Typos: fluoresecence -> fluorescence, componets -> components, Person Correlation -> Pearson Correlation. 2. Tighten the OME-Zarr / HCS section: one-paragraph primer on the row/col/field/level/T/C/Z/Y/X hierarchy before open_ome_zarr is called. 3. Add an augmentation motivation table mapping each MONAI transform to the real-world microscope variation it simulates. 4. Add a pre-model markdown cell explaining the UNeXt2 config (encoder_blocks, dims, decoder_conv_blocks, stem_kernel_size, in_stack_depth) in U-Net terms. 5. Same cell explains MixedLoss (L1 + MS-SSIM tradeoff) and the WarmupCosine LR schedule. 6. Part 2 opener distinguishes regression vs segmentation metric families; fill in the Task 2.1 TODO with real Pearson/SSIM definitions tied to the image-translation setting. Markdown only, no code changes.
Issue dl-janelia/image_translation#16: - Fix the nested duplicate for-loop after Task 1.1 that shadows the loop variable (single loop now). - Replace the broken viscy.transforms source link with authoritative MONAI docs links for RandAffined and RandGaussianNoised. - Add explicit path + public download URL for the VSCyto2D pretrained checkpoint next to Task 2.2 so students don't guess where it went. - Switch spatial dimension labels from (D, H, W) to (Z, Y, X) in Part 3 Gaussian-blur tasks to match the convention used everywhere else in the exercise. - Note MicroSSIM as a microscopy-appropriate SSIM variant in the metrics primer. - Warn students to restart the kernel before re-running training to release GPU memory (CUDA OOM prevention).
Rewrite setup.sh to use uv instead of conda so students can provision the exercise without a working conda install. The script now: - Installs uv via the official installer if missing. - Creates a Python 3.11 venv at .venv/ inside the exercise folder. - Installs cytoland editable from the monorepo plus the tutorial extras (cellpose, torchview, jupyter, ipykernel, ipywidgets, jupytext, nbformat, nbconvert) into that venv. - Registers the venv as a Jupyter kernel named 06_image_translation so VSCode and JupyterLab both surface it by default. - Downloads the training / test OME-Zarr stores and the VSCyto2D pretrained checkpoint into ~/data/06_image_translation/. Script is bash strict-mode (set -euo pipefail) and resolves the monorepo root relative to the script path so it works regardless of the caller's cwd. README updated to match the new flow and document the VSCode / Jupyter kernel selection.
Classic skimage SSIM assumes natural-image dynamic range and scores collapse into a narrow band on sparse, dim, noisy fluorescence microscopy predictions — so the metric can barely rank good vs bad outputs. microSSIM (Ashesh et al. 2024, arXiv:2408.08747) subtracts background and fits a per-image rescale before running SSIM, which restores sensitivity over the intensity range microscopy predictions actually live in. - Replace skimage.metrics.structural_similarity with microssim.micro_structural_similarity at all 6 call sites (two metrics blocks: single-model and phase2fluor vs pretrained). - Drop the now-unused from skimage import metrics import. - Rename DataFrame columns SSIM_nuc / SSIM_mem to microSSIM_nuc / microSSIM_mem so plots and saved CSVs name the metric correctly. - Rewrite the Part 2 Task 2.1 metric definition to explain WHY microSSIM instead of SSIM for microscopy. - Add microssim to setup.sh install list and to the README install summary.
setup_TA.sh stages data + checkpoint to a shared DATA_ROOT (no env). setup_student.sh creates the per-user venv + kernel and skips the download when DATA_ROOT already has the data.
setup_student.sh now defaults to python 3.13 and installs cytoland + viscy from PyPI when run outside a VisCy monorepo clone. When the monorepo is detected (and root pyproject is the viscy umbrella), it falls back to the existing editable workspace install. README updated to match.
- Replace stale torch.no_grad() with torch.inference_mode() in the prediction-visualization block to match the rest of the file. - Renumber the first 'Task 1.5' (model instantiation) to 'Task 1.4'. The exercise was missing a Task 1.4, leaving two Task 1.5s. - Update the source-code reference under that task to point at VSUNet and the fcmae network (the Unet2D link was obsolete).
Three small wording fixes ported from the DL@Janelia version of this exercise where it had clearer phrasing: - Task 1.1 hint: 'what are your options' -> 'what your options are'. - Task 2.4: add the missing alert heading and move the section header outside the alert box (was structurally mismatched). - Task 2.4 question: typo fix 'How do yout model' -> 'How does your model'.
Adds the Task 2.5 section that walks students through the inverse translation: predicting QPI from nuclei + membrane fluorescence using a pretrained model and Test-Time Augmentation. Inserted between Task 2.4 and Part 3. setup_TA.sh and setup_student.sh now also stage the fluor2phase_step668.ckpt checkpoint (previously commented out in TA, absent in student) since Task 2.5 loads it. The hardcoded /mnt/efs/... paths from the original Janelia version are replaced with the top_dir-based DATA_ROOT layout used elsewhere in this exercise.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR ports virtual staining tutorials/demos into applications/cytoland/examples, updates modular imports, adds setup guidance, and relaxes linting for example-style scripts.
Changes:
- Adds Cytoland example/tutorial trees for model inference, VCP tutorials, phase contrast, and DL course exercise.
- Updates documentation to point users to the new examples.
- Adds Ruff per-file ignores for example scripts.
Reviewed changes
Copilot reviewed 19 out of 20 changed files in this pull request and generated 39 comments.
Show a summary per file
| File | Description |
|---|---|
pyproject.toml |
Adds Ruff ignores for examples. |
applications/cytoland/README.md |
Adds tutorials/demos section. |
applications/cytoland/examples/VS_model_inference/plot.py |
Adds plotting helper. |
applications/cytoland/examples/VS_model_inference/demo_vsneuromast.py |
Adds VSNeuromast inference demo. |
applications/cytoland/examples/VS_model_inference/demo_vscyto3d.py |
Adds VSCyto3D inference demo. |
applications/cytoland/examples/VS_model_inference/demo_vscyto2d.py |
Adds VSCyto2D inference demo. |
applications/cytoland/examples/VS_model_inference/demo_vscyto_w_ttas.py |
Adds TTA/sliding-window inference demo. |
applications/cytoland/examples/vcp_tutorials/README.md |
Adds VCP tutorial index. |
applications/cytoland/examples/vcp_tutorials/quick_start.py |
Adds Cytoland quick-start tutorial. |
applications/cytoland/examples/vcp_tutorials/neuromast.py |
Adds neuromast CLI/analysis tutorial. |
applications/cytoland/examples/vcp_tutorials/hek293t.py |
Adds HEK293T CLI/analysis tutorial. |
applications/cytoland/examples/phase_contrast/solution.py |
Adds phase contrast virtual staining demo. |
applications/cytoland/examples/phase_contrast/setup.sh |
Adds phase contrast setup/download script. |
applications/cytoland/examples/phase_contrast/README.md |
Adds phase contrast usage instructions. |
applications/cytoland/examples/phase_contrast/prepare-exercise.sh |
Adds notebook generation helper. |
applications/cytoland/examples/dl-course-exercise/solution.py |
Adds DL course exercise script. |
applications/cytoland/examples/dl-course-exercise/setup_TA.sh |
Adds TA data staging script. |
applications/cytoland/examples/dl-course-exercise/setup_student.sh |
Adds student venv/data setup script. |
applications/cytoland/examples/dl-course-exercise/README.md |
Adds DL course exercise instructions. |
applications/cytoland/examples/dl-course-exercise/prepare-exercise.sh |
Adds exercise/solution notebook generation helper. |
Comments suppressed due to low confidence (5)
applications/cytoland/examples/VS_model_inference/demo_vsneuromast.py:87
HCSDataModuledoes not accept anarchitecturekeyword in the current modular API (packages/viscy-data/src/viscy_data/hcs.py:104-132), so this example raisesTypeErrorbefore prediction starts. Remove this argument and keep the architecture selection on the model only.
architecture="UNeXt2",
applications/cytoland/examples/VS_model_inference/demo_vscyto3d.py:90
HCSDataModuledoes not accept anarchitecturekeyword in the current modular API (packages/viscy-data/src/viscy_data/hcs.py:104-132), so this example raisesTypeErrorbefore prediction starts. Remove this argument and keep the architecture selection on the model only.
architecture="UNeXt2",
applications/cytoland/examples/VS_model_inference/demo_vscyto2d.py:82
HCSDataModuledoes not accept anarchitecturekeyword in the current modular API (packages/viscy-data/src/viscy_data/hcs.py:104-132), so this example raisesTypeErrorbefore prediction starts. Remove this argument and keep the architecture selection on the model only.
architecture="fcmae",
applications/cytoland/examples/vcp_tutorials/quick_start.py:91
- This setup command still installs the retired monolithic
viscy[metrics,visual]package, but the tutorial importscytoland.engine,viscy_data,viscy_transforms, andviscy_utils. A user who follows the notebook setup from a clean environment can end up without the modular packages needed by the code below; update the install instruction to install cytoland/the modular workspace instead.
# !pip install "viscy[metrics,visual]==0.4.0a3"
applications/cytoland/examples/dl-course-exercise/README.md:127
- The README says to run cells top to bottom, but the raw
solution.pyincludes task cells with placeholders that are only stripped byprepare-exercise.shwhen generatingsolution.ipynb. Following these steps directly onsolution.pywill fail before reaching the solution cells.
--display-name "Python (06_image_translation)"
References
</details>
---
💡 <a href="/mehta-lab/VisCy/new/modular-viscy-staging?filename=.github/instructions/*.instructions.md" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Add Copilot custom instructions</a> for smarter, more guided reviews. <a href="https://docs.github.com/en/copilot/customizing-copilot/adding-repository-custom-instructions-for-github-copilot" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Learn how to get started</a>.
| # %% [markdown] | ||
| """ | ||
| # Prerequisites | ||
| Python>=3.11 |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| # Run this from the root of the VisCy monorepo checkout. | ||
| # Find path to the environment - conda activate doesn't work from within shell scripts. | ||
| ENV_PATH=$(conda info --envs | grep vs_Phc | awk '{print $NF}') | ||
| $ENV_PATH/bin/pip install -e "applications/cytoland[metrics]" |
Comment on lines
+55
to
+58
| with open_ome_zarr(path) as ds: | ||
| vol_np = np.asarray(ds.data[0:1, 0:1]) # (1, 1, Z, Y, X) | ||
|
|
||
| vol = torch.from_numpy(vol_np).float().to(DEVICE) |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
HCSDataModule's __init__ (packages/viscy-data/src/viscy_data/hcs.py:104) does not accept an `architecture` parameter; the demos were raising TypeError before prediction. Architecture selection is on VSUNet only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The in-memory sliding-window path was feeding the raw phase volume to the model while the CLI prediction path normalizes Phase3D with the precomputed FOV median / IQR (NormalizeSampled). Read those stats from .zattrs and apply them inline so this demo produces the same results as `viscy predict`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
setup.sh used "applications/cytoland[metrics]" which only resolved
correctly when invoked from the monorepo root. The README instructs
users to `source setup.sh` from the example directory, where that
path doesn't exist. Compute the cytoland directory from
${BASH_SOURCE[0]} so the install works regardless of cwd.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…icit VisCyTrainer was imported but never instantiated. Remove the dead import and chain `.eval()` after `.to(inference_device)` for both checkpoints so it's obvious the models run on GPU. Add a print of the resolved device for visibility. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- top_dir now points directly at the data root populated by
setup_student.sh / setup_TA.sh ($DATA_ROOT/{training,test,pretrained_models}),
defaulting to ~/data/06_image_translation. Drop the dangling
"06_image_translation/" subpath everywhere top_dir is used.
- Move the TTA implementation into the same solution cell as the
TTA metrics so the generated solution.ipynb runs top-to-bottom
(task cells are stripped, so the later TTA solution cell was
unreachable before tta_pred_nuc/mem were referenced).
- Add the first-use imports (skimage.metrics, rescale_intensity)
to the fluor->phase metrics cell so it no longer raises NameError.
- VSUNet.load_from_checkpoint does not accept `accelerator=` (it is
forwarded to VSUNet.__init__ and raises TypeError). Drop the kwarg
and move the loaded module to GPU with .to(...) instead.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…k cells Reorder the "Run the exercise" section to lead with the notebook workflow (which strips task-tagged placeholder cells) and demote the raw solution.py path to an advanced/VSCode option with an explicit warning that task-tagged cells contain TODO/... placeholders and must be skipped manually. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
alxndrkalinin
added a commit
that referenced
this pull request
May 18, 2026
Brings in 4 commits from staging: - 651f969 Port virtual_staining examples into applications/cytoland (#409) - 0c5b834 bump iohub to 0.3.3 - 737cedf Port to iohub 0.3.2 ImageArray API (#407) - a10d4c4 Update packages/viscy-data/pyproject.toml Resolutions: - packages/viscy-data/pyproject.toml: kept staging's iohub>=0.3.3 floor; kept the lightning>=2.3 SECURITY comment from dynacell-models warning against 2.6.2/2.6.3 supply-chain compromise (matches viscy-utils). - packages/viscy-data/src/viscy_data/sliding_window.py: kept dynacell-models' .to(torch.float32, copy=True) on the preloaded read path so it produces the same dtype as the non-preloaded path's .astype(np.float32) below. .clone() would leave the preloaded tensor in its source dtype. - uv.lock: regenerated with `uv lock` from the merged pyproject set; iohub git pin (afe11f6, PR #408 merge commit on iohub main) survives the regen. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
examples/virtual_staining/tree frommainintoapplications/cytoland/examples/on top ofmodular-viscy-staging, covering four tutorial groups: VS_model_inference, dlmbl_exercise, vcp_tutorials, phase_contrast.viscy.*import to the modular package layout (viscy_data,viscy_transforms,viscy_utils,cytoland.engine,viscy_utils.callbacks.HCSPredictionWriter,viscy_utils.losses.MixedLoss,viscy_utils.evaluation.metrics.mean_average_precision).setup.shin dlmbl_exercise and phase_contrast topip install -e "applications/cytoland[metrics]"instead of the retired top-levelviscy[metrics,visual,examples]>=0.2wheel.**/examples/**to[tool.ruff] per-file-ignoresforD,E402,E501,F821so jupytext percent-cell scripts lint cleanly without forcing docstrings, top-of-file imports, short lines, or importing notebook builtins.applications/cytoland/README.mdpointing at the new tree.Notebooks (
.ipynb) are intentionally not ported — regenerate fromsolution.py/*.pywithjupytext --to ipynbwhen needed.Import migration
viscy.data.hcsviscy_data.hcsviscy.transformsviscy_transformsviscy.trainerviscy_utils.trainerviscy.translation.engine.{VSUNet, FcmaeUNet, AugmentedPredictionVSUNet}cytoland.engine.{...}viscy.translation.engine.MixedLossviscy_utils.losses.MixedLossviscy.translation.predict_writer.HCSPredictionWriterviscy_utils.callbacks.HCSPredictionWriterviscy.translation.evaluation_metrics.mean_average_precisionviscy_utils.evaluation.metrics.mean_average_precisionTest plan
python -c "import ast; ast.parse(open(f).read())"passes for all ported.pyfilesruff check+ruff formatpass via pre-commit on every commituv run --package cytoland python -c "from cytoland.engine import ...; from viscy_data.hcs import ...; ..."resolves every ported symbolpublic.czbiohub.org/comp.micro/viscyand a GPU; not part of this PR)