[alxndrkalinin]

Summary

Adds a new dynacell evaluate-grouped subcommand that runs the same (model, organelle) across multiple I/O variants — typically the three A549 treatment plates (mock/denv/zikv) — in one Python process, amortizing the segmenter + DINOv3 + DynaCLR + CELL-DINO cold-start (~30–90 s) across all conditions instead of paying it N times.

The driver is opt-in. Existing per-condition runs via dynacell evaluate are unchanged.

Stacked on #426

This branch is based on eval-parallelism (PR #426). The diff here is only this feature's 3 commits; #426 should land first. The base PR's EvalModels extraction was the prerequisite — that's why c271d3f0 consolidates the dual init_cache_context call sites and the inline-load duplication in precompute_cli.py behind the same bundle.

Commits

SHA	Title
d69d9709	feat: grouped multi-condition eval driver (model_loader extract + evaluate_predictions(..., models=) kwarg + evaluate_predictions_grouped + Hydra entry)
c271d3f0	refactor: simplify grouped driver (lazy load on all-cache-hit; symmetric condition-0 invariant; LoadFlags consolidates precompute_cli load; init_cache_contexts helper)
e84ed6a1	feat: tier process-mode message by cache state (mild note for require_complete_cache=true; loud WARNING for the cache-miss × N-conditions × N-workers worst case)

Performance characteristics

The amortization win depends on the executor mode:

• runtime.executor=serial (recommended for grouped): the parent's EvalModels bundle is passed to evaluate_predictions(..., models=...) for every condition. Models load once total. Full amortization.
• runtime.executor=process: only the parent-side pre-warm is amortized; each condition spawns a fresh ProcessPoolExecutor whose workers re-load their own model copies. The driver detects this and tiers its output:
  • • require_complete_cache=true + n_conditions > 1 → mild informational note (workers don't actually load any models on the cache-only path, so re-init is cheap)
  • • require_complete_cache=false + n_conditions > 1 → loud multi-line WARNING with explicit cost estimate and pointer to executor=serial

Constraints on per-condition overlays

Per-condition entries may freely override io.*, save.*, runtime.*, limit_positions, force_recompute.*, and carry a name label. They MUST NOT change target_name, feature_extractor.*, compute_feature_metrics, or use_gpu — those gate which models get loaded. The invariant check is symmetric across all conditions (including condition 0), so a model-loading override smuggled into the first condition is rejected, not silently re-baselined.

Example config

# eval_grouped_a549_mantis_er.yml
defaults:
  - eval_grouped
  - _self_

target_name: er
compute_feature_metrics: true
feature_extractor: { ... }   # shared across all conditions

conditions:
  - name: a549_mock
    io: { pred_path: ..., gt_path: ..., gt_cache_dir: ..., pred_cache_dir: ..., cell_segmentation_path: ... }
    save: { save_dir: /path/to/out/a549_mock }
  - name: a549_denv
    io: { ... }
    save: { save_dir: /path/to/out/a549_denv }
  - name: a549_zikv
    io: { ... }
    save: { save_dir: /path/to/out/a549_zikv }

Run:

uv run dynacell evaluate-grouped -c eval_grouped_a549_mantis_er.yml

Test plan

• tests/test_evaluation_grouped.py (NEW, 6 tests):
  • test_grouped_matches_sequential_per_condition — byte-equal parity vs sequential per-condition runs (pixel CSV, mask CSV, seg plate)
  • test_grouped_rejects_model_loading_field_overrides — condition N>0 sneaking in target_name raises
  • test_grouped_rejects_condition0_model_field_override — condition 0 sneaking in target_name raises (regression guard for symmetric baseline)
  • test_grouped_rejects_empty_conditions — missing/empty conditions list raises
  • test_grouped_warns_loudly_on_process_plus_cache_miss — WARNING fires on the bad-config combo
  • test_grouped_mild_note_on_process_plus_cache_only — only mild note on cache-only path
• tests/test_evaluation_precompute_cli.py — covers the unified LoadFlags/load_eval_models path now used by precompute-gt
• All 104 tests in affected eval suites pass locally
• Pre-existing 48-test baseline failures (test_benchmark_config_composition, one test_evaluation_metrics case) unchanged
• .github/workflows/test.yml extended to run test_evaluation_grouped.py in the test-dynacell-configs job
• CI green on PR

Files touched

• NEW: applications/dynacell/src/dynacell/evaluation/model_loader.py
• NEW: applications/dynacell/src/dynacell/evaluation/_configs/eval_grouped.yaml
• NEW: applications/dynacell/tests/_eval_fixtures.py (extracted from test_evaluation_pipeline_parallel_cpu.py so the new test imports normally instead of sys.path.insert)
• NEW: applications/dynacell/tests/test_evaluation_grouped.py
• MODIFIED: applications/dynacell/src/dynacell/evaluation/pipeline.py (kwarg + grouped driver + cache-load helper extraction)
• MODIFIED: applications/dynacell/src/dynacell/evaluation/precompute_cli.py (uses unified load_eval_models + init_gt_cache_context)
• MODIFIED: applications/dynacell/src/dynacell/__main__.py (registers evaluate-grouped)
• MODIFIED: applications/dynacell/CLAUDE.md (new "Grouped multi-condition eval" subsection)
• MODIFIED: applications/dynacell/tests/test_evaluation_pipeline_parallel_cpu.py (imports from _eval_fixtures)
• MODIFIED: .github/workflows/test.yml (adds grouped test to CI matrix)

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Adds an opt-in “grouped multi-condition” evaluation path to Dynacell so multiple condition-specific I/O variants can be evaluated in a single Python process, amortizing model/extractor cold-start across conditions while keeping existing dynacell evaluate behavior unchanged.

Changes:

• Introduces dynacell evaluate-grouped + Hydra eval_grouped config, with evaluate_predictions_grouped() driving per-condition overlays and (optionally) shared preloaded models.
• Extracts unified model-loading and cache-context initialization into evaluation/model_loader.py, reused by both evaluation and precompute-gt.
• Adds a shared cache-only eval fixture module and new grouped-driver tests; updates CI to run the new suite.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
applications/dynacell/src/dynacell/evaluation/pipeline.py	Adds models= kwarg to evaluate_predictions, implements evaluate_predictions_grouped, and factors cached-metrics reload helper.
applications/dynacell/src/dynacell/evaluation/model_loader.py	New centralized loader for seg/extractor models + helpers for cache context initialization.
applications/dynacell/src/dynacell/evaluation/precompute_cli.py	Switches GT precompute to the unified model loader + GT cache-context helper.
applications/dynacell/src/dynacell/evaluation/_configs/eval_grouped.yaml	New Hydra config template describing the conditions: overlay list for grouped eval.
applications/dynacell/src/dynacell/main.py	Registers the evaluate-grouped CLI subcommand.
applications/dynacell/tests/_eval_fixtures.py	New shared tiny HCS + cache-only fixture builder utilities for integration tests.
applications/dynacell/tests/test_evaluation_grouped.py	New grouped-vs-sequential parity + invariant/warning behavior tests.
applications/dynacell/tests/test_evaluation_pipeline_parallel_cpu.py	Refactors to reuse _eval_fixtures helpers instead of inline definitions.
applications/dynacell/CLAUDE.md	Documents grouped eval usage, constraints, and process-mode caveats.
.github/workflows/test.yml	Extends the dynacell configs job to run the new grouped evaluation tests.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.