[alxndrkalinin]

What changes

• eval.yaml pixel_metrics.foreground: enabled: true, feather_sigma_um: 0.0 (hard GT interior mask). source: smooth_otsu and smooth_sigma_um: null (per-target smoothing default) are unchanged. Set feather_sigma_um: null for the per-target feathered recipe (metrics.FOREGROUND_SIGMAS_UM).
• Comment above the block, the evaluation README defaults table, and the reporting/tables.py comment no longer say "off by default".
• New test test_default_eval_scores_interior_only_foreground composes the real eval.yaml the way the CLI does (_compose_eval_cfg) and checks enabled is True, feather_sigma_um == 0.0, and that _foreground_settings resolves to {source: smooth_otsu, smooth_sigma_um: 1.0, feather_sigma_um: 0.0} for er. The existing FG tests build configs with OmegaConf.create and cannot see the packaged default, so this test is the only guard on it.

Why

Spotlight v2 judged models on interior-only FG metrics. Whole-image pixel metrics are dominated by background voxels, so the FG columns should be in every eval by default.

What I verified in metrics.py

• foreground_weight builds the weight from the GT alone (smooth, then Otsu, then an optional feather). With feather_sigma_um=0 the Gaussian feather is skipped, so the weight is the binary Otsu mask (1 inside, 0 outside). Constant z-planes (padding) are 0.
• foreground_pixel_metrics never zeroes the images. The weighted means, std, gain, PCC and MSE are all sum(weight * x) / sum(weight), so background voxels get weight 0 and drop out. _weighted_ssim uses normalized convolution: the local statistics come from weight-weighted windows, and the SSIM map is averaged with the weight, so only interior voxels count.
• The whole-image columns are computed separately and are unaffected.
• Every target_name in the tracked configs (er, membrane, mitochondria, nucleus) has an entry in FOREGROUND_SIGMAS_UM. The smooth_sigma_um: null default therefore resolves and does not raise.

No FOREGROUND_METRICS_VERSION bump

Its rule is to bump only when an FG_* value moves for a given recipe, and none does. The recipe (source, sigmas, version, spacing) is already stamped as pixel_foreground in metrics_provenance.json, so the cache gate catches the new default without a bump. A bump would also invalidate every existing FG cache, including the Spotlight-v2 FG buckets.

One-time cost

On its next non-forced run, any existing save dir without a matching pixel_foreground stamp recomputes its final-metrics pass once. That is about 48 single-condition canonical leaves. Grouped leaves already set force_recompute.final_metrics. The Spotlight-v2 FG buckets from generate_fg_buckets.py always write feather_sigma_um explicitly, so the new default does not change their recipe.

Note: the experiment-local trackh/make_eval_leaf.py (untracked, under experiments/2026-09-24_spotlight-v2/) writes only foreground.enabled: true. Leaves it generates after this merges will use the hard mask instead of the per-target feather. Pin feather_sigma_um there if those runs need the feathered recipe.

Merge conditions — met (2026-10-13)

1. feat(dynacell): 2D benchmark on one focus slab — slab3-only predict + focus2d eval #526 merged (996adb4). dynacell-models was merged into this branch in 5867793, with no rebase and no force-push, and the tests were rerun (below).
2. Spotlight-v2 Phase-1 evals closed on 2026-10-10. Phase-B evals that run after this merge get hard-mask FG_* columns in their canonical dirs. Whole-image and seg metrics are unchanged. The FG re-score buckets keep their explicit feather_sigma_um.

Review

• /simplify: 4 fixes.
  • The otsu test is back on a target with no sigma default.
  • One parametrized FG-default test now covers the evaluator's own target list (segmentation.EVAL_TARGETS, which is also used by prepare_segmentation_model).
  • The nucleoli and lysosomes sigmas are derived from mitochondria.
  • The default is stated once.
• /code-review: 2 fixes, the copied (not aliased) sigma dicts and the docs describing the default weight as the hard interior mask. Eight findings were skipped:
  • Intentional and documented: the one-time cache recompute, the code-side enabled fallback for hand-built configs, and the nucleoli/lysosomes defaults Copilot asked for.
  • Measured not to bite:
    • real test volumes are 44 (iPSC) and 48 (A549) planes, so the 11-voxel FG SSIM window fits; whole-image SSIM already fails below 7;
    • FG-on evals ran on A40 with no OOM.
  • Out of scope for this PR: recipe checks in reporting, and a GT weight cache.
• Copilot: re-reviewed at 26a38cd with 0 open findings.

Tests (head 26a38cd)

• The FG-default test fails under two mutations, each checked: setting feather_sigma_um: null in eval.yaml fails 6/6 targets, and dropping the nucleoli/lysosomes defaults fails 2/6.
• Targeted eval and reporting tests: 475 tests, 0 failed, 0 errors.
• Full applications/dynacell (tests + src): 4802 tests, 4603 passed, 2 failed, 0 errors, 197 skipped (junitxml).
  • Both failures are pre-existing on dynacell-models: they fail identically at a23b75c without this PR.
  • test_every_grouped_leaf_is_generated_or_registered_hand_written fails because guard 3ae322e rejects feat(dynacell): 2D benchmark on one focus slab — slab3-only predict + focus2d eval #526's 16 __2d_focus2d buckets.
  • test_registered_hand_written_buckets_all_exist_and_are_unstamped fails because the 26 registered hand-written buckets are untracked and exist only in the main checkout.
• CI has no dynacell job, so these counts are the dynacell evidence.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Gwn9hwgT8uJSoWaCxCAm8J

[Copilot]

🟡 Changes recommended

The new default raises for supported nucleoli and lysosome evaluations because their smoothing defaults are undefined.

1 open finding

• Default evaluator crashes for nucleoli and lysosomes due to missing smoothing d… · New

What changed in this PR

Enables hard-mask foreground metrics by default in DynaCell evaluations.

Changes:

• Enables unfeathered foreground scoring in eval.yaml.
• Updates documentation and reporting comments.
• Adds a composed-default regression test.

File	Description
test_evaluate_compose.py	Tests the resolved ER foreground defaults.
reporting/​tables.py	Updates the foreground-default comment.
evaluation/​README.md	Documents the new defaults.
_configs/​eval.yaml	Enables hard-mask foreground metrics.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

Its stated merge conditions remain unresolved, including the pending #526 integration and Spotlight-v2 evaluations.

0 open findings

1 resolved since last review

• Default evaluator crashes for nucleoli and lysosomes due to missing smoothing d…

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The stated merge dependencies remain pending, including an overlapping change from PR #526.

0 open findings

🧠 Review effort: Balanced