Skip to content

Test metrics - #21

Merged
ziw-liu merged 27 commits into
microdlfrom
test-metrics
Jul 18, 2023
Merged

ziw-liu merged 27 commits into
microdlfrom
test-metrics

Conversation

@ziw-liu

@ziw-liu ziw-liu commented Jul 13, 2023 •

Copy link
Copy Markdown
Contributor

Implement the computation of evaluation metrics as the test stage in the lightning module.

Replaced the custom metrics functions with TorchMetrics. Most of them are GPU-accelerated.

Added mAP and other COCO-style metrics.

The regression metrics are computed on each slice of the input images, and the segmentation metrics are only computed if there is an associated mask. This makes some assumptions about the file name of the masks.

Closes #19.

@ziw-liu ziw-liu added this to the 0.1.0: 2.5D UNet for 3D virtual staining using pytorch lightning milestone Jul 13, 2023
@ziw-liu

ziw-liu commented Jul 14, 2023 •

Copy link
Copy Markdown
Contributor Author

@Soorya19Pradeep The test case for POD_metric still doesn't pass. I'm now seeing precision > 1 for wrong labels:

Edit: sorry it was due to stale cache, all green now.

@ziw-liu ziw-liu added the enhancement New feature or request label Jul 14, 2023
@ziw-liu

ziw-liu commented Jul 14, 2023

Copy link
Copy Markdown
Contributor Author

The CSV logger makes a small annoyance by not saving the config to the log_dir. Users would have to manually move it over. See Lightning-AI/pytorch-lightning#17174

@ziw-liu

ziw-liu commented Jul 14, 2023

Copy link
Copy Markdown
Contributor Author

At test time half precision is not stable enough for metrics involving ops like exponentials or square roots. trainer.precision must be set to 32-true.

@ziw-liu
ziw-liu marked this pull request as ready for review July 15, 2023 06:01
@ziw-liu
ziw-liu merged commit cf0a633 into microdl Jul 18, 2023
@ziw-liu
ziw-liu deleted the test-metrics branch July 18, 2023 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants