feat: add class_names support to Precision, Recall, FBeta metrics by aniketpandey05 · Pull Request #3732 · pytorch/ignite

aniketpandey05 · 2026-04-14T11:35:07Z

Description:
Adds an optional class_names parameter to _BasePrecisionRecall, allowing compute() to return a labeled dict instead of an unnamed tensor when average=False or average=None. Useful for per-class metric tracking where knowing which score belongs to which class matters for logging and visualization.

Check list:

New tests are added (if a new feature is added)
New doc strings: description and/or example code are in RST format
Documentation is updated (if required)

aaishwarymishra · 2026-04-15T05:51:13Z

ok so like wont this code break fbeta metric when precision and recall have average=False and they have class_names, we need to update the fbeta too to support this.

There can be more metrics that can benefit from this change. I am not sure though.

aniketpandey05 · 2026-04-16T12:08:15Z

@aaishwarymishra would raising errors for those condition a better option or should we add full class_name support to fbeta

aaishwarymishra · 2026-04-16T15:00:41Z

I am not sure, adding support for the class names in fbeta would be easier, but fbeta uses the MetricsLambda for overloading operator we can update them, I am not sure @vfdev-5 what you think would be appropriate?

steaphenai · 2026-04-20T19:56:15Z

Could we please clarify Fbeta behavior when class_names is used with average=False/None

vfdev-5 · 2026-04-21T08:26:49Z

I am not sure, adding support for the class names in fbeta would be easier, but fbeta uses the MetricsLambda for overloading operator we can update them, I am not sure @vfdev-5 what you think would be appropriate?

If we can compute F1 score per class than we should propagate the dict structure until the output of the F1 otherwise, take .values() of the dict output from precision and recall.

aniketpandey05 · 2026-04-27T14:46:50Z

Hey @vfdev-5 , would adding a helper in metric.py to handle dict arithmetic in all operator overloads so the dict propagates through to the Fbeta output be a good approach? Also when average=True and class_names is set, should Fbeta return a scalar float or propagate the dict as is?

vfdev-5 · 2026-05-03T16:10:31Z

Hey @vfdev-5 , would adding a helper in metric.py to handle dict arithmetic in all operator overloads so the dict propagates through to the Fbeta output be a good approach? Also when average=True and class_names is set, should Fbeta return a scalar float or propagate the dict as is?

I do not know, what would you suggest? Let's try to make it the most simple possible and intuitively clear.

aniketpandey05 · 2026-05-21T15:52:45Z

hey @vfdev-5 I think Fbeta should return a per-class dict if class_names is set, regardless of average.

vfdev-5 · 2026-05-21T16:16:36Z

@rogueslasher ok, let's implement that this way and see if this works as expected

aniketpandey05 · 2026-06-09T10:08:27Z

Hey @vfdev-5, would adding a _dict_aware_op helper in metric.py and updating all operator overloads to use it for element-wise dict arithmetic, and skipping average in fbeta.py when class_names is set be a good idea? It touches all operator overloads in metric.py which affects all metrics.

i am worried since it touches all operators overloads

aniketpandey05 · 2026-06-17T17:21:07Z

@aaishwarymishra ,would love your help in this can you look into it more please

aaishwarymishra · 2026-06-19T12:47:38Z

@rogueslasher well , I am not sure to be honest i wont touch the metric operators as it kind of affect every metric, for a case which can be only relevant for very few metrics, I think we should modify Fbeta directly.

aniketpandey05 · 2026-06-30T13:49:26Z

@aaishwarymishra the header rules and other three preview checks are taking insane amount of time . is there any issue

aaishwarymishra · 2026-06-30T14:17:25Z

Documentation issue :)

…_names

aniketpandey05 · 2026-07-01T09:38:09Z

@aaishwarymishra

…a metrics

aniketpandey05 · 2026-08-01T15:23:26Z

@aaishwarymishra

aaishwarymishra · 2026-08-03T06:45:30Z

Also there is no early check for if average is True and also the class name has been given

Is this resolved?

… check in Fbeta

aniketpandey05 · 2026-08-03T10:51:02Z

Hi @aaishwarymishra, thanks for the review! I've updated the implementation:

aaishwarymishra · 2026-08-04T09:35:00Z

Also try to reslove previous comments it gets hard to read the code :)

…metrics pattern

aniketpandey05 · 2026-08-10T15:29:59Z

@aaishwarymishra i think the ci failure are unrelated to the changed files

aaishwarymishra · 2026-08-10T16:43:02Z

@vfdev-5 looks good to me, you can check it too and merge :)

vfdev-5

I haven't yet checked the code itself but docstring needs some updates.

@aaishwarymishra resolve all open comments if they were addressed.

vfdev-5 · 2026-08-10T17:42:14Z

Is this resolved?

vfdev-5 · 2026-08-10T17:43:16Z

        device: specifies which device updates are accumulated on. Setting the metric's
            device to be the same as your ``update`` arguments ensures the ``update`` method is non-blocking. By
            default, CPU.
+        class_names: list of class name strings used to label per-class output. Default: ``None``.


Docstring should also contain .. version added tag for new args

aaishwarymishra

@aniketpandey05 can you update docstrings too :)

vfdev-5 · 2026-08-12T05:56:35Z

            default, CPU.
+        class_names: list of class name strings used to label per-class output. Default: ``None``.
+
+            .. versionadded:: 0.6.0


This is a wrong tag, check Contributing guide and also the indent size.

vfdev-5 · 2026-08-12T05:59:37Z

+    target_class_names = class_names
+    if target_class_names is None and precision is not None:
+        target_class_names = precision._class_names
+    if target_class_names is None and recall is not None:
+        target_class_names = recall._class_names


Suggested change

target_class_names = class_names

if target_class_names is None and precision is not None:

target_class_names = precision._class_names

if target_class_names is None and recall is not None:

target_class_names = recall._class_names

if class_names is None and precision is not None:

class_names = precision._class_names

if class_names is None and recall is not None:

class_names = recall._class_names

vfdev-5 · 2026-08-12T05:59:58Z

            output_transform=(lambda x: x) if output_transform is None else output_transform,
            average=False,
            device=cast(str | torch.device, recall._device if recall else device),
+            class_names=target_class_names,


Suggested change

class_names=target_class_names,

class_names=class_names,

vfdev-5 · 2026-08-12T06:00:12Z

            output_transform=(lambda x: x) if output_transform is None else output_transform,
            average=False,
            device=cast(str | torch.device, precision._device if precision else device),
+            class_names=target_class_names,


Suggested change

class_names=target_class_names,

class_names=class_names,

vfdev-5 · 2026-08-12T06:03:40Z

+    if any(m._average for m in active_metrics):
+        raise ValueError("Input precision and recall metrics should have average=False")


This new error message is not precise enough. We should explicitly tell which metric exactly should have average false and not give a generic message

vfdev-5 · 2026-08-12T06:08:27Z


    @sync_all_reduce("_numerator", "_denominator")
-    def compute(self) -> torch.Tensor | float:
+    def compute(self) -> torch.Tensor | float | dict:


Can you make dict type hint more detailed: dict[str, <whatever type is here>]

vfdev-5 · 2026-08-12T06:12:22Z

+            name to its metric value instead of a tensor. Must match the number of classes inferred
+            from the data. Default: ``None``.
+
+            .. versionadded:: 0.6.0


this is a wrong tag, it should not be added here.

vfdev-5 · 2026-08-12T06:14:28Z

+            name to its metric value instead of a tensor. Must match the number of classes inferred
+            from the data. Default: ``None``.
+
+            .. versionadded:: 0.6.0


Suggested change

.. versionadded:: 0.6.0

vfdev-5 · 2026-08-12T06:14:59Z

        Fbeta(0.0)

-    with pytest.raises(ValueError, match=r"Input precision metric should have average=False"):
+    with pytest.raises(ValueError, match=r"Input precision and recall metrics should have average=False"):


Keep this test as it was and update the code

vfdev-5 · 2026-08-12T06:15:13Z

        Fbeta(1.0, precision=p)

-    with pytest.raises(ValueError, match=r"Input recall metric should have average=False"):
+    with pytest.raises(ValueError, match=r"Input precision and recall metrics should have average=False"):


vfdev-5 · 2026-08-12T06:18:18Z

+    with pytest.raises(ValueError, match="Input precision and recall metrics should have average=False"):
+        Fbeta(beta=1.0, average=False, precision=p_avg)
+
+    # Correct computation passing class_names directly to Fbeta


I prefer that these test cases were added to existing tests as new parametrizations.

feat: add class_names support to Precision and Recall metrics

8d511c4

github-actions Bot added the module: metrics Metrics module label Apr 14, 2026

vfdev-5 reviewed Apr 15, 2026

View reviewed changes

Comment thread ignite/metrics/precision.py Outdated

feedback: Validate class_names length in update() instead of compute()

9185199

aniketpandey05 added 5 commits June 27, 2026 17:06

Merge branch 'master' into feature/class-names-precision-recall

fd9bfdc

feat: add class_names support to Fbeta metric

372021f

feat: add class_names support to Precision and Recall metrics

1cb1e2d

feedback: Validate class_names length in update() instead of compute()

7b890cb

feat: add class_names support to Fbeta metric

7ea2eb5

aniketpandey05 force-pushed the feature/class-names-precision-recall branch from 61e7680 to 7ea2eb5 Compare June 30, 2026 13:48

aniketpandey05 added 2 commits July 1, 2026 14:51

fix: update return type annotation and docstring references for class…

2a3afea

…_names

fix: update return type annotation and docstring references for class…

b44a494

…_names

aaishwarymishra reviewed Jul 1, 2026

View reviewed changes

Comment thread ignite/metrics/recall.py Outdated

Comment thread ignite/metrics/recall.py

Comment thread tests/ignite/metrics/test_precision.py

feat(metrics): add class_names support to Precision, Recall, and Fbet…

817f33b

…a metrics

aniketpandey05 force-pushed the feature/class-names-precision-recall branch from 9c71695 to 817f33b Compare August 1, 2026 15:14

aniketpandey05 requested review from aaishwarymishra and vfdev-5 August 1, 2026 16:02

aaishwarymishra reviewed Aug 3, 2026

View reviewed changes

fix(metrics): enforce explicit class_names matching and early average…

7e12121

… check in Fbeta

aniketpandey05 force-pushed the feature/class-names-precision-recall branch from 3a93092 to 7e12121 Compare August 3, 2026 10:49

aaishwarymishra reviewed Aug 4, 2026

View reviewed changes

Comment thread ignite/metrics/fbeta.py Outdated

Comment thread ignite/metrics/fbeta.py Outdated

Comment thread ignite/metrics/fbeta.py Outdated

refactor(metrics): simplify Fbeta class_names validation with active_…

7b8ce31

…metrics pattern

aniketpandey05 force-pushed the feature/class-names-precision-recall branch from aebabb1 to 7b8ce31 Compare August 6, 2026 08:03

Merge branch 'master' into feature/class-names-precision-recall

87cba75

aniketpandey05 requested a review from aaishwarymishra August 6, 2026 08:31

aaishwarymishra reviewed Aug 6, 2026

View reviewed changes

Comment thread ignite/metrics/fbeta.py Outdated

refactor(metrics): remove redundant average check in Fbeta

1e3bdb8

aniketpandey05 requested a review from aaishwarymishra August 9, 2026 13:22

vfdev-5 reviewed Aug 10, 2026

View reviewed changes

aaishwarymishra approved these changes Aug 10, 2026

View reviewed changes

aaishwarymishra requested changes Aug 10, 2026

View reviewed changes

docs(metrics): add versionadded 0.6.0 tags for class_names parameter

2f8887c

aniketpandey05 requested a review from aaishwarymishra August 11, 2026 08:12

vfdev-5 changed the title ~~feat: add class_names support to Precision and Recall metrics~~ feat: add class_names support to Precision, Recall, FBeta metrics Aug 12, 2026

vfdev-5 reviewed Aug 12, 2026

View reviewed changes

		if any(m._average for m in active_metrics):
		raise ValueError("Input precision and recall metrics should have average=False")

Uh oh!

Uh oh!

Conversation

aniketpandey05 commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aaishwarymishra commented Apr 15, 2026

Uh oh!

Uh oh!

aniketpandey05 commented Apr 16, 2026

Uh oh!

aaishwarymishra commented Apr 16, 2026

Uh oh!

steaphenai commented Apr 20, 2026

Uh oh!

vfdev-5 commented Apr 21, 2026

Uh oh!

aniketpandey05 commented Apr 27, 2026

Uh oh!

vfdev-5 commented May 3, 2026

Uh oh!

aniketpandey05 commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vfdev-5 commented May 21, 2026

Uh oh!

aniketpandey05 commented Jun 9, 2026

Uh oh!

aniketpandey05 commented Jun 17, 2026

Uh oh!

aaishwarymishra commented Jun 19, 2026

Uh oh!

aniketpandey05 commented Jun 30, 2026

Uh oh!

aaishwarymishra commented Jun 30, 2026

Uh oh!

aniketpandey05 commented Jul 1, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aniketpandey05 commented Aug 1, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

aniketpandey05 commented Aug 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aaishwarymishra commented Aug 4, 2026

Uh oh!

Uh oh!

aniketpandey05 commented Aug 10, 2026

Uh oh!

aaishwarymishra commented Aug 10, 2026

Uh oh!

vfdev-5 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

aaishwarymishra left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

aniketpandey05 commented Apr 14, 2026 •

edited

Loading

aniketpandey05 commented May 21, 2026 •

edited

Loading