Exact Match¶

Module Interface¶

ExactMatch¶

class torchmetrics.ExactMatch(task: Literal['binary', 'multiclass', 'multilabel'], threshold: float = 0.5, num_classes: Optional[int] = None, num_labels: Optional[int] = None, multidim_average: Literal['global', 'samplewise'] = 'global', ignore_index: Optional[int] = None, validate_args: bool = True, **kwargs: Any)[source]

Computes Exact match (also known as subset accuracy). Exact Match is a stricter version of accuracy where all labels have to match exactly for the sample to be correctly classified.

This module is a simple wrapper to get the task specific versions of this metric, which is done by setting the task argument to either 'multiclass' or multilabel. See the documentation of MulticlassExactMatch and MultilabelExactMatch for the specific details of each argument influence and examples.

Legacy Example: >>> target = torch.tensor([[[0, 1], [2, 1], [0, 2]], [[1, 1], [2, 0], [1, 2]]]) >>> preds = torch.tensor([[[0, 1], [2, 1], [0, 2]], [[2, 2], [2, 1], [1, 0]]]) >>> metric = ExactMatch(task=”multiclass”, num_classes=3, multidim_average=’global’) >>> metric(preds, target) tensor(0.5000)
>>> target = torch.tensor([[[0, 1], [2, 1], [0, 2]], [[1, 1], [2, 0], [1, 2]]])
>>> preds = torch.tensor([[[0, 1], [2, 1], [0, 2]], [[2, 2], [2, 1], [1, 0]]])
>>> metric = ExactMatch(task="multiclass", num_classes=3, multidim_average='samplewise')
>>> metric(preds, target)
tensor([1., 0.])

MulticlassExactMatch¶

class torchmetrics.classification.MulticlassExactMatch(num_classes, multidim_average='global', ignore_index=None, validate_args=True, **kwargs)[source]

Computes Exact match (also known as subset accuracy) for multiclass tasks. Exact Match is a stricter version of accuracy where all labels have to match exactly for the sample to be correctly classified.

Accepts the following input tensors:

preds: (N, ...) (int tensor) or (N, C, ..) (float tensor). If preds is a floating point we apply torch.argmax along the C dimension to automatically convert probabilities/logits into an int tensor.
target (int tensor): (N, ...)

The influence of the additional dimension ... (if present) will be determined by the multidim_average argument.

Parameters

num_classes¶ (int) – Integer specifing the number of labels
multidim_average¶ (Literal[‘global’, ‘samplewise’]) –
Defines how additionally dimensions ... should be handled. Should be one of the following:
- global: Additional dimensions are flatted along the batch dimension
- samplewise: Statistic will be calculated independently for each sample on the N axis. The statistics in this case are calculated over the additional dimensions.
ignore_index¶ (Optional[int]) – Specifies a target value that is ignored and does not contribute to the metric calculation
validate_args¶ (bool) – bool indicating if input arguments and tensors should be validated for correctness. Set to False for faster computations.

Returns

If multidim_average is set to global the output will be a scalar tensor
If multidim_average is set to samplewise the output will be a tensor of shape (N,)

Return type

The returned shape depends on the multidim_average argument

Example (multidim tensors):

>>> from torchmetrics.classification import MulticlassExactMatch
>>> target = torch.tensor([[[0, 1], [2, 1], [0, 2]], [[1, 1], [2, 0], [1, 2]]])
>>> preds = torch.tensor([[[0, 1], [2, 1], [0, 2]], [[2, 2], [2, 1], [1, 0]]])
>>> metric = MulticlassExactMatch(num_classes=3, multidim_average='global')
>>> metric(preds, target)
tensor(0.5000)

Example (multidim tensors):

>>> from torchmetrics.classification import MulticlassExactMatch
>>> target = torch.tensor([[[0, 1], [2, 1], [0, 2]], [[1, 1], [2, 0], [1, 2]]])
>>> preds = torch.tensor([[[0, 1], [2, 1], [0, 2]], [[2, 2], [2, 1], [1, 0]]])
>>> metric = MulticlassExactMatch(num_classes=3, multidim_average='samplewise')
>>> metric(preds, target)
tensor([1., 0.])

Initializes internal Module state, shared by both nn.Module and ScriptModule.

compute()[source]

Override this method to compute the final metric value from state variables synchronized across the distributed backend.

Return type: Tensor

update(preds, target)[source]

Override this method to update the state variables of your metric class.

Return type: None

MultilabelExactMatch¶

class torchmetrics.classification.MultilabelExactMatch(num_labels, threshold=0.5, multidim_average='global', ignore_index=None, validate_args=True, **kwargs)[source]

Computes Exact match (also known as subset accuracy) for multilabel tasks. Exact Match is a stricter version of accuracy where all labels have to match exactly for the sample to be correctly classified.

Accepts the following input tensors:

preds (int or float tensor): (N, C, ...). If preds is a floating point tensor with values outside [0,1] range we consider the input to be logits and will auto apply sigmoid per element. Addtionally, we convert to int tensor with thresholding using the value in threshold.
target (int tensor): (N, C, ...)

The influence of the additional dimension ... (if present) will be determined by the multidim_average argument.

Parameters

num_labels¶ (int) – Integer specifing the number of labels
threshold¶ (float) – Threshold for transforming probability to binary (0,1) predictions
multidim_average¶ (Literal[‘global’, ‘samplewise’]) –
Defines how additionally dimensions ... should be handled. Should be one of the following:
- global: Additional dimensions are flatted along the batch dimension
- samplewise: Statistic will be calculated independently for each sample on the N axis. The statistics in this case are calculated over the additional dimensions.
ignore_index¶ (Optional[int]) – Specifies a target value that is ignored and does not contribute to the metric calculation
validate_args¶ (bool) – bool indicating if input arguments and tensors should be validated for correctness. Set to False for faster computations.

Returns

If multidim_average is set to global the output will be a scalar tensor
If multidim_average is set to samplewise the output will be a tensor of shape (N,)

Return type

The returned shape depends on the multidim_average argument

Example (preds is int tensor):

>>> from torchmetrics.classification import MultilabelExactMatch
>>> target = torch.tensor([[0, 1, 0], [1, 0, 1]])
>>> preds = torch.tensor([[0, 0, 1], [1, 0, 1]])
>>> metric = MultilabelExactMatch(num_labels=3)
>>> metric(preds, target)
tensor(0.5000)

Example (preds is float tensor):

>>> from torchmetrics.classification import MultilabelExactMatch
>>> target = torch.tensor([[0, 1, 0], [1, 0, 1]])
>>> preds = torch.tensor([[0.11, 0.22, 0.84], [0.73, 0.33, 0.92]])
>>> metric = MultilabelExactMatch(num_labels=3)
>>> metric(preds, target)
tensor(0.5000)

Example (multidim tensors):

>>> from torchmetrics.classification import MultilabelExactMatch
>>> target = torch.tensor([[[0, 1], [1, 0], [0, 1]], [[1, 1], [0, 0], [1, 0]]])
>>> preds = torch.tensor(
...     [
...         [[0.59, 0.91], [0.91, 0.99], [0.63, 0.04]],
...         [[0.38, 0.04], [0.86, 0.780], [0.45, 0.37]],
...     ]
... )
>>> metric = MultilabelExactMatch(num_labels=3, multidim_average='samplewise')
>>> metric(preds, target)
tensor([0., 0.])

Initializes internal Module state, shared by both nn.Module and ScriptModule.

compute()[source]

Override this method to compute the final metric value from state variables synchronized across the distributed backend.

Return type: Tensor

update(preds, target)[source]

Update state with predictions and targets.

Parameters

preds¶ (Tensor) – Tensor with predictions
target¶ (Tensor) – Tensor with true labels

Return type

None

Functional Interface¶

exact_match¶

torchmetrics.functional.classification.multilabel_exact_match(preds, target, num_labels, threshold=0.5, multidim_average='global', ignore_index=None, validate_args=True)[source]

Computes Exact match (also known as subset accuracy) for multilabel tasks. Exact Match is a stricter version of accuracy where all labels have to match exactly for the sample to be correctly classified.

Accepts the following input tensors:

preds (int or float tensor): (N, C, ...). If preds is a floating point tensor with values outside [0,1] range we consider the input to be logits and will auto apply sigmoid per element. Addtionally, we convert to int tensor with thresholding using the value in threshold.
target (int tensor): (N, C, ...)

The influence of the additional dimension ... (if present) will be determined by the multidim_average argument.

Parameters

preds¶ (Tensor) – Tensor with predictions
target¶ (Tensor) – Tensor with true labels
num_labels¶ (int) – Integer specifing the number of labels
threshold¶ (float) – Threshold for transforming probability to binary (0,1) predictions
multidim_average¶ (Literal[‘global’, ‘samplewise’]) –
Defines how additionally dimensions ... should be handled. Should be one of the following:
- global: Additional dimensions are flatted along the batch dimension
- samplewise: Statistic will be calculated independently for each sample on the N axis. The statistics in this case are calculated over the additional dimensions.
ignore_index¶ (Optional[int]) – Specifies a target value that is ignored and does not contribute to the metric calculation
validate_args¶ (bool) – bool indicating if input arguments and tensors should be validated for correctness. Set to False for faster computations.

Returns

If multidim_average is set to global the output will be a scalar tensor
If multidim_average is set to samplewise the output will be a tensor of shape (N,)

Return type

The returned shape depends on the multidim_average argument

Example (preds is int tensor):

>>> from torchmetrics.functional.classification import multilabel_exact_match
>>> target = torch.tensor([[0, 1, 0], [1, 0, 1]])
>>> preds = torch.tensor([[0, 0, 1], [1, 0, 1]])
>>> multilabel_exact_match(preds, target, num_labels=3)
tensor(0.5000)

Example (preds is float tensor):

>>> from torchmetrics.functional.classification import multilabel_exact_match
>>> target = torch.tensor([[0, 1, 0], [1, 0, 1]])
>>> preds = torch.tensor([[0.11, 0.22, 0.84], [0.73, 0.33, 0.92]])
>>> multilabel_exact_match(preds, target, num_labels=3)
tensor(0.5000)

Example (multidim tensors):

>>> from torchmetrics.functional.classification import multilabel_exact_match
>>> target = torch.tensor([[[0, 1], [1, 0], [0, 1]], [[1, 1], [0, 0], [1, 0]]])
>>> preds = torch.tensor(
...     [
...         [[0.59, 0.91], [0.91, 0.99], [0.63, 0.04]],
...         [[0.38, 0.04], [0.86, 0.780], [0.45, 0.37]],
...     ]
... )
>>> multilabel_exact_match(preds, target, num_labels=3, multidim_average='samplewise')
tensor([0., 0.])

multiclass_exact_match¶

torchmetrics.functional.classification.multiclass_exact_match(preds, target, num_classes, multidim_average='global', ignore_index=None, validate_args=True)[source]

Computes Exact match (also known as subset accuracy) for multiclass tasks. Exact Match is a stricter version of accuracy where all labels have to match exactly for the sample to be correctly classified.

Accepts the following input tensors:

preds: (N, ...) (int tensor) or (N, C, ..) (float tensor). If preds is a floating point we apply torch.argmax along the C dimension to automatically convert probabilities/logits into an int tensor.
target (int tensor): (N, ...)

The influence of the additional dimension ... (if present) will be determined by the multidim_average argument.

Parameters

preds¶ (Tensor) – Tensor with predictions
target¶ (Tensor) – Tensor with true labels
num_classes¶ (int) – Integer specifing the number of labels
multidim_average¶ (Literal[‘global’, ‘samplewise’]) –
Defines how additionally dimensions ... should be handled. Should be one of the following:
- global: Additional dimensions are flatted along the batch dimension
- samplewise: Statistic will be calculated independently for each sample on the N axis. The statistics in this case are calculated over the additional dimensions.
ignore_index¶ (Optional[int]) – Specifies a target value that is ignored and does not contribute to the metric calculation
validate_args¶ (bool) – bool indicating if input arguments and tensors should be validated for correctness. Set to False for faster computations.

Returns

If multidim_average is set to global the output will be a scalar tensor
If multidim_average is set to samplewise the output will be a tensor of shape (N,)

Return type

The returned shape depends on the multidim_average argument

Example (multidim tensors):

>>> from torchmetrics.functional.classification import multiclass_exact_match
>>> target = torch.tensor([[[0, 1], [2, 1], [0, 2]], [[1, 1], [2, 0], [1, 2]]])
>>> preds = torch.tensor([[[0, 1], [2, 1], [0, 2]], [[2, 2], [2, 1], [1, 0]]])
>>> multiclass_exact_match(preds, target, num_classes=3, multidim_average='global')
tensor(0.5000)

Example (multidim tensors):

>>> from torchmetrics.functional.classification import multiclass_exact_match
>>> target = torch.tensor([[[0, 1], [2, 1], [0, 2]], [[1, 1], [2, 0], [1, 2]]])
>>> preds = torch.tensor([[[0, 1], [2, 1], [0, 2]], [[2, 2], [2, 1], [1, 0]]])
>>> multiclass_exact_match(preds, target, num_classes=3, multidim_average='samplewise')
tensor([1., 0.])

multilabel_exact_match¶

torchmetrics.functional.classification.multilabel_exact_match(preds, target, num_labels, threshold=0.5, multidim_average='global', ignore_index=None, validate_args=True)[source]

Computes Exact match (also known as subset accuracy) for multilabel tasks. Exact Match is a stricter version of accuracy where all labels have to match exactly for the sample to be correctly classified.

Accepts the following input tensors:

preds (int or float tensor): (N, C, ...). If preds is a floating point tensor with values outside [0,1] range we consider the input to be logits and will auto apply sigmoid per element. Addtionally, we convert to int tensor with thresholding using the value in threshold.
target (int tensor): (N, C, ...)

The influence of the additional dimension ... (if present) will be determined by the multidim_average argument.

Parameters

preds¶ (Tensor) – Tensor with predictions
target¶ (Tensor) – Tensor with true labels
num_labels¶ (int) – Integer specifing the number of labels
threshold¶ (float) – Threshold for transforming probability to binary (0,1) predictions
multidim_average¶ (Literal[‘global’, ‘samplewise’]) –
Defines how additionally dimensions ... should be handled. Should be one of the following:
- global: Additional dimensions are flatted along the batch dimension
- samplewise: Statistic will be calculated independently for each sample on the N axis. The statistics in this case are calculated over the additional dimensions.
ignore_index¶ (Optional[int]) – Specifies a target value that is ignored and does not contribute to the metric calculation
validate_args¶ (bool) – bool indicating if input arguments and tensors should be validated for correctness. Set to False for faster computations.

Returns

If multidim_average is set to global the output will be a scalar tensor
If multidim_average is set to samplewise the output will be a tensor of shape (N,)

Return type

The returned shape depends on the multidim_average argument

Example (preds is int tensor):

>>> from torchmetrics.functional.classification import multilabel_exact_match
>>> target = torch.tensor([[0, 1, 0], [1, 0, 1]])
>>> preds = torch.tensor([[0, 0, 1], [1, 0, 1]])
>>> multilabel_exact_match(preds, target, num_labels=3)
tensor(0.5000)

Example (preds is float tensor):

>>> from torchmetrics.functional.classification import multilabel_exact_match
>>> target = torch.tensor([[0, 1, 0], [1, 0, 1]])
>>> preds = torch.tensor([[0.11, 0.22, 0.84], [0.73, 0.33, 0.92]])
>>> multilabel_exact_match(preds, target, num_labels=3)
tensor(0.5000)

Example (multidim tensors):

>>> from torchmetrics.functional.classification import multilabel_exact_match
>>> target = torch.tensor([[[0, 1], [1, 0], [0, 1]], [[1, 1], [0, 0], [1, 0]]])
>>> preds = torch.tensor(
...     [
...         [[0.59, 0.91], [0.91, 0.99], [0.63, 0.04]],
...         [[0.38, 0.04], [0.86, 0.780], [0.45, 0.37]],
...     ]
... )
>>> multilabel_exact_match(preds, target, num_labels=3, multidim_average='samplewise')
tensor([0., 0.])