Precision Recall Curve¶
Module Interface¶
- class torchmetrics.RetrievalPrecisionRecallCurve(max_k=None, adaptive_k=False, empty_target_action='neg', ignore_index=None, **kwargs)[source]
Computes precision-recall pairs for different k (from 1 to max_k).
In a ranked retrieval context, appropriate sets of retrieved documents are naturally given by the top k retrieved documents.
Recall is the fraction of relevant documents retrieved among all the relevant documents. Precision is the fraction of relevant documents among all the retrieved documents.
For each such set, precision and recall values can be plotted to give a recall-precision curve.
Forward accepts:
preds
(float tensor):(N, ...)
target
(long or bool tensor):(N, ...)
indexes
(long tensor):(N, ...)
indexes
,preds
andtarget
must have the same dimension.indexes
indicate to which query a prediction belongs. Predictions will be first grouped byindexes
and then RetrievalRecallAtFixedPrecision will be computed as the mean of the RetrievalRecallAtFixedPrecision over each query.- Parameters
max_k¶ (
Optional
[int
]) – Calculate recall and precision for all possible top k from 1 to max_k (default: None, which considers all possible top k)adaptive_k¶ (
bool
) – adjust k to min(k, number of documents) for each querySpecify what to do with queries that do not have at least a positive
target
. Choose from:'neg'
: those queries count as0.0
(default)'pos'
: those queries count as1.0
'skip'
: skip those queries; if all queries are skipped,0.0
is returned'error'
: raise aValueError
ignore_index¶ (
Optional
[int
]) – Ignore predictions where the target is equal to this number.kwargs¶ (
Any
) – Additional keyword arguments, see Advanced metric settings for more info.
- Raises
ValueError – If
empty_target_action
is not one oferror
,skip
,neg
orpos
.ValueError – If
ignore_index
is not None or an integer.ValueError – If
max_k
parameter is not None or an integer larger than 0.
Example
>>> from torchmetrics import RetrievalPrecisionRecallCurve >>> indexes = tensor([0, 0, 0, 0, 1, 1, 1]) >>> preds = tensor([0.4, 0.01, 0.5, 0.6, 0.2, 0.3, 0.5]) >>> target = tensor([True, False, False, True, True, False, True]) >>> r = RetrievalPrecisionRecallCurve(max_k=4) >>> precisions, recalls, top_k = r(preds, target, indexes=indexes) >>> precisions tensor([1.0000, 0.5000, 0.6667, 0.5000]) >>> recalls tensor([0.5000, 0.5000, 1.0000, 1.0000]) >>> top_k tensor([1, 2, 3, 4])
Initializes internal Module state, shared by both nn.Module and ScriptModule.
- compute()[source]
Override this method to compute the final metric value from state variables synchronized across the distributed backend.
Functional Interface¶
- torchmetrics.functional.retrieval_precision_recall_curve(preds, target, max_k=None, adaptive_k=False)[source]
Computes precision-recall pairs for different k (from 1 to max_k).
In a ranked retrieval context, appropriate sets of retrieved documents are naturally given by the top k retrieved documents.
Recall is the fraction of relevant documents retrieved among all the relevant documents. Precision is the fraction of relevant documents among all the retrieved documents.
For each such set, precision and recall values can be plotted to give a recall-precision curve.
preds
andtarget
should be of the same shape and live on the same device. If notarget
isTrue
,0
is returned.target
must be either bool or integers andpreds
must befloat
, otherwise an error is raised.- Parameters
preds¶ (
Tensor
) – estimated probabilities of each document to be relevant.target¶ (
Tensor
) – ground truth about each document being relevant or not.max_k¶ (
Optional
[int
]) – Calculate recall and precision for all possible top k from 1 to max_k (default: None, which considers all possible top k)adaptive_k¶ (
bool
) – adjust max_k to min(max_k, number of documents) for each query
- Return type
- Returns
tensor with the precision values for each k (at
k
) from 1 to max_k tensor with the recall values for each k (atk
) from 1 to max_k tensor with all possibles k- Raises
ValueError – If
max_k
is not None or an integer larger than 0.ValueError – If
adaptive_k
is not boolean.
Example
>>> from torchmetrics.functional import retrieval_precision_recall_curve >>> preds = tensor([0.2, 0.3, 0.5]) >>> target = tensor([True, False, True]) >>> precisions, recalls, top_k = retrieval_precision_recall_curve(preds, target, max_k=2) >>> precisions tensor([1.0000, 0.5000]) >>> recalls tensor([0.5000, 0.5000]) >>> top_k tensor([1, 2])