Welcome to TorchMetrics

TorchMetrics is a collection of 80+ PyTorch metrics implementations and an easy-to-use API to create custom metrics. It offers:

• A standardized interface to increase reproducibility

• Reduces Boilerplate

• Distributed-training compatible

• Rigorously tested

• Automatic accumulation over batches

• Automatic synchronization between multiple devices

You can use TorchMetrics in any PyTorch model, or within PyTorch Lightning to enjoy the following additional benefits:

• Your data will always be placed on the same device as your metrics

• You can log Metric objects directly in Lightning to reduce even more boilerplate

---

Install TorchMetrics

For pip users

pip install torchmetrics

Or directly from conda

conda install -c conda-forge torchmetrics

---

New to TorchMetrics?

Use this quickstart guide to learn key concepts.

TorchMetrics with PyTorch Lightning

Easily use TorchMetrics in your PyTorch Lightning code.

Metrics

View the full list of metrics and filter by task and data type.

Overview

A detailed overview of the TorchMetrics API and concepts.

Custom Metrics

Learn how to implement a custom metric with TorchMetrics.

API Reference

Detailed descriptions of each API package.

User Guide

• Quick Start
  • Install
  • Using TorchMetrics
  • Implementing your own metric
• Overview
  • Metrics and devices
  • Metrics and 16-bit precision
  • Metric Arithmetics
  • MetricCollection
  • Module vs Functional Metrics
  • Metrics and differentiability
  • Advanced metric settings
• Implementing a Metric
  • Internal implementation details
  • Contributing your metric to Torchmetrics
• TorchMetrics in PyTorch Lightning
  • Logging TorchMetrics
  • Common Pitfalls
• Using Classification Metrics
  • Input types
• Using Retrieval Metrics
  • Input details

Audio

• Perceptual Evaluation of Speech Quality (PESQ)
  • Module Interface
  • Functional Interface
• Permutation Invariant Training (PIT)
  • Module Interface
  • Functional Interface
• Scale-Invariant Signal-to-Distortion Ratio (SI-SDR)
  • Module Interface
  • Functional Interface
• Scale-Invariant Signal-to-Noise Ratio (SI-SNR)
  • Module Interface
  • Functional Interface
• Short-Time Objective Intelligibility (STOI)
  • Module Interface
  • Functional Interface
• Signal to Distortion Ratio (SDR)
  • Module Interface
  • Functional Interface
• Signal-to-Noise Ratio (SNR)
  • Module Interface
  • Functional Interface

Classification

• Accuracy
  • Module Interface
  • Functional Interface
• AUC
  • Module Interface
  • Functional Interface
• AUROC
  • Module Interface
  • Functional Interface
• Average Precision
  • Module Interface
  • Functional Interface
• Binned Average Precision
  • Module Interface
• Binned Precision Recall Curve
  • Module Interface
• Binned Recall At Fixed Precision
  • Module Interface
• Calibration Error
  • Module Interface
  • Functional Interface
• Cohen Kappa
  • Module Interface
  • Functional Interface
• Confusion Matrix
  • Module Interface
  • Functional Interface
• Coverage Error
  • Module Interface
  • Functional Interface
• Dice Score
  • Functional Interface
• F1 Score
  • Module Interface
  • Functional Interface
• FBeta Score
  • Module Interface
  • Functional Interface
• Hamming Distance
  • Module Interface
  • Functional Interface
• Hinge Loss
  • Module Interface
  • Functional Interface
• Jaccard Index
  • Module Interface
  • Functional Interface
• KL Divergence
  • Module Interface
  • Functional Interface
• Label Ranking Average Precision
  • Module Interface
  • Functional Interface
• Label Ranking Loss
  • Module Interface
  • Functional Interface
• Matthews Corr. Coef.
  • Module Interface
  • Functional Interface
• Precision
  • Module Interface
  • Functional Interface
• Precision Recall
  • Functional Interface
• Precision Recall Curve
  • Module Interface
  • Functional Interface
• Recall
  • Module Interface
  • Functional Interface
• ROC
  • Module Interface
  • Functional Interface
• Specificity
  • Module Interface
  • Functional Interface
• Stat Scores
  • Module Interface
  • Functional Interface

Image

• Error Relative Global Dim. Synthesis (ERGAS)
  • Module Interface
  • Functional Interface
• Frechet Inception Distance (FID)
  • Module Interface
• Image Gradients
  • Functional Interface
• Inception Score
  • Module Interface
• Kernel Inception Distance
  • Module Interface
• Learned Perceptual Image Patch Similarity (LPIPS)
  • Module Interface
• Multi-Scale SSIM
  • Module Interface
  • Functional Interface
• Peak Signal-to-Noise Ratio (PSNR)
  • Module Interface
  • Functional Interface
• Spectral Angle Mapper
  • Module Interface
  • Functional Interface
• Spectral Distortion Index
  • Module Interface
  • Functional Interface
• Structural Similarity Index Measure (SSIM)
  • Module Interface
  • Functional Interface
• Universal Image Quality Index
  • Module Interface
  • Functional Interface

Detection

• Mean-Average-Precision (mAP)
  • Module Interface

Pairwise

• Cosine Similarity
  • Functional Interface
• Euclidean Distance
  • Functional Interface
• Linear Similarity
  • Functional Interface
• Manhattan Distance
  • Functional Interface

Regression

• Cosine Similarity
  • Module Interface
  • Functional Interface
• Explained Variance
  • Module Interface
  • Functional Interface
• Mean Absolute Error (MAE)
  • Module Interface
  • Functional Interface
• Mean Absolute Percentage Error (MAPE)
  • Module Interface
  • Functional Interface
• Mean Squared Error (MSE)
  • Module Interface
  • Functional Interface
• Mean Squared Log Error (MSLE)
  • Module Interface
  • Functional Interface
• Pearson Corr. Coef.
  • Module Interface
  • Functional Interface
• R2 Score
  • Module Interface
  • Functional Interface
• Spearman Corr. Coef.
  • Module Interface
  • Functional Interface
• Symmetric Mean Absolute Percentage Error (SMAPE)
  • Module Interface
  • Functional Interface
• Tweedie Deviance Score
  • Module Interface
  • Functional Interface
• Weighted MAPE
  • Module Interface
  • Functional Interface

Retrieval

• Retrieval Fall-Out
  • Module Interface
  • Functional Interface
• Retrieval Hit Rate
  • Module Interface
  • Functional Interface
• Retrieval Mean Average Precision (MAP)
  • Module Interface
  • Functional Interface
• Retrieval Mean Reciprocal Rank (MRR)
  • Module Interface
  • Functional Interface
• Retrieval Normalized DCG
  • Module Interface
  • Functional Interface
• Retrieval Precision
  • Module Interface
  • Functional Interface
• Retrieval R-Precision
  • Module Interface
  • Functional Interface
• Retrieval Recall
  • Module Interface
  • Functional Interface

Text

• BERT Score
  • Module Interface
  • Functional Interface
• BLEU Score
  • Module Interface
  • Functional Interface
• Char Error Rate
  • Module Interface
  • Functional Interface
• ChrF Score
  • Module Interface
  • Functional Interface
• Extended Edit Distance
  • Module Interface
  • Functional Interface
• Match Error Rate
  • Module Interface
  • Functional Interface
• ROUGE Score
  • Module Interface
  • Functional Interface
• Sacre BLEU Score
  • Module Interface
  • Functional Interface
• SQuAD
  • Module Interface
  • Functional Interface
• Translation Edit Rate (TER)
  • Module Interface
  • Functional Interface
• Word Error Rate
  • Module Interface
  • Functional Interface
• Word Info. Lost
  • Module Interface
  • Functional Interface
• Word Info. Preserved
  • Module Interface
  • Functional Interface

Aggregation

• Concatenation
  • Module Interface
• Maximum
  • Module Interface
• Mean
  • Module Interface
• Minimum
  • Module Interface
• Sum
  • Module Interface

Wrappers

• Bootstrapper
  • Module Interface
• Classwise Wrapper
  • Module Interface
• Metric Tracker
  • Module Interface
• Min / Max
  • Module Interface
• Multi-output Wrapper
  • Module Interface

API Reference

• torchmetrics.Metric
• torchmetrics.utilities.data
  • select_topk
  • to_categorical
  • to_onehot

Community

• TorchMetrics Governance
• Contributor Covenant Code of Conduct
• Contributing
• Changelog

Indices and tables

• Index

• Module Index

• Search Page