Table of Contents
Shortcuts

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Note: we move fast, but still we preserve 0.1 version (one feature release) back compatibility.

[0.11.2] - 2023-02-21

[0.11.2] - Fixed

  • Fixed compatibility between XLA in _bincount function (#1471)

  • Fixed type hints in methods belonging to MetricTracker wrapper (#1472)

  • Fixed multilabel in ExactMatch (#1474)

[0.11.1] - 2023-01-30

[0.11.1] - Fixed

  • Fixed type checking on the maximize parameter at the initialization of MetricTracker (#1428)

  • Fixed mixed precision autocast for SSIM metric (#1454)

  • Fixed checking for nltk.punkt in RougeScore if a machine is not online (#1456)

  • Fixed wrongly reset method in MultioutputWrapper (#1460)

  • Fixed dtype checking in PrecisionRecallCurve for target tensor (#1457)

[0.11.0] - 2022-11-30

[0.11.0] - Added

  • Added MulticlassExactMatch to classification metrics (#1343)

  • Added TotalVariation to image package (#978)

  • Added CLIPScore to new multimodal package (#1314)

  • Added regression metrics:

    • KendallRankCorrCoef (#1271)

    • LogCoshError (#1316)

  • Added new nominal metrics:

  • Added option to pass distributed_available_fn to metrics to allow checks for custom communication backend for making dist_sync_fn actually useful (#1301)

  • Added normalize argument to Inception, FID, KID metrics (#1246)

[0.11.0] - Changed

  • Changed minimum Pytorch version to be 1.8 (#1263)

  • Changed interface for all functional and modular classification metrics after refactor (#1252)

[0.11.0] - Removed

  • Removed deprecated BinnedAveragePrecision, BinnedPrecisionRecallCurve, RecallAtFixedPrecision (#1251)

  • Removed deprecated LabelRankingAveragePrecision, LabelRankingLoss and CoverageError (#1251)

  • Removed deprecated KLDivergence and AUC (#1251)

[0.11.0] - Fixed

  • Fixed precision bug in pairwise_euclidean_distance (#1352)

[0.10.3] - 2022-11-16

[0.10.3] - Fixed

  • Fixed bug in Metrictracker.best_metric when return_step=False (#1306)

  • Fixed bug to prevent users from going into an infinite loop if trying to iterate of a single metric (#1320)

[0.10.2] - 2022-10-31

[0.10.2] - Changed

  • Changed in-place operation to out-of-place operation in pairwise_cosine_similarity (#1288)

[0.10.2] - Fixed

  • Fixed high memory usage for certain classification metrics when average='micro' (#1286)

  • Fixed precision problems when structural_similarity_index_measure was used with autocast (#1291)

  • Fixed slow performance for confusion matrix based metrics (#1302)

  • Fixed restrictive dtype checking in spearman_corrcoef when used with autocast (#1303)

[0.10.1] - 2022-10-21

[0.10.1] - Fixed

  • Fixed broken clone method for classification metrics (#1250)

  • Fixed unintentional downloading of nltk.punkt when lsum not in rouge_keys (#1258)

  • Fixed type casting in MAP metric between bool and float32 (#1150)

[0.10.0] - 2022-10-04

[0.10.0] - Added

  • Added a new NLP metric InfoLM (#915)

  • Added Perplexity metric (#922)

  • Added ConcordanceCorrCoef metric to regression package (#1201)

  • Added argument normalize to LPIPS metric (#1216)

  • Added support for multiprocessing of batches in PESQ metric (#1227)

  • Added support for multioutput in PearsonCorrCoef and SpearmanCorrCoef (#1200)

[0.10.0] - Changed

[0.10.0] - Deprecated

  • Deprecated BinnedAveragePrecision, BinnedPrecisionRecallCurve, BinnedRecallAtFixedPrecision (#1163)

    • BinnedAveragePrecision -> use AveragePrecision with thresholds arg

    • BinnedPrecisionRecallCurve -> use AveragePrecisionRecallCurve with thresholds arg

    • BinnedRecallAtFixedPrecision -> use RecallAtFixedPrecision with thresholds arg

  • Renamed and refactored LabelRankingAveragePrecision, LabelRankingLoss and CoverageError (#1167)

    • LabelRankingAveragePrecision -> MultilabelRankingAveragePrecision

    • LabelRankingLoss -> MultilabelRankingLoss

    • CoverageError -> MultilabelCoverageError

  • Deprecated KLDivergence and AUC from classification package (#1189)

    • KLDivergence moved to regression package

    • Instead of AUC use torchmetrics.utils.compute.auc

[0.10.0] - Fixed

  • Fixed a bug in ssim when return_full_image=True where the score was still reduced (#1204)

  • Fixed MPS support for:

  • Fixed bug in ClasswiseWrapper such that compute gave wrong result (#1225)

  • Fixed synchronization of empty list states (#1219)

[0.9.3] - 2022-08-22

[0.9.3] - Added

  • Added global option sync_on_compute to disable automatic synchronization when compute is called (#1107)

[0.9.3] - Fixed

  • Fixed missing reset in ClasswiseWrapper (#1129)

  • Fixed JaccardIndex multi-label compute (#1125)

  • Fix SSIM propagate device if gaussian_kernel is False, add test (#1149)

[0.9.2] - 2022-06-29

[0.9.2] - Fixed

  • Fixed mAP calculation for areas with 0 predictions (#1080)

  • Fixed bug where avg precision state and auroc state was not merge when using MetricCollections (#1086)

  • Skip box conversion if no boxes are present in MeanAveragePrecision (#1097)

  • Fixed inconsistency in docs and code when setting average="none" in AvaragePrecision metric (#1116)

[0.9.1] - 2022-06-08

[0.9.1] - Added

  • Added specific RuntimeError when metric object is on the wrong device (#1056)

  • Added an option to specify own n-gram weights for BLEUScore and SacreBLEUScore instead of using uniform weights only. (#1075)

[0.9.1] - Fixed

  • Fixed aggregation metrics when input only contains zero (#1070)

  • Fixed TypeError when providing superclass arguments as kwargs (#1069)

  • Fixed bug related to state reference in metric collection when using compute groups (#1076)

[0.9.0] - 2022-05-30

[0.9.0] - Added

  • Added RetrievalPrecisionRecallCurve and RetrievalRecallAtFixedPrecision to retrieval package (#951)

  • Added class property full_state_update that determines forward should call update once or twice ( #984, #1033)

  • Added support for nested metric collections (#1003)

  • Added Dice to classification package (#1021)

  • Added support to segmentation type segm as IOU for mean average precision (#822)

[0.9.0] - Changed

  • Renamed reduction argument to average in Jaccard score and added additional options (#874)

[0.9.0] - Removed

[0.9.0] - Fixed

  • Fixed non-empty state dict for a few metrics (#1012)

  • Fixed bug when comparing states while finding compute groups (#1022)

  • Fixed torch.double support in stat score metrics (#1023)

  • Fixed FID calculation for non-equal size real and fake input (#1028)

  • Fixed case where KLDivergence could output Nan (#1030)

  • Fixed deterministic for PyTorch<1.8 (#1035)

  • Fixed default value for mdmc_average in Accuracy (#1036)

  • Fixed missing copy of property when using compute groups in MetricCollection (#1052)

[0.8.2] - 2022-05-06

[0.8.2] - Fixed

  • Fixed multi device aggregation in PearsonCorrCoef (#998)

  • Fixed MAP metric when using custom list of thresholds (#995)

  • Fixed compatibility between compute groups in MetricCollection and prefix/postfix arg (#1007)

  • Fixed compatibility with future Pytorch 1.12 in safe_matmul (#1011, #1014)

[0.8.1] - 2022-04-27

[0.8.1] - Changed

  • Reimplemented the signal_distortion_ratio metric, which removed the absolute requirement of fast-bss-eval (#964)

[0.8.1] - Fixed

  • Fixed “Sort currently does not support bool dtype on CUDA” error in MAP for empty preds (#983)

  • Fixed BinnedPrecisionRecallCurve when thresholds argument is not provided (#968)

  • Fixed CalibrationError to work on logit input (#985)

[0.8.0] - 2022-04-14

[0.8.0] - Added

  • Added WeightedMeanAbsolutePercentageError to regression package (#948)

  • Added new classification metrics:

    • CoverageError (#787)

    • LabelRankingAveragePrecision and LabelRankingLoss (#787)

  • Added new image metric:

    • SpectralAngleMapper (#885)

    • ErrorRelativeGlobalDimensionlessSynthesis (#894)

    • UniversalImageQualityIndex (#824)

    • SpectralDistortionIndex (#873)

  • Added support for MetricCollection in MetricTracker (#718)

  • Added support for 3D image and uniform kernel in StructuralSimilarityIndexMeasure (#818)

  • Added smart update of MetricCollection (#709)

  • Added ClasswiseWrapper for better logging of classification metrics with multiple output values (#832)

  • Added **kwargs argument for passing additional arguments to base class (#833)

  • Added negative ignore_index for the Accuracy metric (#362)

  • Added adaptive_k for the RetrievalPrecision metric (#910)

  • Added reset_real_features argument image quality assessment metrics (#722)

  • Added new keyword argument compute_on_cpu to all metrics (#867)

[0.8.0] - Changed

  • Made num_classes in jaccard_index a required argument (#853, #914)

  • Added normalizer, tokenizer to ROUGE metric (#838)

  • Improved shape checking of permutation_invariant_training (#864)

  • Allowed reduction None (#891)

  • MetricTracker.best_metric will now give a warning when computing on metric that do not have a best (#913)

[0.8.0] - Deprecated

  • Deprecated argument compute_on_step (#792)

  • Deprecated passing in dist_sync_on_step, process_group, dist_sync_fn direct argument (#833)

[0.8.0] - Removed

  • Removed support for versions of Pytorch-Lightning lower than v1.5 (#788)

  • Removed deprecated functions, and warnings in Text (#773)

    • WER and functional.wer

  • Removed deprecated functions and warnings in Image (#796)

    • SSIM and functional.ssim

    • PSNR and functional.psnr

  • Removed deprecated functions, and warnings in classification and regression (#806)

    • FBeta and functional.fbeta

    • F1 and functional.f1

    • Hinge and functional.hinge

    • IoU and functional.iou

    • MatthewsCorrcoef

    • PearsonCorrcoef

    • SpearmanCorrcoef

  • Removed deprecated functions, and warnings in detection and pairwise (#804)

    • MAP and functional.pairwise.manhatten

  • Removed deprecated functions, and warnings in Audio (#805)

    • PESQ and functional.audio.pesq

    • PIT and functional.audio.pit

    • SDR and functional.audio.sdr and functional.audio.si_sdr

    • SNR and functional.audio.snr and functional.audio.si_snr

    • STOI and functional.audio.stoi

  • Removed unused get_num_classes from torchmetrics.utilities.data (#914)

[0.8.0] - Fixed

  • Fixed device mismatch for MAP metric in specific cases (#950)

  • Improved testing speed (#820)

  • Fixed compatibility of ClasswiseWrapper with the prefix argument of MetricCollection (#843)

  • Fixed BestScore on GPU (#912)

  • Fixed Lsum computation for ROUGEScore (#944)

[0.7.3] - 2022-03-23

[0.7.3] - Fixed

  • Fixed unsafe log operation in TweedieDeviace for power=1 (#847)

  • Fixed bug in MAP metric related to either no ground truth or no predictions (#884)

  • Fixed ConfusionMatrix, AUROC and AveragePrecision on GPU when running in deterministic mode (#900)

  • Fixed NaN or Inf results returned by signal_distortion_ratio (#899)

  • Fixed memory leak when using update method with tensor where requires_grad=True (#902)

[0.7.2] - 2022-02-10

[0.7.2] - Fixed

  • Minor patches in JOSS paper.

[0.7.1] - 2022-02-03

[0.7.1] - Changed

  • Used torch.bucketize in calibration error when torch>1.8 for faster computations (#769)

  • Improve mAP performance (#742)

[0.7.1] - Fixed

  • Fixed check for available modules (#772)

  • Fixed Matthews correlation coefficient when the denominator is 0 (#781)

[0.7.0] - 2022-01-17

[0.7.0] - Added

  • Added NLP metrics:

    • MatchErrorRate (#619)

    • WordInfoLost and WordInfoPreserved (#630)

    • SQuAD (#623)

    • CHRFScore (#641)

    • TranslationEditRate (#646)

    • ExtendedEditDistance (#668)

  • Added MultiScaleSSIM into image metrics (#679)

  • Added Signal to Distortion Ratio (SDR) to audio package (#565)

  • Added MinMaxMetric to wrappers (#556)

  • Added ignore_index to retrieval metrics (#676)

  • Added support for multi references in ROUGEScore (#680)

  • Added a default VSCode devcontainer configuration (#621)

[0.7.0] - Changed

  • Scalar metrics will now consistently have additional dimensions squeezed (#622)

  • Metrics having third party dependencies removed from global import (#463)

  • Untokenized for BLEUScore input stay consistent with all the other text metrics (#640)

  • Arguments reordered for TER, BLEUScore, SacreBLEUScore, CHRFScore now expect input order as predictions first and target second (#696)

  • Changed dtype of metric state from torch.float to torch.long in ConfusionMatrix to accommodate larger values (#715)

  • Unify preds, target input argument’s naming across all text metrics (#723, #727)

    • bert, bleu, chrf, sacre_bleu, wip, wil, cer, ter, wer, mer, rouge, squad

[0.7.0] - Deprecated

  • Renamed IoU -> Jaccard Index (#662)

  • Renamed text WER metric (#714)

    • functional.wer -> functional.word_error_rate

    • WER -> WordErrorRate

  • Renamed correlation coefficient classes: (#710)

    • MatthewsCorrcoef -> MatthewsCorrCoef

    • PearsonCorrcoef -> PearsonCorrCoef

    • SpearmanCorrcoef -> SpearmanCorrCoef

  • Renamed audio STOI metric: (#753, #758)

    • audio.STOI to audio.ShortTimeObjectiveIntelligibility

    • functional.audio.stoi to functional.audio.short_time_objective_intelligibility

  • Renamed audio PESQ metrics: (#751)

    • functional.audio.pesq -> functional.audio.perceptual_evaluation_speech_quality

    • audio.PESQ -> audio.PerceptualEvaluationSpeechQuality

  • Renamed audio SDR metrics: (#711)

    • functional.sdr -> functional.signal_distortion_ratio

    • functional.si_sdr -> functional.scale_invariant_signal_distortion_ratio

    • SDR -> SignalDistortionRatio

    • SI_SDR -> ScaleInvariantSignalDistortionRatio

  • Renamed audio SNR metrics: (#712)

    • functional.snr -> functional.signal_distortion_ratio

    • functional.si_snr -> functional.scale_invariant_signal_noise_ratio

    • SNR -> SignalNoiseRatio

    • SI_SNR -> ScaleInvariantSignalNoiseRatio

  • Renamed F-score metrics: (#731, #740)

    • functional.f1 -> functional.f1_score

    • F1 -> F1Score

    • functional.fbeta -> functional.fbeta_score

    • FBeta -> FBetaScore

  • Renamed Hinge metric: (#734)

    • functional.hinge -> functional.hinge_loss

    • Hinge -> HingeLoss

  • Renamed image PSNR metrics (#732)

    • functional.psnr -> functional.peak_signal_noise_ratio

    • PSNR -> PeakSignalNoiseRatio

  • Renamed image PIT metric: (#737)

    • functional.pit -> functional.permutation_invariant_training

    • PIT -> PermutationInvariantTraining

  • Renamed image SSIM metric: (#747)

    • functional.ssim -> functional.scale_invariant_signal_noise_ratio

    • SSIM -> StructuralSimilarityIndexMeasure

  • Renamed detection MAP to MeanAveragePrecision metric (#754)

  • Renamed Fidelity & LPIPS image metric: (#752)

    • image.FID -> image.FrechetInceptionDistance

    • image.KID -> image.KernelInceptionDistance

    • image.LPIPS -> image.LearnedPerceptualImagePatchSimilarity

[0.7.0] - Removed

  • Removed embedding_similarity metric (#638)

  • Removed argument concatenate_texts from wer metric (#638)

  • Removed arguments newline_sep and decimal_places from rouge metric (#638)

[0.7.0] - Fixed

  • Fixed MetricCollection kwargs filtering when no kwargs are present in update signature (#707)

[0.6.2] - 2021-12-15

[0.6.2] - Fixed

  • Fixed torch.sort currently does not support bool dtype on CUDA (#665)

  • Fixed mAP properly checks if ground truths are empty (#684)

  • Fixed initialization of tensors to be on correct device for MAP metric (#673)

[0.6.1] - 2021-12-06

[0.6.1] - Changed

  • Migrate MAP metrics from pycocotools to PyTorch (#632)

  • Use torch.topk instead of torch.argsort in retrieval precision for speedup (#627)

[0.6.1] - Fixed

  • Fix empty predictions in MAP metric (#594, #610, #624)

  • Fix edge case of AUROC with average=weighted on GPU (#606)

  • Fixed forward in compositional metrics (#645)

[0.6.0] - 2021-10-28

[0.6.0] - Added

  • Added audio metrics:

    • Perceptual Evaluation of Speech Quality (PESQ) (#353)

    • Short-Time Objective Intelligibility (STOI) (#353)

  • Added Information retrieval metrics:

    • RetrievalRPrecision (#577)

    • RetrievalHitRate (#576)

  • Added NLP metrics:

    • SacreBLEUScore (#546)

    • CharErrorRate (#575)

  • Added other metrics:

    • Tweedie Deviance Score (#499)

    • Learned Perceptual Image Patch Similarity (LPIPS) (#431)

  • Added MAP (mean average precision) metric to new detection package (#467)

  • Added support for float targets in nDCG metric (#437)

  • Added average argument to AveragePrecision metric for reducing multi-label and multi-class problems (#477)

  • Added MultioutputWrapper (#510)

  • Added metric sweeping:

    • higher_is_better as constant attribute (#544)

    • higher_is_better to rest of codebase (#584)

  • Added simple aggregation metrics: SumMetric, MeanMetric, CatMetric, MinMetric, MaxMetric (#506)

  • Added pairwise submodule with metrics (#553)

    • pairwise_cosine_similarity

    • pairwise_euclidean_distance

    • pairwise_linear_similarity

    • pairwise_manhatten_distance

[0.6.0] - Changed

  • AveragePrecision will now as default output the macro average for multilabel and multiclass problems (#477)

  • half, double, float will no longer change the dtype of the metric states. Use metric.set_dtype instead (#493)

  • Renamed AverageMeter to MeanMetric (#506)

  • Changed is_differentiable from property to a constant attribute (#551)

  • ROC and AUROC will no longer throw an error when either the positive or negative class is missing. Instead return 0 score and give a warning

[0.6.0] - Deprecated

  • Deprecated functional.self_supervised.embedding_similarity in favour of new pairwise submodule

[0.6.0] - Removed

  • Removed dtype property (#493)

[0.6.0] - Fixed

  • Fixed bug in F1 with average='macro' and ignore_index!=None (#495)

  • Fixed bug in pit by using the returned first result to initialize device and type (#533)

  • Fixed SSIM metric using too much memory (#539)

  • Fixed bug where device property was not properly update when metric was a child of a module (#542)

[0.5.1] - 2021-08-30

[0.5.1] - Added

  • Added device and dtype properties (#462)

  • Added TextTester class for robustly testing text metrics (#450)

[0.5.1] - Changed

  • Added support for float targets in nDCG metric (#437)

[0.5.1] - Removed

  • Removed rouge-score as dependency for text package (#443)

  • Removed jiwer as dependency for text package (#446)

  • Removed bert-score as dependency for text package (#473)

[0.5.1] - Fixed

  • Fixed ranking of samples in SpearmanCorrCoef metric (#448)

  • Fixed bug where compositional metrics where unable to sync because of type mismatch (#454)

  • Fixed metric hashing (#478)

  • Fixed BootStrapper metrics not working on GPU (#462)

  • Fixed the semantic ordering of kernel height and width in SSIM metric (#474)

[0.5.0] - 2021-08-09

[0.5.0] - Added

  • Added Text-related (NLP) metrics:

  • Added MetricTracker wrapper metric for keeping track of the same metric over multiple epochs (#238)

  • Added other metrics:

    • Symmetric Mean Absolute Percentage error (SMAPE) (#375)

    • Calibration error (#394)

    • Permutation Invariant Training (PIT) (#384)

  • Added support in nDCG metric for target with values larger than 1 (#349)

  • Added support for negative targets in nDCG metric (#378)

  • Added None as reduction option in CosineSimilarity metric (#400)

  • Allowed passing labels in (n_samples, n_classes) to AveragePrecision (#386)

[0.5.0] - Changed

  • Moved psnr and ssim from functional.regression.* to functional.image.* (#382)

  • Moved image_gradient from functional.image_gradients to functional.image.gradients (#381)

  • Moved R2Score from regression.r2score to regression.r2 (#371)

  • Pearson metric now only store 6 statistics instead of all predictions and targets (#380)

  • Use torch.argmax instead of torch.topk when k=1 for better performance (#419)

  • Moved check for number of samples in R2 score to support single sample updating (#426)

[0.5.0] - Deprecated

  • Rename r2score >> r2_score and kldivergence >> kl_divergence in functional (#371)

  • Moved bleu_score from functional.nlp to functional.text.bleu (#360)

[0.5.0] - Removed

  • Removed restriction that threshold has to be in (0,1) range to support logit input ( #351 #401)

  • Removed restriction that preds could not be bigger than num_classes to support logit input (#357)

  • Removed module regression.psnr and regression.ssim (#382):

  • Removed (#379):

    • function functional.mean_relative_error

    • num_thresholds argument in BinnedPrecisionRecallCurve

[0.5.0] - Fixed

  • Fixed bug where classification metrics with average='macro' would lead to wrong result if a class was missing (#303)

  • Fixed weighted, multi-class AUROC computation to allow for 0 observations of some class, as contribution to final AUROC is 0 (#376)

  • Fixed that _forward_cache and _computed attributes are also moved to the correct device if metric is moved (#413)

  • Fixed calculation in IoU metric when using ignore_index argument (#328)

[0.4.1] - 2021-07-05

[0.4.1] - Changed

[0.4.1] - Fixed

  • Fixed DDP by is_sync logic to Metric (#339)

[0.4.0] - 2021-06-29

[0.4.0] - Added

  • Added Image-related metrics:

    • Fréchet inception distance (FID) (#213)

    • Kernel Inception Distance (KID) (#301)

    • Inception Score (#299)

    • KL divergence (#247)

  • Added Audio metrics: SNR, SI_SDR, SI_SNR (#292)

  • Added other metrics:

    • Cosine Similarity (#305)

    • Specificity (#210)

    • Mean Absolute Percentage error (MAPE) (#248)

  • Added add_metrics method to MetricCollection for adding additional metrics after initialization (#221)

  • Added pre-gather reduction in the case of dist_reduce_fx="cat" to reduce communication cost (#217)

  • Added better error message for AUROC when num_classes is not provided for multiclass input (#244)

  • Added support for unnormalized scores (e.g. logits) in Accuracy, Precision, Recall, FBeta, F1, StatScore, Hamming, ConfusionMatrix metrics (#200)

  • Added squared argument to MeanSquaredError for computing RMSE (#249)

  • Added is_differentiable property to ConfusionMatrix, F1, FBeta, Hamming, Hinge, IOU, MatthewsCorrcoef, Precision, Recall, PrecisionRecallCurve, ROC, StatScores (#253)

  • Added sync and sync_context methods for manually controlling when metric states are synced (#302)

[0.4.0] - Changed

  • Forward cache is reset when reset method is called (#260)

  • Improved per-class metric handling for imbalanced datasets for precision, recall, precision_recall, fbeta, f1, accuracy, and specificity (#204)

  • Decorated torch.jit.unused to MetricCollection forward (#307)

  • Renamed thresholds argument to binned metrics for manually controlling the thresholds (#322)

  • Extend typing (#324, #326, #327)

[0.4.0] - Deprecated

  • Deprecated functional.mean_relative_error, use functional.mean_absolute_percentage_error (#248)

  • Deprecated num_thresholds argument in BinnedPrecisionRecallCurve (#322)

[0.4.0] - Removed

  • Removed argument is_multiclass (#319)

[0.4.0] - Fixed

  • AUC can also support more dimensional inputs when all but one dimension are of size 1 (#242)

  • Fixed dtype of modular metrics after reset has been called (#243)

  • Fixed calculation in matthews_corrcoef to correctly match formula (#321)

[0.3.2] - 2021-05-10

[0.3.2] - Added

  • Added is_differentiable property:

    • To AUC, AUROC, CohenKappa and AveragePrecision (#178)

    • To PearsonCorrCoef, SpearmanCorrcoef, R2Score and ExplainedVariance (#225)

[0.3.2] - Changed

  • MetricCollection should return metrics with prefix on items(), keys() (#209)

  • Calling compute before update will now give warning (#164)

[0.3.2] - Removed

  • Removed numpy as direct dependency (#212)

[0.3.2] - Fixed

  • Fixed auc calculation and add tests (#197)

  • Fixed loading persisted metric states using load_state_dict() (#202)

  • Fixed PSNR not working with DDP (#214)

  • Fixed metric calculation with unequal batch sizes (#220)

  • Fixed metric concatenation for list states for zero-dim input (#229)

  • Fixed numerical instability in AUROC metric for large input (#230)

[0.3.1] - 2021-04-21

  • Cleaning remaining inconsistency and fix PL develop integration ( #191, #192, #193, #194 )

[0.3.0] - 2021-04-20

[0.3.0] - Added

  • Added BootStrapper to easily calculate confidence intervals for metrics (#101)

  • Added Binned metrics (#128)

  • Added metrics for Information Retrieval ((PL^5032)):

    • RetrievalMAP (PL^5032)

    • RetrievalMRR (#119)

    • RetrievalPrecision (#139)

    • RetrievalRecall (#146)

    • RetrievalNormalizedDCG (#160)

    • RetrievalFallOut (#161)

  • Added other metrics:

    • CohenKappa (#69)

    • MatthewsCorrcoef (#98)

    • PearsonCorrcoef (#157)

    • SpearmanCorrcoef (#158)

    • Hinge (#120)

  • Added average='micro' as an option in AUROC for multilabel problems (#110)

  • Added multilabel support to ROC metric (#114)

  • Added testing for half precision (#77, #135 )

  • Added AverageMeter for ad-hoc averages of values (#138)

  • Added prefix argument to MetricCollection (#70)

  • Added __getitem__ as metric arithmetic operation (#142)

  • Added property is_differentiable to metrics and test for differentiability (#154)

  • Added support for average, ignore_index and mdmc_average in Accuracy metric (#166)

  • Added postfix arg to MetricCollection (#188)

[0.3.0] - Changed

  • Changed ExplainedVariance from storing all preds/targets to tracking 5 statistics (#68)

  • Changed behaviour of confusionmatrix for multilabel data to better match multilabel_confusion_matrix from sklearn (#134)

  • Updated FBeta arguments (#111)

  • Changed reset method to use detach.clone() instead of deepcopy when resetting to default (#163)

  • Metrics passed as dict to MetricCollection will now always be in deterministic order (#173)

  • Allowed MetricCollection pass metrics as arguments (#176)

[0.3.0] - Deprecated

  • Rename argument is_multiclass -> multiclass (#162)

[0.3.0] - Removed

  • Prune remaining deprecated (#92)

[0.3.0] - Fixed

  • Fixed when _stable_1d_sort to work when n>=N (PL^6177)

  • Fixed _computed attribute not being correctly reset (#147)

  • Fixed to Blau score (#165)

  • Fixed backwards compatibility for logging with older version of pytorch-lightning (#182)

[0.2.0] - 2021-03-12

[0.2.0] - Changed

  • Decoupled PL dependency (#13)

  • Refactored functional - mimic the module-like structure: classification, regression, etc. (#16)

  • Refactored utilities - split to topics/submodules (#14)

  • Refactored MetricCollection (#19)

[0.2.0] - Removed

  • Removed deprecated metrics from PL base (#12, #15)

[0.1.0] - 2021-02-22

  • Added Accuracy metric now generalizes to Top-k accuracy for (multi-dimensional) multi-class inputs using the top_k parameter (PL^4838)

  • Added Accuracy metric now enables the computation of subset accuracy for multi-label or multi-dimensional multi-class inputs with the subset_accuracy parameter (PL^4838)

  • Added HammingDistance metric to compute the hamming distance (loss) (PL^4838)

  • Added StatScores metric to compute the number of true positives, false positives, true negatives and false negatives (PL^4839)

  • Added R2Score metric (PL^5241)

  • Added MetricCollection (PL^4318)

  • Added .clone() method to metrics (PL^4318)

  • Added IoU class interface (PL^4704)

  • The Recall and Precision metrics (and their functional counterparts recall and precision) can now be generalized to Recall@K and Precision@K with the use of top_k parameter (PL^4842)

  • Added compositional metrics (PL^5464)

  • Added AUC/AUROC class interface (PL^5479)

  • Added QuantizationAwareTraining callback (PL^5706)

  • Added ConfusionMatrix class interface (PL^4348)

  • Added multiclass AUROC metric (PL^4236)

  • Added PrecisionRecallCurve, ROC, AveragePrecision class metric (PL^4549)

  • Classification metrics overhaul (PL^4837)

  • Added F1 class metric (PL^4656)

  • Added metrics aggregation in Horovod and fixed early stopping (PL^3775)

  • Added persistent(mode) method to metrics, to enable and disable metric states being added to state_dict (PL^4482)

  • Added unification of regression metrics (PL^4166)

  • Added persistent flag to Metric.add_state (PL^4195)

  • Added classification metrics (PL^4043)

  • Added new Metrics API. (PL^3868, PL^3921)

  • Added EMB similarity (PL^3349)

  • Added SSIM metrics (PL^2671)

  • Added BLEU metrics (PL^2535)

Read the Docs v: v0.11.2
Versions
latest
stable
v0.11.2
v0.11.1
v0.11.0
v0.10.3
v0.10.2
v0.10.1
v0.10.0
v0.9.3
v0.9.2
v0.9.1
v0.9.0
v0.8.2
v0.8.1
v0.8.0
v0.7.3
v0.7.2
v0.7.1
v0.7.0
v0.6.2
v0.6.1
v0.6.0
v0.5.1
v0.5.0
v0.4.1
v0.4.0
v0.3.2
v0.3.1
v0.3.0
v0.2.0
v0.1.0
Downloads
On Read the Docs
Project Home
Builds
Free document hosting provided by Read the Docs.

To analyze traffic and optimize your experience, we serve cookies on this site. By clicking or navigating, you agree to allow our usage of cookies. Read PyTorch Lightning's Privacy Policy.