Shortcuts

Perplexity

Module Interface

class torchmetrics.text.perplexity.Perplexity(ignore_index=None, **kwargs)[source]

Perplexity measures how well a language model predicts a text sample. It’s calculated as the average number of bits per word a model needs to represent the sample.

As input to forward and update the metric accepts the following input:

  • preds (Tensor): Probabilities assigned to each token in a sequence with shape

    [batch_size, seq_len, vocab_size]

  • target (Tensor): Ground truth values with a shape [batch_size, seq_len]

As output of forward and compute the metric returns the following output:

  • perp (Tensor): A tensor with the perplexity score

Parameters
  • ignore_index (Optional[int]) – Integer specifying a target class to ignore. If given, this class index does not contribute to the returned score.

  • kwargs (Dict[str, Any]) – Additional keyword arguments, see Advanced metric settings for more info.

Examples

>>> import torch
>>> preds = torch.rand(2, 8, 5, generator=torch.manual_seed(22))
>>> target = torch.randint(5, (2, 8), generator=torch.manual_seed(22))
>>> target[0, 6:] = -100
>>> perp = Perplexity(ignore_index=-100)
>>> perp(preds, target)
tensor(5.2545)

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Functional Interface

torchmetrics.functional.text.perplexity.perplexity(preds, target, ignore_index=None)[source]

Perplexity measures how well a language model predicts a text sample. It’s calculated as the average number of bits per word a model needs to represent the sample.

Parameters
  • preds (Tensor) – Log probabilities assigned to each token in a sequence with shape [batch_size, seq_len, vocab_size].

  • target (Tensor) – Ground truth values with a shape [batch_size, seq_len].

  • ignore_index (Optional[int]) – Integer specifying a target class to ignore. If given, this class index does not contribute to the returned score.

Return type

Tensor

Returns

Perplexity value

Examples

>>> import torch
>>> preds = torch.rand(2, 8, 5, generator=torch.manual_seed(22))
>>> target = torch.randint(5, (2, 8), generator=torch.manual_seed(22))
>>> target[0, 6:] = -100
>>> perplexity(preds, target, ignore_index=-100)
tensor(5.2545)
Read the Docs v: stable
Versions
latest
stable
v0.11.1
v0.11.0
v0.10.3
v0.10.2
v0.10.1
v0.10.0
v0.9.3
v0.9.2
v0.9.1
v0.9.0
v0.8.2
v0.8.1
v0.8.0
v0.7.3
v0.7.2
v0.7.1
v0.7.0
v0.6.2
v0.6.1
v0.6.0
v0.5.1
v0.5.0
v0.4.1
v0.4.0
v0.3.2
v0.3.1
v0.3.0
v0.2.0
v0.1.0
Downloads
pdf
html
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.