BLEU Score

Module Interface

class torchmetrics.BLEUScore(n_gram=4, smooth=False, compute_on_step=None, **kwargs)

Calculate BLEU score of machine translated text with one or more references.

Parameters

• n_gram (int) – Gram value ranged from 1 to 4

• smooth (bool) – Whether or not to apply smoothing, see [2]

• compute_on_step (Optional[bool]) –

  Forward only calls update() and returns None if this is set to False.

  Deprecated since version v0.8: Argument has no use anymore and will be removed v0.9.

• kwargs (Dict[str, Any]) – Additional keyword arguments, see Advanced metric settings for more info.

Example

>>> from torchmetrics import BLEUScore
>>> preds = ['the cat is on the mat']
>>> target = [['there is a cat on the mat', 'a cat is on the mat']]
>>> metric = BLEUScore()
>>> metric(preds, target)
tensor(0.7598)

References

[1] BLEU: a Method for Automatic Evaluation of Machine Translation by Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu BLEU

[2] Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics by Chin-Yew Lin and Franz Josef Och Machine Translation Evolution

Initializes internal Module state, shared by both nn.Module and ScriptModule.

compute()

Calculate BLEU score.

Return type

Tensor

Returns

Tensor with BLEU Score

update(preds, target)

Compute Precision Scores.

Parameters

• preds (Sequence[str]) – An iterable of machine translated corpus

• target (Sequence[Sequence[str]]) – An iterable of iterables of reference corpus

Return type

None

Functional Interface

torchmetrics.functional.bleu_score(preds, target, n_gram=4, smooth=False)

Calculate BLEU score of machine translated text with one or more references.

Parameters

• preds (Union[str, Sequence[str]]) – An iterable of machine translated corpus

• target (Sequence[Union[str, Sequence[str]]]) – An iterable of iterables of reference corpus

• n_gram (int) – Gram value ranged from 1 to 4 (Default 4)

• smooth (bool) – Whether to apply smoothing – see [2]

Return type

Tensor

Returns

Tensor with BLEU Score

Example

>>> from torchmetrics.functional import bleu_score
>>> preds = ['the cat is on the mat']
>>> target = [['there is a cat on the mat', 'a cat is on the mat']]
>>> bleu_score(preds, target)
tensor(0.7598)

References

[1] BLEU: a Method for Automatic Evaluation of Machine Translation by Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu BLEU

[2] Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics by Chin-Yew Lin and Franz Josef Och Machine Translation Evolution