Shortcuts

## Module Interface¶

Calculate SQuAD Metric which corresponds to the scoring script for version 1 of the Stanford Question Answering Dataset (SQuAD).

Parameters

Example

>>> from torchmetrics import SQuAD
>>> preds = [{"prediction_text": "1976", "id": "56e10a3be3433e1400422b22"}]
>>> target = [{"answers": {"answer_start": [97], "text": ["1976"]}, "id": "56e10a3be3433e1400422b22"}]
{'exact_match': tensor(100.), 'f1': tensor(100.)}


References

[1] SQuAD: 100,000+ Questions for Machine Comprehension of Text by Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang SQuAD Metric .

Initializes internal Module state, shared by both nn.Module and ScriptModule.

compute()[source]

Aggregate the F1 Score and Exact match for the batch.

Return type
Returns

Dictionary containing the F1 score, Exact match score for the batch.

update(preds, target)[source]

Compute F1 Score and Exact Match for a collection of predictions and references.

Parameters
• A Dictionary or List of Dictionary-s that map id and prediction_text to the respective values. Example prediction:

{"prediction_text": "TorchMetrics is awesome", "id": "123"}


• A Dictionary or List of Dictionary-s that contain the answers and id in the SQuAD Format. Example target:

{
'id': '1',
}


{
'answers': {'answer_start': [1], 'text': ['This is a test text']},
'context': 'This is a test context.',
'id': '1',
'question': 'Is this a test?',
'title': 'train test'
}


Raises

KeyError – If the required keys are missing in either predictions or targets.

Return type

None

## Functional Interface¶

Calculate SQuAD Metric .

Parameters
• A Dictionary or List of Dictionary-s that map id and prediction_text to the respective values.

Example prediction:

{"prediction_text": "TorchMetrics is awesome", "id": "123"}


• A Dictionary or List of Dictionary-s that contain the answers and id in the SQuAD Format.

Example target:

{
'id': '1',
}


{
'answers': {'answer_start': [1], 'text': ['This is a test text']},
'context': 'This is a test context.',
'id': '1',
'question': 'Is this a test?',
'title': 'train test'
}


Return type
Returns

Dictionary containing the F1 score, Exact match score for the batch.

Example

>>> from torchmetrics.functional.text.squad import squad
>>> preds = [{"prediction_text": "1976", "id": "56e10a3be3433e1400422b22"}]
>>> target = [{"answers": {"answer_start": [97], "text": ["1976"]},"id": "56e10a3be3433e1400422b22"}]
{'exact_match': tensor(100.), 'f1': tensor(100.)}

Raises

KeyError – If the required keys are missing in either predictions or targets.

References

[1] SQuAD: 100,000+ Questions for Machine Comprehension of Text by Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang SQuAD Metric .

© Copyright Copyright (c) 2020-2022, PyTorchLightning et al... Revision 45cc7044.

Built with Sphinx using a theme provided by Read the Docs.
Read the Docs v: v0.8.0
Versions
latest
stable
v0.8.0
v0.7.3
v0.7.2
v0.7.1
v0.7.0
v0.6.2
v0.6.1
v0.6.0
v0.5.1
v0.5.0
v0.4.1
v0.4.0
v0.3.2
v0.3.1
v0.3.0
v0.2.0
v0.1.0