speechbrain.integrations.nlp.bleu module

Library for computing the BLEU score

Authors

• Mirco Ravanelli 2021

• Titouan Parcollet 2025

Summary

Classes:

BLEUStats

A class for tracking corpus-level BLEU (https://www.aclweb.org/anthology/P02-1040.pdf).

Reference

class speechbrain.integrations.nlp.bleu.BLEUStats(max_ngram_order=4)

Bases: MetricStats

A class for tracking corpus-level BLEU (https://www.aclweb.org/anthology/P02-1040.pdf). Each hypothesis can be matched against one or multiple references.

Parameters:

max_ngram_order (int, default 4) – The maximum length of the ngrams to use for BLEU scoring. Default is 4.

Example

>>> bleu = BLEUStats()
>>> bleu.append(
...     ids=["utterance1", "utterance2"],
...     predict=["The dog bit the man.", "It was not surprising."],
...     targets=[
...         ["The dog bit the man.", "It was not unexpected."],
...         ["The dog had bit the man.", "No one was surprised."],
...     ],
... )
>>> stats = bleu.summarize()
>>> stats["BLEU"]
74.19446627365011

append(ids, predict, targets)

Add stats to the relevant containers. * See MetricStats.append() :param ids: List of ids corresponding to utterances. :type ids: list :param predict: A str which represent the hypotheses. Of dimension [nb_hypotheses] :type predict: list[str] :param targets: List of list of reference. The dimensions are as follow:

[nb_references, nb_hypotheses].

summarize(field=None)

Summarize the BLEU and return relevant statistics. * See MetricStats.summarize()

write_stats(filestream)

Write all relevant info (e.g., error rate alignments) to file. * See MetricStats.write_stats()