speechbrain.utils.bleu module

Library for computing the BLEU score based on SacreBLEU

SacreBLEU github: https://github.com/mjpost/sacrebleu

Authors

• Titouan Parcollet 2025

• Mirco Ravanelli 2021

Summary

Classes:

BLEUStats

A class for tracking corpus-level BLEU (https://www.aclweb.org/anthology/P02-1040.pdf).

Reference

class speechbrain.utils.bleu.BLEUStats(max_ngram_order=4)

Bases: MetricStats

A class for tracking corpus-level BLEU (https://www.aclweb.org/anthology/P02-1040.pdf). Each hypothesis can be matched against one or multiple references.

Parameters:

max_ngram_order (int, default 4) – The maximum length of the ngrams to use for BLEU scoring. Default is 4.

Example

>>> bleu = BLEUStats()
>>> bleu.append(
...     ids=['utterance1', 'utterance2'],
...     predict=[
...         'The dog bit the man.',
...         'It was not surprising.'],
...     targets=[
...                ['The dog bit the man.', 'It was not unexpected.'],
...                ['The dog had bit the man.', 'No one was surprised.']
...             ]
... )
>>> stats = bleu.summarize()
>>> stats['BLEU']
74.19446627365011

append(ids, predict, targets)

Add stats to the relevant containers. * See MetricStats.append() :param ids: List of ids corresponding to utterances. :type ids: list :param predict: A str which represent the hypotheses. Of dimension [nb_hypotheses] :type predict: list[str] :param targets: List of list of reference. The dimensions are as follow:

[nb_references, nb_hypotheses].

summarize(field=None)

Summarize the BLEU and return relevant statistics. * See MetricStats.summarize()

write_stats(filestream)

Write all relevant info (e.g., error rate alignments) to file. * See MetricStats.write_stats()