speechbrain.utils.DER module

Calculates Diarization Error Rate (DER) which is the sum of Missed Speaker (MS), False Alarm (FA), and Speaker Error Rate (SER) using md-eval-22.pl from NIST RT Evaluation.

Authors

  • Neville Ryant 2018

  • Nauman Dawalatabad 2020

Credits

This code is adapted from https://github.com/nryant/dscore

Summary

Functions:

DER

Computes Missed Speaker percentage (MS), False Alarm (FA), Speaker Error Rate (SER), and Diarization Error Rate (DER).

rectify

Corrects corner cases and converts scores into percentage.

Reference

speechbrain.utils.DER.rectify(arr)[source]

Corrects corner cases and converts scores into percentage.

speechbrain.utils.DER.DER(ref_rttm, sys_rttm, ignore_overlap=False, collar=0.25, individual_file_scores=False)[source]

Computes Missed Speaker percentage (MS), False Alarm (FA), Speaker Error Rate (SER), and Diarization Error Rate (DER).

Parameters

  • ref_rttm (str) – The path of reference/groundtruth RTTM file.

  • sys_rttm (str) – The path of the system generated RTTM file.

  • individual_file (bool) – If True, returns scores for each file in order.

  • collar (float) – Forgiveness collar.

  • ignore_overlap (bool) – If True, ignores overlapping speech during evaluation.

Returns

  • MS (float array) – Missed Speech.

  • FA (float array) – False Alarms.

  • SER (float array) – Speaker Error Rates.

  • DER (float array) – Diarization Error Rates.

Example

>>> import pytest
>>> pytest.skip('Skipping because of Perl dependency')
>>> ref_rttm = "../../samples/rttm_samples/ref_rttm/ES2014c.rttm"
>>> sys_rttm = "../../samples/rttm_samples/sys_rttm/ES2014c.rttm"
>>> ignore_overlap = True
>>> collar = 0.25
>>> individual_file_scores = True
>>> Scores = DER(ref_rttm, sys_rttm, ignore_overlap, collar, individual_file_scores)
>>> print (Scores)
(array([0., 0.]), array([0., 0.]), array([7.16923618, 7.16923618]), array([7.16923618, 7.16923618]))