speechbrain.decoders.transducer module
Decoders and output normalization for Transducer sequence.
- Author:
Abdelwahab HEBA 2020 Sung-Lin Yeh 2020
Summary
Classes:
This class implements the beam-search algorithm for the transducer model. |
Functions:
Argument function to customize the sort order (in sorted & max). |
Reference
- class speechbrain.decoders.transducer.TransducerBeamSearcher(decode_network_lst, tjoint, classifier_network, blank_id, beam_size=4, nbest=5, lm_module=None, lm_weight=0.0, state_beam=2.3, expand_beam=2.3)[source]
Bases:
Module
This class implements the beam-search algorithm for the transducer model.
- Parameters:
decode_network_lst (list) – List of prediction network (PN) layers.
tjoint (transducer_joint module) – This module perform the joint between TN and PN.
classifier_network (list) – List of output layers (after performing joint between TN and PN) exp: (TN,PN) => joint => classifier_network_list [DNN bloc, Linear..] => chars prob
blank_id (int) – The blank symbol/index.
beam (int) – The width of beam. Greedy Search is used when beam = 1.
nbest (int) – Number of hypotheses to keep.
lm_module (torch.nn.ModuleList) – Neural networks modules for LM.
lm_weight (float) – The weight of LM when performing beam search (λ). log P(y|x) + λ log P_LM(y). (default: 0.3)
state_beam (float) – The threshold coefficient in log space to decide if hyps in A (process_hyps) is likely to compete with hyps in B (beam_hyps), if not, end the while loop. Reference: https://arxiv.org/pdf/1911.01629.pdf
expand_beam (float) – The threshold coefficient to limit the number of expanded hypotheses that are added in A (process_hyp). Reference: https://arxiv.org/pdf/1911.01629.pdf Reference: https://github.com/kaldi-asr/kaldi/blob/master/src/decoder/simple-decoder.cc (See PruneToks)
Example
- searcher = TransducerBeamSearcher(
decode_network_lst=[hparams[“emb”], hparams[“dec”]], tjoint=hparams[“Tjoint”], classifier_network=[hparams[“transducer_lin”]], blank_id=0, beam_size=hparams[“beam_size”], nbest=hparams[“nbest”], lm_module=hparams[“lm_model”], lm_weight=hparams[“lm_weight”], state_beam=2.3, expand_beam=2.3,
) >>> from speechbrain.nnet.transducer.transducer_joint import Transducer_joint >>> import speechbrain as sb >>> emb = sb.nnet.embedding.Embedding( … num_embeddings=35, … embedding_dim=3, … consider_as_one_hot=True, … blank_id=0 … ) >>> dec = sb.nnet.RNN.GRU( … hidden_size=10, input_shape=(1, 40, 34), bidirectional=False … ) >>> lin = sb.nnet.linear.Linear(input_shape=(1, 40, 10), n_neurons=35) >>> joint_network= sb.nnet.linear.Linear(input_shape=(1, 1, 40, 35), n_neurons=35) >>> tjoint = Transducer_joint(joint_network, joint=”sum”) >>> searcher = TransducerBeamSearcher( … decode_network_lst=[emb, dec], … tjoint=tjoint, … classifier_network=[lin], … blank_id=0, … beam_size=1, … nbest=1, … lm_module=None, … lm_weight=0.0, … ) >>> enc = torch.rand([1, 20, 10]) >>> hyps, scores, _, _ = searcher(enc)
- forward(tn_output)[source]
- Parameters:
tn_output (torch.tensor) – Output from transcription network with shape [batch, time_len, hiddens].
- Return type:
Topk hypotheses
- transducer_greedy_decode(tn_output)[source]
- Transducer greedy decoder is a greedy decoder over batch which apply Transducer rules:
- 1- for each time step in the Transcription Network (TN) output:
- -> Update the ith utterance only if
the previous target != the new one (we save the hiddens and the target)
-> otherwise: —> keep the previous target prediction from the decoder
- Parameters:
tn_output (torch.tensor) – Output from transcription network with shape [batch, time_len, hiddens].
- Returns:
Outputs a logits tensor [B,T,1,Output_Dim]; padding has not been removed.
- Return type:
torch.tensor
- transducer_beam_search_decode(tn_output)[source]
- Transducer beam search decoder is a beam search decoder over batch which apply Transducer rules:
- 1- for each utterance:
- 2- for each time steps in the Transcription Network (TN) output:
-> Do forward on PN and Joint network -> Select topK <= beam -> Do a while loop extending the hyps until we reach blank
-> otherwise: –> extend hyp by the new token
- Parameters:
tn_output (torch.tensor) – Output from transcription network with shape [batch, time_len, hiddens].
- Returns:
Outputs a logits tensor [B,T,1,Output_Dim]; padding has not been removed.
- Return type:
torch.tensor