speechbrain.tokenizersΒΆ
Package defining the SentencePiece tokenizer
Library for Byte-pair-encoding (BPE) tokenization. Authors * Abdelwahab Heba 2020 * Loren Lugosch 2020. |
Package defining the SentencePiece tokenizer
Library for Byte-pair-encoding (BPE) tokenization. Authors * Abdelwahab Heba 2020 * Loren Lugosch 2020. |