speechbrain.nnet.activations module

Library implementing activation functions.

Authors

Summary

Classes:

`GumbelSoftmax`	Samples from the Gumbel-Softmax distribution and optionally discretizes.
`Softmax`	Computes the softmax of a 2d, 3d, or 4d input tensor.
`Swish`	The class implements the Swish activation function from https://arxiv.org/pdf/2005.03191.pdf

class speechbrain.nnet.activations.Softmax(apply_log=False, dim=-1)[source]

Computes the softmax of a 2d, 3d, or 4d input tensor.

Parameters:

Example

>>> classifier = Softmax()
>>> inputs = torch.rand(10, 50, 40)
>>> output = classifier(inputs)
>>> output.shape
torch.Size([10, 50, 40])

Returns the softmax of the input tensor.

class speechbrain.nnet.activations.GumbelSoftmax(tau, hard=False, apply_log=False)[source]

Samples from the Gumbel-Softmax distribution and optionally discretizes.

Parameters:

tau (float) – non-negative scalar temperature
hard (bool) – if True, the returned samples will be discretized as one-hot vectors, but will be differentiated as if it is the soft sample in autograd
dim (int) – A dimension along which softmax will be computed (default: -1).

Example

>>> x = torch.randn((8, 40, 120))
>>> act = GumbelSoftmax(0.8, True)
>>> x = act(x)

Returns the Gumbel softmax of the input tensor.

class speechbrain.nnet.activations.Swish(beta=1)[source]

The class implements the Swish activation function from https://arxiv.org/pdf/2005.03191.pdf

given input x. Swish(x) = x / (1 + exp(beta * x))

Example

>>> x = torch.randn((8, 40, 120))
>>> act = Swish()
>>> x = act(x)

Returns the Swished input tensor.