speechbrain.nnet.complex_networks.c_normalization module¶
Library implementing complex-valued normalization.
- Authors
Titouan Parcollet 2020
Summary¶
Classes:
This class is implements the complex-valued batch-normalization as introduced by “Deep Complex Networks”, Trabelsi C. |
|
This class is used to instanciate the complex layer-normalization as introduced by “Deep Complex Networks”, Trabelsi C. |
Functions:
This function is used to apply the complex normalization as introduced by “Deep Complex Networks”, Trabelsi C. |
|
This function is used to standardize a centred tensor of complex numbers (mean of the set must be 0). |
Reference¶
-
class
speechbrain.nnet.complex_networks.c_normalization.
CBatchNorm
(input_shape=None, input_size=None, dim=- 1, eps=0.0001, momentum=0.1, scale=True, center=True, track_running_stats=True)[source]¶ Bases:
torch.nn.modules.module.Module
This class is implements the complex-valued batch-normalization as introduced by “Deep Complex Networks”, Trabelsi C. et al.
- Parameters
input_shape (tuple) – Expected shape of the input.
input_size (int) – Expected size of the input.
dim (int, optional) – It defines the axis that should be normalized. It usually correspond to the channel dimension (default -1).
eps (float, optional) – Term used to stabilize operation (default 1e-4).
momentum (float, optional) – It defines the momentum as for the real-valued batch-normalization (default 0.1).
scale (bool, optional,) – It defines if scaling should be used or not. It is equivalent to the real-valued batchnormalization scaling (default True).
center (bool, optional) – It defines if centering should be used or not. It is equivalent to the real-valued batchnormalization centering (default True).
track_running_stats (bool, optional) – Equivalent to the real-valued batchnormalization parameter. When True, stats are tracked. When False, solely statistics computed over the batch are used (default True).
Example
>>> inp_tensor = torch.rand([10, 16, 30]) >>> CBN = CBatchNorm(input_shape=inp_tensor.shape) >>> out_tensor = CBN(inp_tensor) >>> out_tensor.shape torch.Size([10, 16, 30])
-
forward
(input)[source]¶ Returns the normalized input tensor.
- Parameters
input (torch.Tensor (batch, time, [channels])) – Input to normalize. It can be 2d, 3d, 4d.
-
class
speechbrain.nnet.complex_networks.c_normalization.
CLayerNorm
(input_shape=None, input_size=None, dim=- 1, eps=0.0001, scale=True, center=True)[source]¶ Bases:
torch.nn.modules.module.Module
This class is used to instanciate the complex layer-normalization as introduced by “Deep Complex Networks”, Trabelsi C. et al.
- Parameters
input_shape (tuple) – Expected shape of the input.
input_size (int) – Expected size of the input dimension.
dim (int, optional) – It defines the axis that should be normalized. It usually correspond to the channel dimension (default -1).
eps (float, optional) – Term used to stabilize operation (default 1e-4).
scale (bool, optional,) – It defines if scaling should be used or not. It is equivalent to the real-valued batchnormalization scaling (default True).
center (bool, optional) – It defines if centering should be used or not. It is equivalent to the real-valued batchnormalization centering (default True).
Example
>>> inp_tensor = torch.rand([10, 16, 30]) >>> CBN = CLayerNorm(input_shape=inp_tensor.shape) >>> out_tensor = CBN(inp_tensor) >>> out_tensor.shape torch.Size([10, 16, 30])
-
speechbrain.nnet.complex_networks.c_normalization.
c_norm
(input_centred, Vrr, Vii, Vri, beta, gamma_rr, gamma_ri, gamma_ii, scale=True, center=True, layernorm=False, dim=- 1)[source]¶ This function is used to apply the complex normalization as introduced by “Deep Complex Networks”, Trabelsi C. et al.
- Parameters
input_centred (torch.Tensor) – It is the tensor to be normalized. The features dimension is divided by 2 with the first half corresponding to the real-parts and the second half to the imaginary parts.
Vrr (torch.Tensor) – It is a tensor that contains the covariance between real-parts.
Vii (torch.Tensor) – It is a tensor that contains the covariance between imaginary-parts.
Vri (torch.Tensor) – It is a tensor that contains the covariance between real-parts and imaginary-parts.
beta (torch.Tensor) – It is a tensor corresponding to the beta parameter on the real-valued batch-normalization, but in the complex-valued space.
gamma_rr (torch.Tensor) – It is a tensor that contains the gamma between real-parts.
gamma_ii (torch.Tensor) – It is a tensor that contains the gamma between imaginary-parts.
gamma_ri (torch.Tensor) – It is a tensor that contains the gamma between real-parts and imaginary-parts.
scale (bool, optional) – It defines if scaling should be used or not. It is equivalent to the real-valued batchnormalization scaling (default True).
center (bool, optional,) – It defines if centering should be used or not. It is equivalent to the real-valued batchnormalization centering (default True).
layernorm (bool, optional) – It defines is c_standardization is called from a layernorm or a batchnorm layer (default False).
dim (int, optional) – It defines the axis that should be considered as the complex-valued axis (divided by 2 to get r and i) (default -1).
-
speechbrain.nnet.complex_networks.c_normalization.
c_standardization
(input_centred, Vrr, Vii, Vri, layernorm=False, dim=- 1)[source]¶ This function is used to standardize a centred tensor of complex numbers (mean of the set must be 0).
- Parameters
input_centred (torch.Tensor) – It is the tensor to be normalized. The features dimension is divided by 2 with the first half corresponding to the real-parts and the second half to the imaginary parts.
Vrr (torch.Tensor) – It is a tensor that contains the covariance between real-parts.
Vii (torch.Tensor) – It is a tensor that contains the covariance between imaginary-parts.
Vri (torch.Tensor) – It is a tensor that contains the covariance between real-parts and imaginary-parts.
layernorm (bool, optional) – It defines is c_standardization is called from a layernorm or a batchnorm layer (default False).
dim (int, optional) – It defines the axis that should be considered as the complex-valued axis (divided by 2 to get r and i) (default -1).