speechbrain.nnet.complex_networks.c_normalization moduleο
Library implementing complex-valued normalization.
- Authors
Titouan Parcollet 2020
Summaryο
Classes:
This class is implements the complex-valued batch-normalization as introduced by "Deep Complex Networks", Trabelsi C. |
|
This class is used to instantiate the complex layer-normalization as introduced by "Deep Complex Networks", Trabelsi C. |
Functions:
This function is used to apply the complex normalization as introduced by "Deep Complex Networks", Trabelsi C. |
|
This function is used to standardize a centered tensor of complex numbers (mean of the set must be 0). |
Referenceο
- class speechbrain.nnet.complex_networks.c_normalization.CBatchNorm(input_shape=None, input_size=None, dim=-1, eps=0.0001, momentum=0.1, scale=True, center=True, track_running_stats=True)[source]ο
Bases:
Module
This class is implements the complex-valued batch-normalization as introduced by βDeep Complex Networksβ, Trabelsi C. et al.
- Parameters:
input_shape (tuple) β Expected shape of the input.
input_size (int) β Expected size of the input.
dim (int, optional) β It defines the axis that should be normalized. It usually correspond to the channel dimension (default -1).
eps (float, optional) β Term used to stabilize operation (default 1e-4).
momentum (float, optional) β It defines the momentum as for the real-valued batch-normalization (default 0.1).
scale (bool, optional,) β It defines if scaling should be used or not. It is equivalent to the real-valued batchnormalization scaling (default True).
center (bool, optional) β It defines if centering should be used or not. It is equivalent to the real-valued batchnormalization centering (default True).
track_running_stats (bool, optional) β Equivalent to the real-valued batchnormalization parameter. When True, stats are tracked. When False, solely statistics computed over the batch are used (default True).
Example
>>> inp_tensor = torch.rand([10, 16, 30]) >>> CBN = CBatchNorm(input_shape=inp_tensor.shape) >>> out_tensor = CBN(inp_tensor) >>> out_tensor.shape torch.Size([10, 16, 30])
- class speechbrain.nnet.complex_networks.c_normalization.CLayerNorm(input_shape=None, input_size=None, dim=-1, eps=0.0001, scale=True, center=True)[source]ο
Bases:
Module
This class is used to instantiate the complex layer-normalization as introduced by βDeep Complex Networksβ, Trabelsi C. et al.
- Parameters:
input_shape (tuple) β Expected shape of the input.
input_size (int) β Expected size of the input dimension.
dim (int, optional) β It defines the axis that should be normalized. It usually correspond to the channel dimension (default -1).
eps (float, optional) β Term used to stabilize operation (default 1e-4).
scale (bool, optional,) β It defines if scaling should be used or not. It is equivalent to the real-valued batchnormalization scaling (default True).
center (bool, optional) β It defines if centering should be used or not. It is equivalent to the real-valued batchnormalization centering (default True).
Example
>>> inp_tensor = torch.rand([10, 16, 30]) >>> CBN = CLayerNorm(input_shape=inp_tensor.shape) >>> out_tensor = CBN(inp_tensor) >>> out_tensor.shape torch.Size([10, 16, 30])
- speechbrain.nnet.complex_networks.c_normalization.c_norm(input_centred, Vrr, Vii, Vri, beta, gamma_rr, gamma_ri, gamma_ii, scale=True, center=True, layernorm=False, dim=-1)[source]ο
This function is used to apply the complex normalization as introduced by βDeep Complex Networksβ, Trabelsi C. et al.
- Parameters:
input_centred (torch.Tensor) β It is the tensor to be normalized. The features dimension is divided by 2 with the first half corresponding to the real-parts and the second half to the imaginary parts.
Vrr (torch.Tensor) β It is a tensor that contains the covariance between real-parts.
Vii (torch.Tensor) β It is a tensor that contains the covariance between imaginary-parts.
Vri (torch.Tensor) β It is a tensor that contains the covariance between real-parts and imaginary-parts.
beta (torch.Tensor) β It is a tensor corresponding to the beta parameter on the real-valued batch-normalization, but in the complex-valued space.
gamma_rr (torch.Tensor) β It is a tensor that contains the gamma between real-parts.
gamma_ri (torch.Tensor) β It is a tensor that contains the gamma between real-parts and imaginary-parts.
gamma_ii (torch.Tensor) β It is a tensor that contains the gamma between imaginary-parts.
scale (bool, optional) β It defines if scaling should be used or not. It is equivalent to the real-valued batchnormalization scaling (default True).
center (bool, optional,) β It defines if centering should be used or not. It is equivalent to the real-valued batchnormalization centering (default True).
layernorm (bool, optional) β It defines is c_standardization is called from a layernorm or a batchnorm layer (default False).
dim (int, optional) β It defines the axis that should be considered as the complex-valued axis (divided by 2 to get r and i) (default -1).
- Return type:
The complex normed tensor.
- speechbrain.nnet.complex_networks.c_normalization.c_standardization(input_centred, Vrr, Vii, Vri, layernorm=False, dim=-1)[source]ο
This function is used to standardize a centered tensor of complex numbers (mean of the set must be 0).
- Parameters:
input_centred (torch.Tensor) β It is the tensor to be normalized. The features dimension is divided by 2 with the first half corresponding to the real-parts and the second half to the imaginary parts.
Vrr (torch.Tensor) β It is a tensor that contains the covariance between real-parts.
Vii (torch.Tensor) β It is a tensor that contains the covariance between imaginary-parts.
Vri (torch.Tensor) β It is a tensor that contains the covariance between real-parts and imaginary-parts.
layernorm (bool, optional) β It defines is c_standardization is called from a layernorm or a batchnorm layer (default False).
dim (int, optional) β It defines the axis that should be considered as the complex-valued axis (divided by 2 to get r and i) (default -1).
- Return type:
The standardizes centered tensor.