speechbrain.nnet.quaternion_networks.q_ops module¶
This library implements different operations needed by quaternion- valued architectures. This work is inspired by: “Quaternion neural networks” - Parcollet T. “Quaternion recurrent neural networks” - Parcollet T. et al. “Quaternion convolutional neural networks for end-to-end automatic speech recognition” - Parcollet T. et al. “Deep quaternion networks” - Gaudet Chase J. et al.
- Authors
Titouan Parcollet 2020
Summary¶
Classes:
This class redefine the backpropagation of a quaternion linear layer (not a spinor layer). |
Functions:
Applies the weight initialization function given to the parameters. |
|
Applies the weight initialization function given to the parameters. |
|
Check the quaternion-valued shape for a linear layer. |
|
Applies a quaternion convolution transformation to the incoming data: It is important to notice that the forward phase of a QCNN is defined as W * Inputs (with * equal to the Hamilton product). |
|
Applies a quaternion rotation transformation to the incoming data: The rotation W*x*W^t can be replaced by R*x following: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation Works for unitary and non-unitary weights (they will be normalized). |
|
Returns a matrix of quaternion numbers initialized with the method described in “Quaternion Recurrent Neural Network ” - Parcollt T. |
|
Applies a quaternion linear transformation to the incoming data: It is important to notice that the forward phase of a QNN is defined as W * Inputs (with * equal to the Hamilton product). |
|
Applies a quaternion rotation transformation to the incoming data: The rotation W*x*W^t can be replaced by R*x following: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation Works for unitary and non-unitary weights (they will be normalized). |
|
Returns a matrix of unitary quaternion numbers. |
Reference¶
-
class
speechbrain.nnet.quaternion_networks.q_ops.
QuaternionLinearCustomBackward
[source]¶ Bases:
torch.autograd.function.Function
This class redefine the backpropagation of a quaternion linear layer (not a spinor layer). By doing so, we can save up to 4x memory, but it is also 2x slower than ‘quaternion_linear_op’. It should be used within speechbrain.nnet.quaternion_networks.linear.QuaternionLinear.
-
static
forward
(ctx, input, r_weight, i_weight, j_weight, k_weight, bias)[source]¶ Applies a quaternion linear transformation to the incoming data: It is important to notice that the forward phase of a QNN is defined as W * Inputs (with * equal to the Hamilton product). The constructed cat_kernels_4_quaternion is a modified version of the quaternion representation so when we do torch.mm(Input,W) it’s equivalent to W * Inputs.
- Parameters
input (torch.Tensor) – Quaternion input tensor to be transformed. Shape: [batch*time, X].
r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.
i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.
j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.
k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.
bias (torch.Parameter) –
-
static
backward
(ctx, grad_output)[source]¶ Run the backward phase of the forward call defined above. This implementation follows the quaternion backpropagation of a quaternion layer that can be found in “Quaternion neural networks” - Parcollet T. Page 48.
- Parameters
input (torch.Tensor) – Quaternion input tensor to be transformed.
r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.
i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.
j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.
k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.
bias (torch.Parameter) –
-
static
-
speechbrain.nnet.quaternion_networks.q_ops.
quaternion_linear_op
(input, r_weight, i_weight, j_weight, k_weight, bias)[source]¶ Applies a quaternion linear transformation to the incoming data: It is important to notice that the forward phase of a QNN is defined as W * Inputs (with * equal to the Hamilton product). The constructed cat_kernels_4_quaternion is a modified version of the quaternion representation so when we do torch.mm(Input,W) it’s equivalent to W * Inputs.
- Parameters
input (torch.Tensor) – Quaternion input tensor to be transformed.
r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.
i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.
j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.
k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.
bias (torch.Parameter) –
-
speechbrain.nnet.quaternion_networks.q_ops.
quaternion_linear_rotation_op
(input, r_weight, i_weight, j_weight, k_weight, bias, scale, zero_kernel)[source]¶ Applies a quaternion rotation transformation to the incoming data: The rotation W*x*W^t can be replaced by R*x following: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation Works for unitary and non-unitary weights (they will be normalized). The initial size of the input must be a multiple of 4 with the real part equal to zero. Rotations only affect the vector part of a quaternion.
- Parameters
input (torch.Tensor) – Quaternion input tensor to be transformed.
r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.
i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.
j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.
k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.
bias (torch.Parameter) –
scale (torch.Parameter) – In the context of a spinor neural network, multiple rotations of the input vector x are performed and summed. Hence, the norm of the output vector always increases with the number of layers, making the neural network instable with deep configurations. The scale parameters are learnable parameters that acts like gates by multiplying the output vector with a small trainable parameter.
zero_kernel (torch.Parameter) – The zero kernel is simply a tensor of zeros with require grad = False. Its shape is equivalent to a quaternion component shape. In fact, it is only needed to make the dimensions match when using the rotation matrix : https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation
-
speechbrain.nnet.quaternion_networks.q_ops.
quaternion_conv_rotation_op
(input, r_weight, i_weight, j_weight, k_weight, bias, scale, zero_kernel, stride: int, padding: int, groups: int, dilation: int, conv1d: bool)[source]¶ Applies a quaternion rotation transformation to the incoming data: The rotation W*x*W^t can be replaced by R*x following: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation Works for unitary and non-unitary weights (they will be normalized). The initial size of the input must be a multiple of 4 with the real part equal to zero. Rotations only affect the vector part of a quaternion.
- Parameters
input (torch.Tensor) – Quaternion input tensor to be transformed.
conv1d (bool) – If true, a 1D convolution operation will be applied. Otherwise, a 2D convolution is called.
r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.
i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.
j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.
k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.
bias (torch.Parameter) –
scale (torch.Parameter) – In the context of a spinor neural network, multiple rotations of the input vector x are performed and summed. Hence, the norm of the output vector always increases with the number of layers, making the neural network instable with deep configurations. The scale parameters are learnable parameters that acts like gates by multiplying the output vector with a small trainable parameter.
zero_kernel (torch.Parameter) – The zero kernel is simply a tensor of zeros with require grad = False. Its shape is equivalent to a quaternion component shape. In fact, it is only needed to make the dimensions match when using the rotation matrix : https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation
-
speechbrain.nnet.quaternion_networks.q_ops.
quaternion_conv_op
(input, r_weight, i_weight, j_weight, k_weight, bias, stride: int, padding: int, groups: int, dilation: int, conv1d: bool)[source]¶ Applies a quaternion convolution transformation to the incoming data: It is important to notice that the forward phase of a QCNN is defined as W * Inputs (with * equal to the Hamilton product). The constructed cat_kernels_4_quaternion is a modified version of the quaternion representation so when we do torch.mm(Input,W) it’s equivalent to W * Inputs.
- Parameters
input (torch.Tensor) – Quaternion input tensor to be transformed.
conv1d (bool) – If true, a 1D convolution operation will be applied. Otherwise, a 2D convolution is called.
r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.
i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.
j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.
k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.
bias (torch.Parameter) –
stride (int) – Stride factor of the convolutional filters.
padding (int) – Amount of padding. See torch.nn documentation for more information.
groups (int) – This option specifies the convolutional groups. See torch.nn documentation for more information.
dilation (int) – Dilation factor of the convolutional filters.
-
speechbrain.nnet.quaternion_networks.q_ops.
quaternion_init
(in_features, out_features, kernel_size=None, criterion='glorot')[source]¶ Returns a matrix of quaternion numbers initialized with the method described in “Quaternion Recurrent Neural Network ” - Parcollt T.
-
speechbrain.nnet.quaternion_networks.q_ops.
unitary_init
(in_features, out_features, kernel_size=None, criterion='he')[source]¶ Returns a matrix of unitary quaternion numbers.
-
speechbrain.nnet.quaternion_networks.q_ops.
affect_init
(r_weight, i_weight, j_weight, k_weight, init_func, init_criterion)[source]¶ Applies the weight initialization function given to the parameters.
- Parameters
r_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
i_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
j_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
k_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
init_func (function) – (unitary_init, quaternion_init)
init_criterion (str) – (glorot, he)
-
speechbrain.nnet.quaternion_networks.q_ops.
affect_conv_init
(r_weight, i_weight, j_weight, k_weight, kernel_size, init_func, init_criterion)[source]¶ Applies the weight initialization function given to the parameters. This is specificaly written for convolutional layers.
- Parameters
r_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
i_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
j_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
k_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
kernel_size (int) – Kernel size.
init_func (function) – (unitary_init, quaternion_init)
init_criterion (str) – (glorot, he)