speechbrain.nnet.quaternion_networks.q_ops module
This library implements different operations needed by quaternion- valued architectures. This work is inspired by: “Quaternion neural networks” - Parcollet T. “Quaternion recurrent neural networks” - Parcollet T. et al. “Quaternion convolutional neural networks for end-to-end automatic speech recognition” - Parcollet T. et al. “Deep quaternion networks” - Gaudet Chase J. et al.
- Authors
Titouan Parcollet 2020
Summary
Classes:
This class redefine the backpropagation of a quaternion linear layer (not a spinor layer). |
Functions:
Applies the weight initialization function given to the parameters. |
|
Applies the weight initialization function given to the parameters. |
|
Check the quaternion-valued shape for a linear layer. |
|
Applies a quaternion convolution transformation to the incoming data: It is important to notice that the forward phase of a QCNN is defined as W * Inputs (with * equal to the Hamilton product). |
|
Applies a quaternion rotation transformation to the incoming data: The rotation W*x*W^t can be replaced by R*x following: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation Works for unitary and non-unitary weights (they will be normalized). |
|
Returns a matrix of quaternion numbers initialized with the method described in "Quaternion Recurrent Neural Network " - Parcollet T. |
|
Applies a quaternion linear transformation to the incoming data: It is important to notice that the forward phase of a QNN is defined as W * Inputs (with * equal to the Hamilton product). |
|
Applies a quaternion rotation transformation to the incoming data: The rotation W*x*W^t can be replaced by R*x following: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation Works for unitary and non-unitary weights (they will be normalized). |
|
Returns a matrix of unitary quaternion numbers. |
Reference
- class speechbrain.nnet.quaternion_networks.q_ops.QuaternionLinearCustomBackward(*args, **kwargs)[source]
Bases:
Function
This class redefine the backpropagation of a quaternion linear layer (not a spinor layer). By doing so, we can save up to 4x memory, but it is also 2x slower than ‘quaternion_linear_op’. It should be used within speechbrain.nnet.quaternion_networks.linear.QuaternionLinear.
- static forward(ctx, input, r_weight, i_weight, j_weight, k_weight, bias)[source]
Applies a quaternion linear transformation to the incoming data: It is important to notice that the forward phase of a QNN is defined as W * Inputs (with * equal to the Hamilton product). The constructed cat_kernels_4_quaternion is a modified version of the quaternion representation so when we do torch.mm(Input,W) it’s equivalent to W * Inputs.
- Parameters:
ctx (PyTorch context object) – Used to save the context necessary to perform a backwards pass.
input (torch.Tensor) – Quaternion input tensor to be transformed. Shape: [batch*time, X].
r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.
i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.
j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.
k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.
bias (torch.Parameter)
- Return type:
The linearly transformed quaternions
- static backward(ctx, grad_output)[source]
Run the backward phase of the forward call defined above. This implementation follows the quaternion backpropagation of a quaternion layer that can be found in “Quaternion neural networks” - Parcollet T. Page 48.
- Parameters:
ctx (Pytorch context object) – Contains saved weights and bias
grad_output (torch.Tensor) – The output of the forward part
- Return type:
The corresponding gradients of this op
- speechbrain.nnet.quaternion_networks.q_ops.quaternion_linear_op(input, r_weight, i_weight, j_weight, k_weight, bias)[source]
Applies a quaternion linear transformation to the incoming data: It is important to notice that the forward phase of a QNN is defined as W * Inputs (with * equal to the Hamilton product). The constructed cat_kernels_4_quaternion is a modified version of the quaternion representation so when we do torch.mm(Input,W) it’s equivalent to W * Inputs.
- Parameters:
input (torch.Tensor) – Quaternion input tensor to be transformed.
r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.
i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.
j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.
k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.
bias (torch.Parameter)
- Return type:
The linearly transformed quaternions
- speechbrain.nnet.quaternion_networks.q_ops.quaternion_linear_rotation_op(input, r_weight, i_weight, j_weight, k_weight, bias, scale, zero_kernel)[source]
Applies a quaternion rotation transformation to the incoming data: The rotation W*x*W^t can be replaced by R*x following: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation Works for unitary and non-unitary weights (they will be normalized). The initial size of the input must be a multiple of 4 with the real part equal to zero. Rotations only affect the vector part of a quaternion.
- Parameters:
input (torch.Tensor) – Quaternion input tensor to be transformed.
r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.
i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.
j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.
k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.
bias (torch.Parameter)
scale (torch.Parameter) – In the context of a spinor neural network, multiple rotations of the input vector x are performed and summed. Hence, the norm of the output vector always increases with the number of layers, making the neural network instable with deep configurations. The scale parameters are learnable parameters that acts like gates by multiplying the output vector with a small trainable parameter.
zero_kernel (torch.Parameter) – The zero kernel is simply a tensor of zeros with require grad = False. Its shape is equivalent to a quaternion component shape. In fact, it is only needed to make the dimensions match when using the rotation matrix : https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation
- Return type:
The linearly rotated quaternions
- speechbrain.nnet.quaternion_networks.q_ops.quaternion_conv_rotation_op(input, r_weight, i_weight, j_weight, k_weight, bias, scale, zero_kernel, stride: int, padding: int, groups: int, dilation: int, conv1d: bool)[source]
Applies a quaternion rotation transformation to the incoming data: The rotation W*x*W^t can be replaced by R*x following: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation Works for unitary and non-unitary weights (they will be normalized). The initial size of the input must be a multiple of 4 with the real part equal to zero. Rotations only affect the vector part of a quaternion.
- Parameters:
input (torch.Tensor) – Quaternion input tensor to be transformed.
r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.
i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.
j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.
k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.
bias (torch.Parameter)
scale (torch.Parameter) – In the context of a spinor neural network, multiple rotations of the input vector x are performed and summed. Hence, the norm of the output vector always increases with the number of layers, making the neural network instable with deep configurations. The scale parameters are learnable parameters that acts like gates by multiplying the output vector with a small trainable parameter.
zero_kernel (torch.Parameter) – The zero kernel is simply a tensor of zeros with require grad = False. Its shape is equivalent to a quaternion component shape. In fact, it is only needed to make the dimensions match when using the rotation matrix : https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation
stride (int) – Stride factor of the convolutional filters.
padding (int) – Amount of padding. See torch.nn documentation for more information.
groups (int) – This option specifies the convolutional groups. See torch.nn documentation for more information.
dilation (int) – Dilation factor of the convolutional filters.
conv1d (bool) – If true, a 1D convolution operation will be applied. Otherwise, a 2D convolution is called.
- Return type:
The rotated quaternion inputs
- speechbrain.nnet.quaternion_networks.q_ops.quaternion_conv_op(input, r_weight, i_weight, j_weight, k_weight, bias, stride: int, padding: int, groups: int, dilation: int, conv1d: bool)[source]
Applies a quaternion convolution transformation to the incoming data: It is important to notice that the forward phase of a QCNN is defined as W * Inputs (with * equal to the Hamilton product). The constructed cat_kernels_4_quaternion is a modified version of the quaternion representation so when we do torch.mm(Input,W) it’s equivalent to W * Inputs.
- Parameters:
input (torch.Tensor) – Quaternion input tensor to be transformed.
r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.
i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.
j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.
k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.
bias (torch.Parameter)
stride (int) – Stride factor of the convolutional filters.
padding (int) – Amount of padding. See torch.nn documentation for more information.
groups (int) – This option specifies the convolutional groups. See torch.nn documentation for more information.
dilation (int) – Dilation factor of the convolutional filters.
conv1d (bool) – If true, a 1D convolution operation will be applied. Otherwise, a 2D convolution is called.
- Return type:
The convolved quaternion inputs
- speechbrain.nnet.quaternion_networks.q_ops.quaternion_init(in_features, out_features, kernel_size=None, criterion='glorot')[source]
Returns a matrix of quaternion numbers initialized with the method described in “Quaternion Recurrent Neural Network “ - Parcollet T.
- Parameters:
- Return type:
Matrix of initialized quaternion numbers
- speechbrain.nnet.quaternion_networks.q_ops.unitary_init(in_features, out_features, kernel_size=None, criterion='he')[source]
Returns a matrix of unitary quaternion numbers.
- Parameters:
- Return type:
Matrix of unitary quaternion numbers.
- speechbrain.nnet.quaternion_networks.q_ops.affect_init(r_weight, i_weight, j_weight, k_weight, init_func, init_criterion)[source]
Applies the weight initialization function given to the parameters.
- Parameters:
r_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
i_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
j_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
k_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
init_func (function) – (unitary_init, quaternion_init)
init_criterion (str) – (glorot, he)
- speechbrain.nnet.quaternion_networks.q_ops.affect_conv_init(r_weight, i_weight, j_weight, k_weight, kernel_size, init_func, init_criterion)[source]
Applies the weight initialization function given to the parameters. This is specifically written for convolutional layers.
- Parameters:
r_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
i_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
j_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
k_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)
kernel_size (int) – Kernel size.
init_func (function) – (unitary_init, quaternion_init)
init_criterion (str) – (glorot, he)