speechbrain.nnet.quaternion_networks.q_ops module

This library implements different operations needed by quaternion- valued architectures. This work is inspired by: “Quaternion neural networks” - Parcollet T. “Quaternion recurrent neural networks” - Parcollet T. et al. “Quaternion convolutional neural networks for end-to-end automatic speech recognition” - Parcollet T. et al. “Deep quaternion networks” - Gaudet Chase J. et al.

Authors
  • Titouan Parcollet 2020

Summary

Classes:

QuaternionLinearCustomBackward

This class redefine the backpropagation of a quaternion linear layer (not a spinor layer).

Functions:

affect_conv_init

Applies the weight initialization function given to the parameters.

affect_init

Applies the weight initialization function given to the parameters.

check_quaternion_input

Check the quaternion-valued shape for a linear layer.

quaternion_conv_op

Applies a quaternion convolution transformation to the incoming data: It is important to notice that the forward phase of a QCNN is defined as W * Inputs (with * equal to the Hamilton product).

quaternion_conv_rotation_op

Applies a quaternion rotation transformation to the incoming data: The rotation W*x*W^t can be replaced by R*x following: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation Works for unitary and non-unitary weights (they will be normalized).

quaternion_init

Returns a matrix of quaternion numbers initialized with the method described in "Quaternion Recurrent Neural Network " - Parcollt T.

quaternion_linear_op

Applies a quaternion linear transformation to the incoming data: It is important to notice that the forward phase of a QNN is defined as W * Inputs (with * equal to the Hamilton product).

quaternion_linear_rotation_op

Applies a quaternion rotation transformation to the incoming data: The rotation W*x*W^t can be replaced by R*x following: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation Works for unitary and non-unitary weights (they will be normalized).

unitary_init

Returns a matrix of unitary quaternion numbers.

Reference

class speechbrain.nnet.quaternion_networks.q_ops.QuaternionLinearCustomBackward(*args, **kwargs)[source]

Bases: Function

This class redefine the backpropagation of a quaternion linear layer (not a spinor layer). By doing so, we can save up to 4x memory, but it is also 2x slower than ‘quaternion_linear_op’. It should be used within speechbrain.nnet.quaternion_networks.linear.QuaternionLinear.

static forward(ctx, input, r_weight, i_weight, j_weight, k_weight, bias)[source]

Applies a quaternion linear transformation to the incoming data: It is important to notice that the forward phase of a QNN is defined as W * Inputs (with * equal to the Hamilton product). The constructed cat_kernels_4_quaternion is a modified version of the quaternion representation so when we do torch.mm(Input,W) it’s equivalent to W * Inputs.

Parameters:
  • input (torch.Tensor) – Quaternion input tensor to be transformed. Shape: [batch*time, X].

  • r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.

  • i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.

  • j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.

  • k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.

  • bias (torch.Parameter) –

static backward(ctx, grad_output)[source]

Run the backward phase of the forward call defined above. This implementation follows the quaternion backpropagation of a quaternion layer that can be found in “Quaternion neural networks” - Parcollet T. Page 48.

Parameters:
  • input (torch.Tensor) – Quaternion input tensor to be transformed.

  • r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.

  • i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.

  • j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.

  • k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.

  • bias (torch.Parameter) –

speechbrain.nnet.quaternion_networks.q_ops.quaternion_linear_op(input, r_weight, i_weight, j_weight, k_weight, bias)[source]

Applies a quaternion linear transformation to the incoming data: It is important to notice that the forward phase of a QNN is defined as W * Inputs (with * equal to the Hamilton product). The constructed cat_kernels_4_quaternion is a modified version of the quaternion representation so when we do torch.mm(Input,W) it’s equivalent to W * Inputs.

Parameters:
  • input (torch.Tensor) – Quaternion input tensor to be transformed.

  • r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.

  • i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.

  • j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.

  • k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.

  • bias (torch.Parameter) –

speechbrain.nnet.quaternion_networks.q_ops.quaternion_linear_rotation_op(input, r_weight, i_weight, j_weight, k_weight, bias, scale, zero_kernel)[source]

Applies a quaternion rotation transformation to the incoming data: The rotation W*x*W^t can be replaced by R*x following: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation Works for unitary and non-unitary weights (they will be normalized). The initial size of the input must be a multiple of 4 with the real part equal to zero. Rotations only affect the vector part of a quaternion.

Parameters:
  • input (torch.Tensor) – Quaternion input tensor to be transformed.

  • r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.

  • i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.

  • j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.

  • k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.

  • bias (torch.Parameter) –

  • scale (torch.Parameter) – In the context of a spinor neural network, multiple rotations of the input vector x are performed and summed. Hence, the norm of the output vector always increases with the number of layers, making the neural network instable with deep configurations. The scale parameters are learnable parameters that acts like gates by multiplying the output vector with a small trainable parameter.

  • zero_kernel (torch.Parameter) – The zero kernel is simply a tensor of zeros with require grad = False. Its shape is equivalent to a quaternion component shape. In fact, it is only needed to make the dimensions match when using the rotation matrix : https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation

speechbrain.nnet.quaternion_networks.q_ops.quaternion_conv_rotation_op(input, r_weight, i_weight, j_weight, k_weight, bias, scale, zero_kernel, stride: int, padding: int, groups: int, dilation: int, conv1d: bool)[source]

Applies a quaternion rotation transformation to the incoming data: The rotation W*x*W^t can be replaced by R*x following: https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation Works for unitary and non-unitary weights (they will be normalized). The initial size of the input must be a multiple of 4 with the real part equal to zero. Rotations only affect the vector part of a quaternion.

Parameters:
  • input (torch.Tensor) – Quaternion input tensor to be transformed.

  • conv1d (bool) – If true, a 1D convolution operation will be applied. Otherwise, a 2D convolution is called.

  • r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.

  • i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.

  • j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.

  • k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.

  • bias (torch.Parameter) –

  • scale (torch.Parameter) – In the context of a spinor neural network, multiple rotations of the input vector x are performed and summed. Hence, the norm of the output vector always increases with the number of layers, making the neural network instable with deep configurations. The scale parameters are learnable parameters that acts like gates by multiplying the output vector with a small trainable parameter.

  • zero_kernel (torch.Parameter) – The zero kernel is simply a tensor of zeros with require grad = False. Its shape is equivalent to a quaternion component shape. In fact, it is only needed to make the dimensions match when using the rotation matrix : https://en.wikipedia.org/wiki/Quaternions_and_spatial_rotation

speechbrain.nnet.quaternion_networks.q_ops.quaternion_conv_op(input, r_weight, i_weight, j_weight, k_weight, bias, stride: int, padding: int, groups: int, dilation: int, conv1d: bool)[source]

Applies a quaternion convolution transformation to the incoming data: It is important to notice that the forward phase of a QCNN is defined as W * Inputs (with * equal to the Hamilton product). The constructed cat_kernels_4_quaternion is a modified version of the quaternion representation so when we do torch.mm(Input,W) it’s equivalent to W * Inputs.

Parameters:
  • input (torch.Tensor) – Quaternion input tensor to be transformed.

  • conv1d (bool) – If true, a 1D convolution operation will be applied. Otherwise, a 2D convolution is called.

  • r_weight (torch.Parameter) – Real part of the quaternion weight matrix of this layer.

  • i_weight (torch.Parameter) – First imaginary part of the quaternion weight matrix of this layer.

  • j_weight (torch.Parameter) – Second imaginary part of the quaternion weight matrix of this layer.

  • k_weight (torch.Parameter) – Third imaginary part of the quaternion weight matrix of this layer.

  • bias (torch.Parameter) –

  • stride (int) – Stride factor of the convolutional filters.

  • padding (int) – Amount of padding. See torch.nn documentation for more information.

  • groups (int) – This option specifies the convolutional groups. See torch.nn documentation for more information.

  • dilation (int) – Dilation factor of the convolutional filters.

speechbrain.nnet.quaternion_networks.q_ops.quaternion_init(in_features, out_features, kernel_size=None, criterion='glorot')[source]

Returns a matrix of quaternion numbers initialized with the method described in “Quaternion Recurrent Neural Network ” - Parcollt T.

Parameters:
  • in_features (int) – Number of real values of the input layer (quaternion // 4).

  • out_features (int) – Number of real values of the output layer (quaternion // 4).

  • kernel_size (int) – Kernel_size for convolutional layers (ex: (3,3)).

  • criterion (str) – (glorot, he)

speechbrain.nnet.quaternion_networks.q_ops.unitary_init(in_features, out_features, kernel_size=None, criterion='he')[source]

Returns a matrix of unitary quaternion numbers.

Parameters:
  • in_features (int) – Number of real values of the input layer (quaternion // 4).

  • out_features (int) – Number of real values of the output layer (quaternion // 4).

  • kernel_size (int) – Kernel_size for convolutional layers (ex: (3,3)).

  • criterion (str) – (glorot, he)

speechbrain.nnet.quaternion_networks.q_ops.affect_init(r_weight, i_weight, j_weight, k_weight, init_func, init_criterion)[source]

Applies the weight initialization function given to the parameters.

Parameters:
  • r_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)

  • i_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)

  • j_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)

  • k_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)

  • init_func (function) – (unitary_init, quaternion_init)

  • init_criterion (str) – (glorot, he)

speechbrain.nnet.quaternion_networks.q_ops.affect_conv_init(r_weight, i_weight, j_weight, k_weight, kernel_size, init_func, init_criterion)[source]

Applies the weight initialization function given to the parameters. This is specifically written for convolutional layers.

Parameters:
  • r_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)

  • i_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)

  • j_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)

  • k_weight (torch.Parameters) – (nb_quaternion_in, nb_quaternion_out)

  • kernel_size (int) – Kernel size.

  • init_func (function) – (unitary_init, quaternion_init)

  • init_criterion (str) – (glorot, he)

speechbrain.nnet.quaternion_networks.q_ops.check_quaternion_input(input_shape)[source]

Check the quaternion-valued shape for a linear layer.

Parameters:

input_shape (tuple) – Expected shape of the input.