linearalgebra.lib
A library of linear algebra functions. Its official prefix is la.
This library adds some new linear algebra functions:
determinant
minor
inverse
transpose2
matMul matrix multiplication
identity
diag
How does it work? An NxM matrix can be flattened into a bus si.bus(N*M). These buses can be passed to functions as long as N and sometimes M (if the matrix need not be square) are passed too.
Some things to think about going forward
Implications for ML in Faust
Next step of making a "Dense"/"Linear" layer from machine learning.
Where in the libraries should ReLU go?
What about 3D tensors instead of 2D matrices? Image convolutions take place on 3D tensors shaped HxWxC.
Design of matMul
Currently the design is matMul(J, K, L, M, leftHandMat, rightHandMat) where leftHandMat is JxK and rightHandMat is LxM.
It would also be neat to have matMul(J, K, rightHandMat, L, M, leftHandMat).
Then a "packed" matrix could be consistently stored as a combination of a 2-channel "header" N, M and the values si.bus(N*M).
This would ultimately enable result = packedLeftHand : matMul(packedRightHand); for the equivalent numpy code: result = packedLeftHand @ packedRightHand;.
References
(la.)determinant
Calculates the determinant of a bus that represents
an NxN matrix.
Usage
si.bus(N*N) : determinant(N) : _
Where:
N: the size of each axis of the matrix.
(la.)minor
An utility for finding the matrix minor when inverting a matrix.
It returns the determinant of the submatrix formed by deleting the row at
index ROW and column at index COL.
The following implementation doesn't work but looks simple.
minor(N, ROW, COL) = par(r, N, par(c, N, select2((ROW==r)||(COL==c),_,!))) : determinant(N-1);
Usage
si.bus(N*N) : minor(N, ROW, COL) : _
Where:
N: the size of each axis of the matrix.ROW: the selected position on 0th dimension of the matrix (0 <= ROW < N)COL: the selected position on the 1st dimension of the matrix (0 <= COL < N)
References
(la.)inverse
Inverts a matrix. The incoming bus represents an NxN matrix.
Note, this is an unsafe operation since not all matrices are invertible.
Usage
si.bus(N*N) : inverse(N) : si.bus(N*N)
Where:
N: the size of each axis of the matrix.
(la.)transpose2
Transposes an NxM matrix stored in row-major order, resulting
in an MxN matrix stored in row-major order.
Usage
si.bus(N*M) : transpose2(N, M) : si.bus(M*N)
Where:
N: the number of rows in the input matrixM: the number of columns in the input matrix
(la.)matMul
Multiply a JxK matrix (mat1) and an LxM matrix (mat2) to produce a JxM matrix.
Note that K==L.
Both matrices should use row-major order.
In terms of numpy, this function is mat1 @ mat2.
Usage
matMul(J, K, L, M, si.bus(J*K), si.bus(L*M)) : si.bus(J*M)
Where:
J: the number of rows inmat1K: the number of columns inmat1L: the number of rows inmat2M: the number of columns inmat2
(la.)identity
Creates an NxN identity matrix.
Usage
identity(N) : si.bus(N*N)
Where:
N: The size of each axis of the identity matrix.
(la.)diag
Creates a diagonal matrix of size NxN with specified
values along the diagonal.
Usage
si.bus(N) : diag(N) : si.bus(N*N)
Where:
N: The size of each axis of the matrix.