Basics of Matrix, Vector, and various matrix operations
Linear Algebra is the field of mathematics concerned with vectors, matrices, and linear transforms. It creates a strong foundation for Machine learning and the implementation of algorithms in code.
We frequently face regression or classification problems in machine learning, and then error minimization techniques are used. We use linear algebra for these computations on large amounts of data.
These are the essential mathematical areas to understand the basic concepts of machine learning and deep learning.
The Matrix is a table of numbers arranged in rows and columns. It is like a 2-dimensional array. We write Dimensions of the Matrix as [ number of rows X number of columns ]
Matrices of i x j dimensions can also be represented as.
Entries of a Matrix are represented by their row and column numbers. For example, We have a matrix A. Any element in this Matrix can be described as, where i and j represent row and column of the given element.
The vector is a particular case of a Matrix, having a single column.
It's an n x 1 Matrix.
Here y is a 3-dimensional vector and can be represented as.
To represent an element in a vector, we use the following notation:
( Here y1 = 546; y2 = 678; y3=345 )
Vectors can be 1-indexed and also 0-indexed unless specified. It's safe to use 1-indexed.
Usually, uppercase letters are used to refer to Matrices, whereas lowercase letters are used for Vectors.
Elements of the first Matrix are added to the corresponding elements of the second Matrix by using row and column numbers. Only the same dimension matrices can be added, as the new Matrix formed will also have the exact dimensions.
A scalar is just a real number that is multiplied by every element of the Matrix, one element at a time.
Matrix multiplication produces a resultant matrix from two matrices. The primary condition for matrix multiplication is that the number of columns of the first Matrix must be the same as the number of rows of the second Matrix. The resultant Matrix has the number of rows from the first and the number of columns from the second Matrix. The product of matrices X and Y is denoted as XY.
Here we multiply each element in the row of the first Matrix with the corresponding elements in the column of the second Matrix and add them to obtain element ij of
the resultant Matrix. For example c1 = a1.b1 + a2.b2 + a3.b3
Multiplication with vector
To obtain, we multiply A’s row with elements of vector x and add them up.
The number of columns in Matrix A has to match with the number of rows in matrix B.
Matrix multiplication properties
- Non-Commutative: Let A and B be matrices. Then.
E.g., We are getting different resultant matrices in the below cases.
2. Associative: Matrix multiplication is associative, Let A, B, C be three matrices.
Then A x (B x C) = (A x B) x C
Identity Matrix of size n is defined as [ n x n ] square Matrix with 1's along the diagonal and 0's elsewhere. It is denoted by.
For any matrix A: A.I = I.A = A
Below are some examples of Identity matrices :
A diagonal matrix of size n is defined as [ n x n ] square Matrix with all the elements as 0 except those on the diagonal.
Matrix Inverse operation
The inverse of a matrix is another matrix that results in an identity matrix on multiplication with the given Matrix. Not all numbers have an inverse, i.e., 0^-1 does not exist; Similarly, not all matrices have an Inverse. Matrices that do not have an inverse are "singular" or "degenerate" matrices.
If A is an [m x m] matrix and if it has an inverse,
Matrix Transpose operation
Transpose of a Matrix A is denoted by. Let A be an [m x n] Matrix and let. Then B is an [n x m] Matrix where. Transpose of a Matrix is similar to mirroring the Matrix over the diagonal axis.
A simple application of the above concepts in ML
Let’s assume we have three competing hypotheses:
And we need to test it for the following values of x:
It can be done very efficiently by creating matrices for x and hypothesis and using matrix multiplication.
Source: Linear algebra review by Andrew ng
Frequently Asked Questions
- Is non-square Matrix Invertible?
Non-square matrices (m x n matrix for which m ≠ n) do not have an inverse, But in some cases, such matrices might have a left inverse or right inverse.
- Is Linear algebra required for machine learning?
It is not necessary to learn linear algebra before getting started with machine learning; However, It is advisable to learn linear algebra if you wish to get a better and deep understanding of machine learning algorithms and their applications.
- What are the applications of linear algebra in computer science?
Below are some basic applications of linear algebra in computer science:
(vi)Graph-Algorithms and many more.
In this article, we have explored the basic Linear Algebra and Basics of Matrix, Vector, and various matrix operations for Machine Learning. I hope you are clear with all the material which is present in this article.
Linear algebra plays a key role in machine learning algorithms due to vectors, matrices, and several operations to handle vectors. Linear algebra can handle a large amount of data, or we can say, "linear algebra is the basic mathematics of data.". I hope this article was helpful.