Machine Learning - Linear Algebra

Linear Algebra review
Fundation of machine leaning(Matrices and vectors)


  • Rectangular array of numbers
  • Dimension of matrix number of rows x number of columns


  • An n x 1 matrix

Scalar multiplication

  • Matrix Addition: you can only add two same dimension matrix
  • Multiplication: scalar number multiply matrix will take every elements to multiple result in matrix, the result is same as dimension matrix

Matrix-vector multiplication

  • m x n matrix multiply n dimention vector turns out m demention vector.

matrix multiply

  • m x n matrix multiply n x o matrix turns out m x o matrix.

Matrix Multiplication Properties

  • Let A and B be matrices. Then in general, A X B ≠ B X A.(not commutative)

  • A X B X C. Let D = B X C, then A X D, equals E = A X B, then E X C.(associative)

  • Identity Matrix, Denoted I ( or Inxn). For any matrix A, A X I = I X A = A, but the I is not same maybe.

Inverse and transpose

  • Not all numbers have an inverse.

  • If A is an m x m matrix(square matrix), and if it has an inverse, A A-1 = A-1 A = I.

  • Matrices that don’t have an inverse are “singular” or “degenerate”

  • Matrix transpose. A and AT

  • Let A be an m x n matrix, and let B = AT. Then B is an n x m matrix, and Bij = Aji