Zhaoyan LU
From Dusk Till Dawn

Linear Algebra

Published on • Updated on

This post contains notes on the MIT OpenCourseWare Linear Algebra course.

Contents

LECTURE 1: The Geometry of Linear Equations

  • Ax is a combination of the columns of A.

$$ A = \begin{bmatrix} 2 & 5 \\ 1 & 2 \end{bmatrix}, x = \begin{bmatrix} 1 \\ 2 \end{bmatrix} $$

$$ Ax = \begin{bmatrix} 2 & 5 \\ 1 & 2 \end{bmatrix} \begin{bmatrix} 1 \\ 2 \end{bmatrix} = 1 \begin{bmatrix} 2 \\ 1 \end{bmatrix} + 2 \begin{bmatrix} 5 \\ 2 \end{bmatrix} = \begin{bmatrix} 12 \\ 7 \end{bmatrix} $$

LECTURE 2: Elimination with Matrices

2.1 Example Equations

$$ \begin{align*} x + 2y + z &= 2 \\ 3x + 8y + z &= 12 \\ 4y + z &= 2 \end{align*} $$

2.2 Elimination

2.2.1 Success

$$ A = \begin{bmatrix} 1 & 2 & 1 \\ 3 & 8 & 1 \\ 0 & 4 & 1 \end{bmatrix} \underrightarrow{(2,1)} \begin{bmatrix} 1 & 2 & 1 \\ 0 & 2 & -2 \\ 0 & 4 & 1 \end{bmatrix} \underrightarrow{(3,2)} \begin{bmatrix} 1 & 2 & 1 \\ 0 & 2 & -2 \\ 0 & 0 & 5 \end{bmatrix} $$

Augmented matrix

$$ b = \begin{bmatrix} 2 \\ 12 \\ 2 \end{bmatrix} \rarr \begin{bmatrix} 2 \\ 6 \\ 2 \end{bmatrix} \rarr \begin{bmatrix} 2 \\ 6 \\ -10 \end{bmatrix} $$

$$ 1^{st}\space pivot \rarr A_{1,1} \rarr 1 $$

$$ U \coloneqq \begin{bmatrix} 1 & 2 & 1 \\ 0 & 2 & -2 \\ 0 & 0 & 5 \end{bmatrix} \\ c \coloneqq \begin{bmatrix} 2 \\ 6 \\ -10 \end{bmatrix} $$

2.2.2 Failure

  • Temporary failure: If a zero is encountered in the pivot position, we can perform a row exchange to rectify the situation.
  • Complete failure: If a zero is encountered and there are no rows below it for exchange, the matrix is not invertible.

2.3 Back-Substitution

$$ \begin{align*} x + 2y + z &= 2 \\ 2y - 2z &= 6 \\ 5z &= -10 \end{align*} \rarr \begin{align*} x &= 2 \\ y &= 1 \\ z &= -2 \end{align*} $$

2.4 Matrices Multiplication

$$ \begin{align*} matrix \times column &= matrix \\ row \times matrix &= row \end{align*} $$

$$ \begin{bmatrix} ? & ? & ? \\ ? & ? & ? \\ ? & ? & ? \end{bmatrix} \begin{bmatrix} 1 & 2 & 1 \\ 3 & 8 & 1 \\ 0 & 4 & 1 \end{bmatrix} = \begin{bmatrix} 1 & 2 & 1 \\ 0 & 2 & -2 \\ 0 & 4 & 1 \end{bmatrix} \rarr \begin{bmatrix} 1 & 0 & 0 \\ -3 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} 1 & 2 & 1 \\ 3 & 8 & 1 \\ 0 & 4 & 1 \end{bmatrix} = \begin{bmatrix} 1 & 2 & 1 \\ 0 & 2 & -2 \\ 0 & 4 & 1 \end{bmatrix} $$

2.4.1 Elementary Matrix

We call it E for elementary or elimination.

$$ \begin{align*} E_{21} &\coloneqq \begin{bmatrix} 1 & 0 & 0 \\ -3 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \\ E_{32} &\coloneqq \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -2 & 1 \end{bmatrix} \end{align*} $$

  • Associative Law

$$ \begin{align*} E_{32}(E_{21}A) &= U \\ (E_{32}E_{21})A &= U \end{align*} \rarr E_{32}E_{21} = \begin{bmatrix} 1 & 2 & 1 \\ 0 & 2 & -2 \\ 0 & 4 & 1 \end{bmatrix} $$

2.4.2 Permutation Matrix

  • Exchange rows 1 and 2

$$ \begin{bmatrix} ? & ? \\ ? & ? \end{bmatrix} \begin{bmatrix} a & b \\ c & d \end{bmatrix} = \begin{bmatrix} c & d \\ a & b \end{bmatrix} \rarr \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} \begin{bmatrix} a & b \\ c & d \end{bmatrix} = \begin{bmatrix} c & d \\ a & b \end{bmatrix} $$

We call it P for permutation.

$$ P \coloneqq \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} $$

2.4.3 Column Operations

To do column operations, the matrix multiples on the right. To do row operations, it multiples on the left.

$$ \begin{bmatrix} a & b \\ c & d \end{bmatrix} \begin{bmatrix} ? & ? \\ ? & ? \end{bmatrix} = \begin{bmatrix} b & a \\ d & c \end{bmatrix} \rarr \begin{bmatrix} a & b \\ c & d \end{bmatrix} \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} = \begin{bmatrix} b & a \\ d & c \end{bmatrix} $$

  • NO commutaitive law: AB != BA

2.4.4 Inverses Preview

  • Q: How do you get from U back to A?
  • A: The inverse matrix.

$$ \begin{bmatrix} ? & ? & ? \\ ? & ? & ? \\ ? & ? & ? \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 \\ -3 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \rarr \begin{bmatrix} 1 & 0 & 0 \\ 3 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 \\ -3 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} $$

$$ \rarr E^{-1}E=I $$

Back to top