These operations can be represented by elementary matrices, $E_{i}.$ There is one elementary matrix for each operation.

The operation is achieved by multiplying a given matrix on the left by $E_{i}.$

The steps in Gaussian elimination can then be written as a product sequence:

We can systematically write out row reduction on an augmented matrix:

We have just seen that the reduction process creates the tracking identity:

$$\big(I \mid \mathbf s\big) = \big(E_{m}\cdots E_{1}A \mid E_{m}\cdots E_{1}\mathbf q\big)$$

Which explicitly means that:

It follows directly that if we start with the identity matrix inside the augmented side:

Conclusion: We can use standard row reduction on the augmented matrix $\big(A \mid I\big)$ to safely find the inverse matrix $A^{-1}.$

Find the inverse of $A = \left(\begin{array}{cc} 2 & 1 \\ 1 & 1 \end{array}\right). $

Show that the inverse of $ A= \begin{pmatrix} 3&2&-1\\ 3&1&-1\\ 3&2&1 \end{pmatrix} $ is $ A^{-1}= \begin{pmatrix} -\frac12&\frac23&\frac16\\ 1&-1&0\\ -\frac12&0&\frac12 \end{pmatrix}. $

We can confirm our results very easily via Octave or Matlab, using the inv command:

So $\mathbf x=\mathbf 0$ is the only solution.

Theorem (Equivalent conditions for nonsingularity)

Statements: A ($A$ is invertible), B ($A\mathbf x=\mathbf 0 \Rightarrow \mathbf x=\mathbf 0$), C ($A$ is row-equivalent to $I$).

Proof Outline (Cont'd)

For $B \Rightarrow C$: Suppose $\mathbf x=\mathbf 0$ is the only solution to $A\mathbf x=\mathbf 0.$ Reduce the augmented matrix $(A \mid \mathbf 0)$ to $(U \mid \mathbf 0)$ where $U$ is in reduced row echelon form.

If $U$ contains a zero row, then there is a family of solutions. But $\mathbf x=\mathbf 0$ is the only solution, so $U$ has no zero rows.

Since $A$ is an $n \times n$ matrix, $U = I.$ So there exist elementary matrices $E_1, E_2, \dots, E_m$ such that $E_m \dots E_1 A = I.$ Hence $A$ is row-equivalent to $I.$

Theorem (Equivalent conditions for nonsingularity)

Statements: A ($A$ is invertible), B ($A\mathbf x=\mathbf 0 \Rightarrow \mathbf x=\mathbf 0$), C ($A$ is row-equivalent to $I$).

Proof Outline (Cont'd)

For $C \Rightarrow A$: If $A$ is row-equivalent to $I,$ then there exist elementary matrices $E_1, E_2, \dots, E_m$ such that $E_m \dots E_1 A = I.$

Therefore, $A^{-1} = E_m \dots E_1.$ So $A^{-1}$ exists, and hence $A$ is invertible.

A condition for a nonsingular matrix

We have just seen that:

$A \text{ is nonsingular} \iff A\mathbf x=\mathbf b \text{ has a unique solution}$

We would like a measurable property of the matrix itself that distinguishes singular from nonsingular matrices.

To achieve this, we introduce a quantity known as the determinant.

• We denote the determinant of a square matrix $A$ by $\det (A)$ or by $|A|.$
• $\det (A)$ is calculated from the matrix elements $a_{ij}.$
• It will turn out that $\det A \neq 0$ iff $A$ is nonsingular.

So how do we calculate $\det A$?

• But what if $a_{11} = 0$?
• Let's assume $a_{21} \neq 0$:

  $\begin{pmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{pmatrix} \rightarrow \begin{pmatrix} a_{21} & a_{22} \\ 0 & a_{12} \end{pmatrix}$

• We will be able to row reduce $A$ to $I$ as long as $a_{12} \neq 0$ and $a_{21} \neq 0$...
  which is equivalent to $a_{11}a_{22} - a_{12}a_{21} \neq 0$ when $a_{11} = 0.$
• Finally, we note that if $a_{11} = a_{21} = 0,$ then $A$ is not row-reducible to $I$...
  but $a_{11}a_{22} - a_{12}a_{21} = 0$ in this case!

$\left| \begin{array}{cc} a_{11}a_{22}-a_{21}a_{12} & a_{11}a_{23}-a_{21}a_{13} \\ a_{11}a_{32}-a_{31}a_{12} & a_{11}a_{33}-a_{31}a_{13} \end{array} \right|$

$= [a_{11}a_{22}-a_{21}a_{12}][a_{11}a_{33}-a_{31}a_{13}] - [a_{11}a_{32}-a_{31}a_{12}][a_{11}a_{23}-a_{21}a_{13}]$

Just as in the $2 \times 2$ case, the $3 \times 3$ definition for $\det A$ works in all the special cases where $a_{11}$, $a_{21}$ or $a_{31}$ are zero.

But remembering a formula like this is difficult... and it won't get any easier as our matrices get bigger.

For a $2\times2$ matrix $ A= \left( \begin{array}{cc} a_{11} & a_{12}\\ a_{21} & a_{22} \end{array} \right). $

Choose the first row.

For a $2\times2$ matrix $ A= \left( \begin{array}{cc} a_{11} & a_{12}\\ a_{21} & a_{22} \end{array} \right). $

Cross out the row and column containing each selected element.
The remaining $1\times1$ matrix has determinant equal to its single entry.

For a $2\times2$ matrix $ A= \left( \begin{array}{cc} a_{11} & a_{12}\\ a_{21} & a_{22} \end{array} \right). $

Apply the alternating sign pattern: $ \left( \begin{array}{cc} + & -\\ - & + \end{array} \right). $

For a $2\times2$ matrix $ A= \left( \begin{array}{cc} a_{11} & a_{12}\\ a_{21} & a_{22} \end{array} \right). $ The determinant is given by:

For a $3\times3$ matrix:

Formal Definition of Determinant

For any matrix $A$, we define the minor $M_{ij}$ to be the matrix comprising the elements remaining once the row and column containing $a_{ij}$ are removed.

Example: For a $4 \times 4$ matrix, removing row 2 and column 3 yields $M_{23}$:

The determinant of a square matrix $A \in \mathbb{R}^{n \times n}$ is defined inductively by:

where $C_{1i}$ is the cofactor of the corresponding element $a_{1i}$ in the first row.

We are not forced to sum the terms $a_{ij}C_{ij}$ along the top row only!

• $\det (A) = \sum_{i=1}^{n} a_{ki} C_{ki}$ along any row $k$
• $\det (A )= \sum_{i=1}^{n} a_{ik} C_{ik}$ down any column $k$

Example: We can expand down the first column to take advantage of the zeros:

Now, expanding down the last column of the remaining $3 \times 3$ matrix:

Solving a Linear System

Use Cramer's rule to solve the system:

The determinant of the coefficient matrix is $\det M= -4.$

Cramer's rule yields:

General Case Formulation

In general, given a system $M\mathbf x = \mathbf y$ where $\det M \neq 0$, the unique solution for each variable $x_i$ can be calculated by replacing the $i$-th column of $M$ with the vector $\mathbf y$: $$x_i = \frac{\det(M_i)}{\det(M)}$$

Where $M_i$ is constructed as:$$M_i = \begin{pmatrix} M_{11} & \dots & M_{1(i-1)} & \color{red}{y_1} & M_{1(i+1)} & \dots & M_{1n} \\ M_{21} & \dots & M_{2(i-1)} & \color{red}{y_2} & M_{2(i+1)} & \dots & M_{2n} \\ \vdots & & \vdots & \vdots & \vdots & & \vdots \\ M_{n1} & \dots & M_{n(i-1)} & \color{red}{y_n} & M_{n(i+1)} & \dots & M_{nn} \end{pmatrix}$$

• When $i=j$, this matches the standard definition of $\det A.$
• When $i \neq j$, this corresponds to evaluating the determinant of an altered matrix where rows $i$ and $j$ are identical, which must equal $0.$

If $\mathbf x$ is the unique solution to the linear system $A\mathbf x=\mathbf b$, we can isolate $\mathbf x$ using the adjoint matrix inverse:

$$\mathbf x = A^{-1}\mathbf b = \frac{1}{\det A}(\text{adj } A)\mathbf b$$

Evaluating for an individual component entry $x_i$ gives:

$\ds x_i = \frac{b_1 C_{1i} + b_2 C_{2i} + \dots + b_n C_{ni}}{\det A}$ $ =\ds \frac{\det A_i}{\det A}$

This holds because the expression $b_1 C_{1i} + b_2 C_{2i} + \dots + b_n C_{ni}$ acts exactly as the cofactor expansion down the $i$-th column of a modified matrix $A_i$, where the original $i$-th column elements of $A$ have been replaced by the vector elements of $\mathbf b$: $$A_i = \begin{pmatrix} a_{11} & \dots & b_1 & \dots & a_{1n} \\ a_{21} & \dots & b_2 & \dots & a_{2n} \\ \vdots & & \vdots & & \vdots \\ a_{n1} & \dots & b_n & \dots & a_{nn} \end{pmatrix}$$

It follows directly that the scalar triple product can be computed as: