Linear Algebra & Applications
2201NSC
Matrices
Part I
Matrix notation
A matrix is an $m\times n$ array of numbers
$ A = \left( \begin{array}{cccc} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{12n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end{array} \right) $
- The order of the subscripts is (row, column)
- We can also denote $A = (a_{ij})$
- For real matrices (matrices with real elements), we write $A\in \mathbb R^{m\times n}$
-
A matrix with $m = 1$ or $n = 1$ is usually called a vector
- we will denote column vectors as $\mathbf a,$ $\mathbf b,$ etc.
- and we will denote row vectors as $\overrightarrow{\mathbf a},$ $\overrightarrow{\mathbf b},$ etc.
Row and column vector notation
If
$A = \left( \begin{array}{ccc} 3 & 2 & 5 \\ -1 & 8 & 4 \\ \end{array} \right) $
then we denote
$\mathbf a_1 = \left( \begin{array}{c} 3 \\ -1 \\ \end{array} \right),\; $ $\mathbf a_2 = \left( \begin{array}{c} 2 \\ 8 \\ \end{array} \right),\; $ $\mathbf a_3 = \left( \begin{array}{c} 5 \\ 4 \\ \end{array} \right); $
and
$\overrightarrow{\mathbf a_1} = \left( \begin{array}{ccc} 3 & 2 & 5\\ \end{array} \right),\; $ $\overrightarrow{\mathbf a_2} = \left( \begin{array}{ccc} -1 & 8 & 4 \\ \end{array} \right). $
Matrix equality
Two matrices, $A$ and $B,$ are equal iff $a_{ij}= b_{ij}$ for each $i$ and $j.$
Examples:
$A = \left( \begin{array}{ccc} 3 & 2 & 5 \\ -1 & 8 & 4 \\ \end{array} \right),\quad B = \left( \begin{array}{ccc} 3 & 2 & 5 \\ -1 & 8 & 4 \\ \end{array} \right). $
For all $i$ and $j$, $a_{ij}= b_{ij}.$ Therefore $A=B.$
$C = \left( \begin{array}{ccc} 3 & 2 & 5 \\ -1 & 8 & 4 \\ \end{array} \right),\quad D = \left( \begin{array}{ccc} 3 & 2 & 0 \\ -1 & 8 & 4 \\ \end{array} \right). $
Here $c_{13} \neq d_{13}.$ Therefore $C\neq D.$
Matrix multiplication by a scalar
If $A\in \mathbb R^{m\times n}$ and $\alpha\in \mathbb R$ (i.e. a scalar), then
$\alpha A$ $ = \left( \begin{array}{cccc} \alpha a_{11} & \alpha a_{12} & \cdots & \alpha a_{1n} \\ \alpha a_{21} & \alpha a_{22} & \cdots & \alpha a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ \alpha a_{m1} & \alpha a_{m2} & \cdots & \alpha a_{mn} \\ \end{array} \right). $
If $\; A = \left( \begin{array}{ccc} 3 & 2 & 5 \\ -1 & 8 & 4 \\ \end{array} \right),\; $ then $ \;2A= \left( \begin{array}{ccc} 6 & 4 & 10 \\ -2 & 16 & 8 \\ \end{array} \right). $
If $\;B = \left( \begin{array}{ccc} 3 & 3 \\ -6 & 9 \\ 0 & 3 \\ \end{array} \right),\; $ then $ \;\dfrac{1}{3}B = \left( \begin{array}{ccc} 1 & 1 \\ -2 & 3 \\ 0 & 1 \\ \end{array} \right).$
Matrix addition
If $A\in \mathbb R^{m\times n}$ and $B\in \mathbb R^{m\times n}$, then
$A+B$ $ = \left( \begin{array}{cccc} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end{array} \right)+ \left( \begin{array}{cccc} b_{11} & b_{12} & \cdots & b_{1n} \\ b_{21} & b_{22} & \cdots & b_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ b_{m1} & b_{m2} & \cdots & b_{mn} \\ \end{array} \right) $
$ = \left( \begin{array}{cccc} a_{11}+ b_{11} & a_{12} + b_{12}& \cdots & a_{1n} + b_{1n}\\ a_{21} + b_{21}& a_{22} + b_{22}& \cdots & a_{2n} + b_{2n}\\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} + b_{m1}& a_{m2}+ b_{m2} & \cdots & a_{mn} + b_{mn}\\ \end{array} \right). $
Matrix addition
If $A\in \mathbb R^{m\times n}$ and $B\in \mathbb R^{m\times n}$, then
$A+B$ $ = \left( \begin{array}{cccc} a_{11}+ b_{11} & a_{12} + b_{12}& \cdots & a_{1n} + b_{1n}\\ a_{21} + b_{21}& a_{22} + b_{22}& \cdots & a_{2n} + b_{2n}\\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} + b_{m1}& a_{m2}+ b_{m2} & \cdots & a_{mn} + b_{mn}\\ \end{array} \right). $
If $\; A = \left( \begin{array}{ccc} 3 & 2 & 5 \\ -1 & 8 & 4 \\ \end{array} \right)\; $ and $ \;B= \left( \begin{array}{ccc} 2 & 2 & 1 \\ 2 & 0 & -1 \\ \end{array} \right), $
then $\;A+B = \left( \begin{array}{ccc} 5 & 4 & 6 \\ 1 & 8 & 3 \\ \end{array} \right),\; $ and $ \;A-B = \left( \begin{array}{ccc} 1 & 0 & 4 \\ -3 & 8 & 5\\ \end{array} \right).$
Matrix multiplication
For matrices $A\in \mathbb R^{m\times n}$ and $B\in \mathbb R^{n\times p},$ we define th product
$C = A B$ $ = \left( \begin{array}{cccc} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end{array} \right) \left( \begin{array}{cccc} b_{11} & b_{12} & \cdots & b_{1p} \\ b_{21} & b_{22} & \cdots & b_{2p} \\ \vdots & \vdots & \ddots & \vdots \\ b_{n1} & b_{n2} & \cdots & b_{np} \\ \end{array} \right) $
$\qquad = \left( \begin{array}{cccc} \sum_{i}a_{1i} b_{i1} & \sum_{i}a_{1i} b_{i2}& \cdots & \sum_{i}a_{1i} b_{ip}\\ \sum_{i}a_{2i} b_{i1}& \sum_{i}a_{2i}b_{i2}& \cdots & \sum_{i}a_{2i} b_{ip}\\ \vdots & \vdots & \ddots & \vdots \\ \sum_{i}a_{mi} b_{i1}& \sum_{i}a_{mi} b_{i2} & \cdots & \sum_{i}a_{mi} b_{ip}\\ \end{array} \right). $
Matrix multiplication
$C = A B = \left( \begin{array}{cccc} \sum_{i}a_{1i} b_{i1} & \sum_{i}a_{1i} b_{i2}& \cdots & \sum_{i}a_{1i} b_{ip}\\ \sum_{i}a_{2i} b_{i1}& \sum_{i}a_{2i}b_{i2}& \cdots & \sum_{i}a_{2i} b_{ip}\\ \vdots & \vdots & \ddots & \vdots \\ \sum_{i}a_{mi} b_{i1}& \sum_{i}a_{mi} b_{i2} & \cdots & \sum_{i}a_{mi} b_{ip}\\ \end{array} \right). $
-
Each element of C is the sum of products
between a row of $A$ and a column of $B$
$c_{22} = \ds \sum_{i} a_{2i}b_{i2}$ $=a_{21}b_{12} +a_{22}b_{22} +\cdots +a_{2n}b_{n2} $
-
For this to work, we require that
the number of columns of $A=$ the number of rows of $B.$
-
If $A\in \mathbb R^{m\times n}$ and $B\in \mathbb R^{n\times p},$
then the result of $AB$ is a matrix with $m$ rows and $p$
columns.
Example: the matrix $C$, above, is an $m \times p$ matrix ($C\in \mathbb R^{m\times p}$).
Matrix multiplication examples
$A = \left( \begin{array}{cc} 3 & -2 \\ 2 & 4 \\ 1 & -3 \\ \end{array} \right),\quad B= \left( \begin{array}{ccc} -2 & 1 & 3 \\ 4 & 1 & 6 \\ \end{array} \right). $
$A B $ $ = \left( \begin{array}{cc} 3 & -2 \\ 2 & 4 \\ 1 & -3 \\ \end{array} \right) \left( \begin{array}{ccc} -2 & 1 & 3 \\ 4 & 1 & 6 \\ \end{array} \right)$ $ = \left( \begin{array}{rrr} -14 & 1 & -3 \\ 12 & 6 & 30 \\ -14 & -2 & -15 \\ \end{array} \right) \quad\quad $
| 1st row | $3\times (-2) + (-2)\times 4\quad$ | $3\times 1 + (-2)\times 1\quad$ | $3\times 3 + (-2)\times 6$ |
| 2nd row | $2\times (-2) + 4\times 4$ | $2\times 1 + 4\times 1$ | $2\times 3 + 4\times 6$ |
| 3rd row | $1\times (-2) + (-3)\times 4$ | $1\times 1 + (-3)\times 1$ | $1\times 3 + (-3)\times 6$ |
| 1st column | 2nd column | 3rd column |
Matrix multiplication examples
Matrix multiplication examples
$A = \left( \begin{array}{cc} 3 & -2 \\ 2 & 4 \\ 1 & -3 \\ \end{array} \right),\quad B= \left( \begin{array}{ccc} -2 & 1 & 3 \\ 4 & 1 & 6 \\ \end{array} \right),\quad AB = \left( \begin{array}{rrr} -14 & 1 & -3 \\ 12 & 6 & 30 \\ -14 & -2 & -15 \\ \end{array} \right). $
$ BA $ $ = \left( \begin{array}{ccc} -2 & 1 & 3 \\ 4 & 1 & 6 \\ \end{array} \right) \left( \begin{array}{cc} 3 & -2 \\ 2 & 4 \\ 1 & -3 \\ \end{array} \right) $ $ = \left( \begin{array}{rr} -1 & -1 \\ 20 & -22 \\ \end{array} \right) \quad\quad $
| 1st row | $(-2)\times 3 + 1\times 2 + 3 \times 1\quad$ | $(-2)\times (-2) + 1\times 4 + 3 \times (-3) $ |
| 2nd row | $4\times 3 + 1\times 2 + 6 \times 1$ | $4\times (-2) + 1\times 4 + 6 \times (-3) $ |
| 1st column | 2nd column |
👉 Note that here $AB\neq BA$!!!
Matrix multiplication examples
Matrix multiplication examples
$A = \left( \begin{array}{cc} 1 & 2 \\ 4 & 5 \\ 3 & 6 \\ \end{array} \right),\quad B= \left( \begin{array}{ccc} 3 & 4 \\ 1 & 2 \\ \end{array} \right) $
$ AB $ $ = \left( \begin{array}{cc} 1 & 2 \\ 4 & 5 \\ 3 & 6 \\ \end{array} \right) \left( \begin{array}{ccc} 3 & 4 \\ 1 & 2 \\ \end{array} \right) $ $ = \left( \begin{array}{rr} 5 & 8 \\ 17 & 26 \\ 15 & 24 \\ \end{array} \right) \quad $
| 1st row | $1\times 3 + 2\times 1 \quad$ | $1\times 4 + 2\times 2$ |
| 2nd row | $4\times 3 + 5\times 1 $ | $4\times 4 + 5\times 2 $ |
| 3rd row | $3\times 3 + 6\times 1 $ | $3\times 4 + 6\times 2 $ |
| 1st column | 2nd column |
👉 In this case, $BA $ is not even defined!!!
Matrix multiplication examples
Motivation for Matrix Multiplication
Consider the following system of linear equations:
$u = 2x\quad$
$v = x + y$
With matrix multiplication defined, we can write this system as:
$\left( \begin{array}{c} u \\ v \end{array} \right) = \left( \begin{array}{cc} 2 & 0 \\ 1 & 1 \end{array} \right) \left( \begin{array}{c} x \\ y \end{array} \right)$
- We can represent linear systems as matrix equations.
- This makes them much easier to work with, especially when they get complicated.
- But what are the rules when working with matrices? Do the rules of regular algebra apply?
Important Rules of Algebra
Whenever we have a set of mathematical objects with addition and multiplication operations, we check standard algebraic properties:
- Commutativity: $a + b = b + a$
- Associativity: $(a + b) + c = a + (b + c) = a + b + c$
- Distributivity: $a(b + c) = ab + ac$
- Identity: Existence of a structure $i$ such that $ia = ai = a$
- Inverse: For each structure $a$, there exists $a^{-1}$ such that
$aa^{-1} = a^{-1}a = i.$
If such structure exists, we call it the "inverse of $a$".
Do these rules apply to Matrices?
With the exception of an inverse for zero, these properties all hold for numbers.
Tip: That's why numbers do such a good job of quantifying our daily lives!
But do these properties hold for structures other than numbers?
How about... matrices?
Matrix Algebra: Most Rules Hold...
| 1. $A+B = B+A$
(Addition Commutativity) |
6. $(\alpha\beta)A = \alpha(\beta A)$ |
| 2. $(A+B)+C =
A+(B+C)$ (Addition Associativity) |
7. $\alpha(AB) = (\alpha A)B = A(\alpha B)$ |
| 3. $(AB)C = A(BC)$
(Multiplication Associativity) |
8. $(\alpha+\beta)B = \alpha B + \beta B$ |
| 4. $A(B+C) = AB + AC$
(Left-distributivity) |
9. $\alpha(A+B) = \alpha A + \alpha B$ |
| 5. $(A+B)C = AC + BC$
(Right-distributivity) |
That's because it doesn't hold for all matrices.
Identity Matrix
The multiplicative identity for numbers, $1$, leaves a number unchanged: $1n = n1 = n.$
For matrix multiplication, only square matrices have identities.
The $n \times n$ identity matrix: $I_{n} = (\delta_{ij}) = \begin{cases} 1 & \text{if } i=j \\ 0 & \text{if } i \neq j \end{cases}$
$ I_{2} = \left( \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right), \quad $ $ I_{3} = \left( \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right),\quad$ etc.
Example:
$ \left( \begin{array}{cc} 2 & 3 \\ 4 & 5 \end{array} \right) \left( \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right) $ $ = \left( \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right) \left( \begin{array}{cc} 2 & 3 \\ 4 & 5 \end{array} \right) $ $= \left( \begin{array}{cc} 2 & 3 \\ 4 & 5 \end{array} \right)$
Inverse Matrix
The multiplicative inverse $A^{-1}$ of matrix $A$ is a matrix such that:
$A^{-1}A$ $ =AA^{-1} $ $= I$
Important Notes:
- Only square matrices have inverses, but not all square matrices do. $\qquad $
- A square matrix is invertible (or nonsingular) iff it has an inverse.
- We can use Gaussian Elimination to find $A^{-1}.$
Inverse Matrix Properties
-
Inverses are unique: If $B$ and $C$ are inverses of $A$,
then:
$B = BI $ $= B(AC) $ $= (BA)C $ $= IC = C$
- Double Inverse: $\left(A^{-1}\right)^{-1} = A$
-
Product Inverse: For any invertible matrices $A_1, A_2, \dots, A_k$:
$\left(A_1 A_2 \dots A_k\right)^{-1} = A_k^{-1} \dots A_2^{-1} A_1^{-1}$
Property (3) is particularly useful for deriving results and solving complex matrix equations.
Matrix Transpose
Let $A = (a_{ij})$ be a matrix. Then the transpose of $A$, denoted $A^T$, is given by $$A^T =(a_{ji}).$$
The element in row $i$, column $j$ of $A$ moves to row $j$, column $i$ of $A^T.$
Example:
$A = \left( \begin{array}{cc} 3 & -2 \\ 2 & 4 \\ 1 & -3 \end{array} \right)$ $\implies A^T = \left( \begin{array}{ccc} 3 & 2 & 1 \\ -2 & 4 & -3 \end{array} \right)$
$B = \left( \begin{array}{ccc} 3 & -2 & 1 \\ 1 & 4 & 5 \\ -2 & 1 & -3 \end{array} \right) $ $\implies B^T = \left( \begin{array}{ccc} 3 & 1 & -2 \\ -2 & 4 & 1 \\ 1 & 5 & -3 \end{array} \right)$
Matrix Transpose Properties
- $\left(A^T\right)^T = A$
- $(\alpha A)^T = \alpha A^T$ (where $\alpha$ is any scalar number)
- $(A + B)^T = A^T + B^T$
- $(A_1 A_2 \dots A_k)^T = A_k^T \dots A_2^T A_1^T$
Symmetric Matrix Definition:
Finally, a matrix $A$ is symmetric iff $A = A^T$
Application: Marital Population
Imagine the following probabilities within a population setup over a given year:
- The probability a married person divorces is 30%.
- The probability an unmarried person marries is 20%.
If 80% of the population is currently married, what percentage is married after:
- 1 year?
- 2 years?
- 10 years?
Translating into a Mathematical Problem
Let's describe the population state as a vector:
$\left( \begin{array}{c} \text{% married} \\ \text{% unmarried} \end{array} \right)$
The proportions of married vs. unmarried people in the next year are given by
$\text{% married next year} = [\% \text{ married}] \times [\% \text{ don't divorce}] + [\% \text{ unmarried}] \times [\% \text{ marry}]$
$\text{% unmarried next year} = [\% \text{ married}] \times [\% \text{ divorce}] + [\% \text{ unmarried}] \times [\% \text{ don't marry}]$
Given a 30% divorce rate and 20% marriage rate:
$\left( \begin{array}{c} \text{% married next yr} \\ \text{% unmarried next yr} \end{array} \right) = \left( \begin{array}{c} [\% \text{ married}] \times 0.7 + [\% \text{ unmarried}] \times 0.2 \\ [\% \text{ married}] \times 0.3 + [\% \text{ unmarried}] \times 0.8 \end{array} \right)\qquad\quad$
$\qquad = \left( \begin{array}{cc} 0.7 & 0.2 \\ 0.3 & 0.8 \end{array} \right) \left( \begin{array}{c} \text{% married} \\ \text{% unmarried} \end{array} \right)$
Solving the Population Problem
Initially, 80% of the population is married:
$\mathbf x_0 = \left( \begin{array}{c} 80 \\ 20 \end{array} \right)$
After 1 year ($\mathbf x_1$):
$\mathbf x_1 = \left( \begin{array}{cc} 0.7 & 0.2 \\ 0.3 & 0.8 \end{array} \right) \left( \begin{array}{c} 80 \\ 20 \end{array} \right) $ $= \left( \begin{array}{c} 56 + 4 \\ 24 + 16 \end{array} \right) $ $= \left( \begin{array}{c} 60 \\ 40 \end{array} \right)$
After 2 years ($\mathbf x_2$): 👉 $ A^2\mathbf x_0 = A\left(A\mathbf x_0\right)= A\mathbf x_1$
$\mathbf x_2 = \left( \begin{array}{cc} 0.7 & 0.2 \\ 0.3 & 0.8 \end{array} \right)^2 \left( \begin{array}{c} 80 \\ 20 \end{array} \right)$ $ =\left( \begin{array}{cc} 0.7 & 0.2 \\ 0.3 & 0.8 \end{array} \right) \left( \begin{array}{c} 60 \\ 40 \end{array} \right)$ $= \left( \begin{array}{c} 42 + 8 \\ 18 + 32 \end{array} \right) $ $ = \left( \begin{array}{c} 50 \\ 50 \end{array} \right)$
After 10 years ($\mathbf x_{10}$):
$\mathbf x_{10} = \left( \begin{array}{cc} 0.7 & 0.2 \\ 0.3 & 0.8 \end{array} \right)^{10} \left( \begin{array}{c} 80 \\ 20 \end{array} \right)$ $ \approx \left( \begin{array}{c} 40 \\ 60 \end{array} \right)$
Application: Turtle population model
Loggerhead Sea Turtles have the following demographic stages:
| Stage # |
Description (age in years) |
Annual survivorship |
Transition to next stage |
Eggs laid per year |
|---|---|---|---|---|
| 1 | Hatchlings ($\lt1$) | 0.67 | 100% become juveniles | 0 |
| 2 | Juveniles (1–21) | 0.7394 | 0.0006 become novice adults | 0 |
| 3 | Novice adults (22) | 0.81 | 100% become mature adults | 127 |
| 4 | Mature adults (23+) | 0.81 | Remain mature adults | 79 |
From one year to the next, these populations must change through turtles dying, growing older and thus changing demographic group, or through eggs hatching to produce hatchlings.
Application: Turtle population model
| Stage # |
Description (age in years) |
Annual survivorship |
Transition to next stage |
Eggs laid per year |
|---|---|---|---|---|
| 1 | Hatchlings ($\lt 1$) | 0.67 | 100% become juveniles | 0 |
| 2 | Juveniles (1–21) | 0.7394 | 0.0006 become novice adults | 0 |
| 3 | Novice adults (22) | 0.81 | 100% become mature adults | 127 |
| 4 | Mature adults (23+) | 0.81 | Remain mature adults | 79 |
We can represent these changes using a Leslie matrix, which relates the population across the demographic groups to the populations at the previous year. That is:
$ \begin{pmatrix} \text{# hatchlings}_{n+1} \\ \text{# juveniles}_{n+1} \\ \text{# novice adults}_{n+1} \\ \text{# mature adults}_{n+1} \end{pmatrix} $ $ = \begin{pmatrix} 0 & 0 & 127 & 79 \\ 0.67 & 0.7394 & 0 & 0 \\ 0 & 0.0006 & 0 & 0 \\ 0 & 0 & 0.81 & 0.81 \end{pmatrix} $ $ \begin{pmatrix} \text{# hatchlings}_n \\ \text{# juveniles}_n \\ \text{# novice adults}_n \\ \text{# mature adults}_n \end{pmatrix} $
Application: Turtle population model
| Stage # |
Description (age in years) |
Annual survivorship |
Transition to next stage |
Eggs laid per year |
|---|---|---|---|---|
| 1 | Hatchlings ($\lt 1$) | 0.67 | 100% become juveniles | 0 |
| 2 | Juveniles (1–21) | 0.7394 | 0.0006 become novice adults | 0 |
| 3 | Novice adults (22) | 0.81 | 100% become mature adults | 127 |
| 4 | Mature adults (23+) | 0.81 | Remain mature adults | 79 |
We thus define the Leslie matrix $L$, which allows us to calculate the populations after $n$ years, $\xx_n$, from the initial state $\xx_0$.
$ L= \begin{pmatrix} 0 & 0 & 127 & 79 \\ 0.67 & 0.7394 & 0 & 0 \\ 0 & 0.0006 & 0 & 0 \\ 0 & 0 & 0.81 & 0.81 \end{pmatrix}.\;\; $ $\,\text{That is}\;\; \Large \xx_n = L^n \xx_0 $
Application: Turtle population model
Given an initial population, the Leslie population model then makes the following predictions
| Stage # | Initial population | 10 years | 25 years | 50 years |
|---|---|---|---|---|
| 1 | 200,000 | 114,264 | 74,039 | 35,966 |
| 2 | 300,000 | 329,212 | 213,669 | 103,795 |
| 3 | 500 | 214 | 139 | 68 |
| 4 | 1,500 | 1,061 | 687 | 334 |
That is, the populations decrease over time.
Application: Turtle population model
Suppose we know the populations this year:
| Stage # | Last year ($\mathbf x_0$) | This year |
|---|---|---|
| 1 | ? | 182,000 |
| 2 | ? | 355,810 |
| 3 | ? | 190 |
| 4 | ? | 1,620 |
Can we determine the population last year? We know that $L\mathbf x_0 = \mathbf x_1.$
If $L$ has an inverse, $L^{-1}$, then
$L\mathbf x_0=\mathbf x_1$ $\Ra L^{-1}L \mathbf x_0 = L^{-1} \mathbf x_1$ $\Ra I \mathbf x_0 = L^{-1} \mathbf x_1$ $\Ra \mathbf x_0 = L^{-1} \mathbf x_1$
Application: Turtle population model
After finding $L^{-1}$, we can calculate the populations last year:
| Stage # | Last year ($\mathbf x_0$) | This year |
|---|---|---|
| 1 | 199,259 | 182,000 |
| 2 | 300,671 | 355,810 |
| 3 | 499 | 190 |
| 4 | 1,501 | 1,620 |
But... how did we find $L^{-1}$?? And what if $L$ didn't have an inverse??
We'll soon see a way to determine whether an inverse exists. But first we'll need some more important results from matrix algebra...
Sub-matrices
Let $A$ be an $m \times n$ matrix. Then a matrix $B$ is called a sub-matrix of $A$ iff $B$ is formed by removing up to $m-1$ rows and up to $n-1$ columns of $A.$
Example:
$ A = \left( \begin{array}{cccc} 1 & 2 & 3 & 4 \\ 5 & 6 & 7 & 8 \\ 9 & 10 & 11 & 12 \end{array} \right) \qquad B = \left( \begin{array}{cc} 1 & 3 \\ 5 & 7 \end{array} \right) $
$B$ was formed by removing the third row, and the second and fourth columns of $A.$ So $B$ is a sub-matrix of $A.$
Matrices: block form and block matrices
With the idea of sub-matrices, we can write matrices in "block form".
For example:
Let $U = \left( \begin{array}{ccc} 1 & 2 & 3 \end{array} \right), \;\; V = \left( \begin{array}{ccc} 4 & 5 & 6 \end{array} \right), \;\; \text{and}\;\;$ $W = \left( \begin{array}{ccc} 1 & 2 & 3 \\ 4 & 5 & 6 \end{array} \right)$
Then $W$ can be written as $W = \left( \begin{array}{c} U \\ V \end{array} \right)$ which is in block form.
A matrix that is in block form is called a "block matrix".
Matrix multiplication: reminder
Recall the definition of matrix multiplication:
$ AB = \left( \begin{array}{cccc} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end{array} \right) \left( \begin{array}{cccc} b_{11} & b_{12} & \cdots & b_{1p} \\ b_{21} & b_{22} & \cdots & b_{2p} \\ \vdots & \vdots & \ddots & \vdots \\ b_{n1} & b_{n2} & \cdots & b_{np} \\ \end{array} \right) $
$ = \left( \begin{array}{cccc} \sum_{i}a_{1i} b_{i1} & \sum_{i}a_{1i} b_{i2}& \cdots & \sum_{i}a_{1i} b_{ip}\\ \sum_{i}a_{2i} b_{i1}& \sum_{i}a_{2i}b_{i2}& \cdots & \sum_{i}a_{2i} b_{ip}\\ \vdots & \vdots & \ddots & \vdots \\ \sum_{i}a_{mi} b_{i1}& \sum_{i}a_{mi} b_{i2} & \cdots & \sum_{i}a_{mi} b_{ip}\\ \end{array} \right) $
Block matrix multiplication
This definition can also be used when multiplying block matrices.
For example: if we write an $m \times n$ matrix $A$ and an $n \times p$ matrix $B$ in terms of blocks $A_{ij}$ and $B_{ij}$ (respectively), we then have:
$ AB = \left( \begin{array}{cccc} A_{11} & A_{12} & \cdots & A_{1r} \\ A_{21} & A_{22} & \cdots & A_{2r} \\ \vdots & \vdots & \ddots & \vdots \\ A_{s1} & A_{s2} & \cdots & A_{sr} \end{array} \right) \left( \begin{array}{cccc} B_{11} & B_{12} & \cdots & B_{1p} \\ B_{21} & B_{22} & \cdots & B_{2p} \\ \vdots & \vdots & \ddots & \vdots \\ B_{r1} & B_{r2} & \cdots & B_{rp} \end{array} \right) $
$ = \left( \begin{array}{cccc} \sum_{k} A_{1k}B_{k1} & \sum_{k} A_{1k}B_{k2} & \cdots & \sum_{k} A_{1k}B_{kp} \\ \sum_{k} A_{2k}B_{k1} & \sum_{k} A_{2k}B_{k2} & \cdots & \sum_{k} A_{2k}B_{kp} \\ \vdots & \vdots & \ddots & \vdots \\ \sum_{k} A_{sk}B_{k1} & \sum_{k} A_{sk}B_{k2} & \cdots & \sum_{k} A_{sk}B_{kp} \end{array} \right) $
as long as the block products are defined!
Block multiplication cont'd
This form of matrix multiplication is called "block multiplication".
It's a very important form of multiplication!!
It can greatly simplify the calculation of matrix products, and has some very, very important results that get used extensively in Linear Algebra.
First, let's see an example of block multiplication.
Block multiplication: example
The following example shows how block multiplication can greatly simplify the calculation of matrix products. Suppose
$\;A= \left( \begin{array}{cccc} 1 & 0 & 0 & 2 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 3 \\ 0 & 0 & 0 & 1 \\ \end{array} \right) $ $ = \left( \begin{array}{ccc|c} & & & \\ & A_{11}=I & & A_{12} \\ & & & \\\hline & A_{21} & & A_{22} \\ \end{array} \right) $ and $\;B= \left( \begin{array}{cc} -1 & 2 \\ -3 & 4 \\ 1 & -2 \\ 3 & -4 \\ \end{array} \right) $ $ = \left( \begin{array}{cc} \\ B_{11} \\ \\\hline B_{21} \\ \end{array} \right) $
then
$\;AB= \left( \begin{array}{c} \\ I B_{11}+ A_{12}B_{21} \\ \\ \hline A_{21} B_{11}+ I B_{21} \\ \end{array} \right)\;$ $= \left( \begin{array}{c} \\ \color{red}{B_{11}} + \color{blue}{A_{12}B_{21}}\\ \\ \hline B_{21} \\ \end{array} \right)\;$ $= \left( \begin{array}{cc} \color{red}{-1}+\color{blue}{6} & \color{red}{2}+\color{blue}{(-8)}\\ \color{red}{-3}+\color{blue}{0} & \color{red}{4}+\color{blue}{0}\\ \color{red}{1}+\color{blue}{9} & \color{red}{(-2)}+\color{blue}{(-12)}\\ \hline 3 & -4 \\ \end{array} \right)\;$ $= \left( \begin{array}{cc} 5 & -6\\ -3 & 4\\ 10 & -14\\ 3 & -4 \\ \end{array} \right)\;$
Blocks were then defined so that block multiplication could use those sub-matrices to significantly reduce the amount of working required for the calculation of the product.
Important results of block matrix multiplication (I)
We now look at some important results of block multiplication.
Firstly, consider the product $A\mathbf b$ where:
- $A$ is divided into blocks made from its columns, and
- $\mathbf b$ is divided into blocks of its individual elements.
It then follows that
$A\mathbf b = \begin{pmatrix} | & | & & | \\ \mathbf a_1 & \mathbf a_2 & \cdots & \mathbf a_n \\ | & | & & | \end{pmatrix} \begin{pmatrix} b_1 \\ b_2 \\ \vdots \\ b_n \end{pmatrix} = b_1 \mathbf a_1 + b_2 \mathbf a_2 + \cdots + b_n \mathbf a_n$
This first result is a very, very important one!!
Why?? 🤔 Because it's an example of a linear combination...
Linear combinations
We define a linear combination of vectors, matrices, functions, etc. $A_1, A_2, \cdots, A_n$ as any sum:
$$\sum_{i=1}^{n}x_{i}A_{i} = x_{1}A_{1} + x_{2}A_{2} + \cdots + x_{n}A_{n}$$ for real numbers $x_{1}, x_{2}, \dots, x_{n}.$
On the previous slide, we saw that:
$$A\mathbf x = \begin{pmatrix} | & | & & | \\ \mathbf a_{1} & \mathbf a_{2} & \cdots & \mathbf a_{n} \\ | & | & & | \end{pmatrix} \begin{pmatrix} x_{1} \\ x_{2} \\ \vdots \\ x_{n} \end{pmatrix} = x_{1}\mathbf a_{1} + x_{2}\mathbf a_{2} + \cdots + x_{n}\mathbf a_{n}$$
Linear combinations cont'd
So by the definition of linear combinations, we have the following further results:
- The product $A\mathbf x$ is a linear combination of the columns of $A$;
- The linear system $A\mathbf x = \mathbf b$ has a solution iff $\mathbf b$ is a linear combination of the columns of $A$.
These two results are very, very important!!
We'll use them extensively later in the course. For now though, a few more important results of block multiplication before we move on...
Important results of block matrix multiplication (II)
We finish this section with the following three important results of block multiplication.
For matrices $A \in \mathbb{R}^{m \times n}$ and $B \in \mathbb{R}^{n \times p}$:
1. Row-Column Block Expansion:
$AB = \begin{pmatrix} - & \overrightarrow{\mathbf a_{1}} & - \\ - & \overrightarrow{\mathbf a_{2}} & - \\ & \vdots & \\ - & \overrightarrow{\mathbf a_{m}}& - \end{pmatrix} \begin{pmatrix} | & | & & | \\ \mathbf b_{1} & \mathbf b_{2}& \cdots & \mathbf b_{p} \\ | & | & & | \end{pmatrix} $ $ = \begin{pmatrix} \overrightarrow{\mathbf a_{1}}\,\mathbf b_{1} & \overrightarrow{\mathbf a_{1}}\,\mathbf b_{2} & \cdots & \overrightarrow{\mathbf a_{1}}\,\mathbf b_{p} \\ \overrightarrow{\mathbf a_{2}}\,\mathbf b_{1} & \overrightarrow{\mathbf a_{2}}\,\mathbf b_{2} & \cdots & \overrightarrow{\mathbf a_{2}}\,\mathbf b_{p} \\ \vdots & \vdots & \ddots & \vdots \\ \overrightarrow{\mathbf a_{m}}\,\mathbf b_{1} & \overrightarrow{\mathbf a_{m}}\,\mathbf b_{2} & \cdots & \overrightarrow{\mathbf a_{m}}\,\mathbf b_{p} \end{pmatrix} $
Important results of block matrix multiplication (II) cont'd
2. Matrix-Column Block Multiplication:
$AB = A \begin{pmatrix} | & | & & | \\ \mathbf b_{1} & \mathbf b_{2} & \cdots & \mathbf b_{p} \\ | & | & & | \end{pmatrix} $ $ = \begin{pmatrix} | & | & & | \\ A\mathbf b_{1} & A\mathbf b_{2} & \cdots & A\mathbf b_{p} \\ | & | & & | \end{pmatrix} $
3. Column-Row (Outer Product) Expansion:
$AB = \begin{pmatrix} | & | & & | \\ \mathbf a_{1} & \mathbf a_{2} & \cdots & \mathbf a_{n} \\ | & | & & | \end{pmatrix} \begin{pmatrix} - & \overrightarrow{\mathbf b_{1}} & - \\ - & \overrightarrow{\mathbf b_{2}} & - \\ & \vdots & \\ - & \overrightarrow{\mathbf b_{n}} & - \end{pmatrix} $ $ = \mathbf a_{1} \overrightarrow{\mathbf b_{1}} + \mathbf a_{2} \overrightarrow{\mathbf b_{2}} + \cdots + \mathbf a_{n} \overrightarrow{\mathbf b_{n}} $