2201NSC - Week 7

Linear Algebra & Applications

2201NSC

Week 7 - Inner Product Spaces

Problem 1

Find the angle between the vectors $\mathbf{v}$ and $\mathbf{w}$ using the standard inner product $\langle \mathbf{x}, \mathbf{y} \rangle = \mathbf{x}^T \mathbf{y}$.

$\mathbf{v}=(2,-3)^T$ and $\mathbf{w}=(3,2)^T$
$\mathbf{v}=(4,1)^T$ and $\mathbf{w}=(3,2)^T$
$\mathbf{v}=(-2,3,1)^T$ and $\mathbf{w}=(1,2,4)^T$
$\mathbf{v}=(1,3,-1,4)^T$ and $\mathbf{w}=(-1,1,-2,1)^T$
$\mathbf{v}=(2,1,3)^T$ and $\mathbf{w}=(6,3,9)^T$

Problem 1

$\mathbf{v}=(2,-3)^T$ and $\mathbf{w}=(3,2)^T$

Problem 2

Inner product given for $\mathbf{v}=(v_1,v_2)^T,$ $\mathbf{w}=(w_1,w_2)^T \in \mathbb{R}^2$ is

$ \langle \mathbf{v}, \mathbf{w} \rangle = 2v_1 w_1 + v_1 w_2 + v_2 w_1 + 2v_2 w_2. $

Find the inner product of the given vectors using (1) the standard inner product and (2) the inner product given above.

$\mathbf{v}=(1,0)^T$ and $\mathbf{w}=(-1,2)^T$
$\mathbf{v}=(2,-1)^T$ and $\mathbf{w}=(3,6)^T$
$\mathbf{v}=(1,-2)^T$ and $\mathbf{w}=(2,1)^T$
$\mathbf{v}=(0,1)^T$ and $\mathbf{w}=(-2,1)^T$
Show that $\langle \mathbf{v}, \mathbf{w} \rangle= \langle \mathbf{w}, \mathbf{v} \rangle$ for the inner product given above.

Problem 2 (cont.)

Inner product given for $\mathbf{v}=(v_1,v_2)^T,$ $\mathbf{w}=(w_1,w_2)^T \in \mathbb{R}^2$ is

$ \langle \mathbf{v}, \mathbf{w} \rangle = 2v_1 w_1 + v_1 w_2 + v_2 w_1 + 2v_2 w_2. $

Find the inner product of the given vectors using (1) the standard inner product and (2) the inner product given above.

$\mathbf{v}=(1,0)^T$ and $\mathbf{w}=(-1,2)^T$

Problem 3

Verify that each of the sets below is an orthogonal set (using the standard inner product) and check if the set is an orthonormal set. If it is not, adjust it so that it is.

$S=\left\{(1,1)^T, (-1,1)^T\right\}$
$S=\left\{\left(\frac{1}{\sqrt{3}},\frac{1}{\sqrt{3}},\frac{1}{\sqrt{3}}\right)^T, \left(-\frac{2}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}}\right)^T, \left(0,-\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}}\right)^T\right\}$
$S=\left\{(\cos{\theta},\sin{\theta})^T, (-\sin{\theta},\cos{\theta})^T\right\}$
$S=\left\{(1,1,0)^T, (1,-1,2)^T, (-1,1,1)^T\right\}$
Let $\mathbf{v}_1=\left(\frac{1}{\sqrt{3}},\frac{1}{\sqrt{3}},\frac{1}{\sqrt{3}}\right)^T$ and $\mathbf{v}_2=\left(-\frac{1}{\sqrt{2}},0,\frac{1}{\sqrt{2}}\right)^T$. Find a vector $\mathbf{v}_3$ so that the set $\left\{\mathbf{v}_1,\mathbf{v}_2,\mathbf{v}_3\right\}$ is orthonormal.

Problem 3 (cont.)

$S=\left\{(1,1)^T, (-1,1)^T\right\}$

Problem 4

For what values of $b$ are the vectors $(-6,b,2)^T$ and $(b,b^2,b)^T$ orthogonal?

Problem 5

Verify that the matrix $A$ is orthogonal. Use two different methods.

$ A= \begin{pmatrix} ~\frac{1}{3}&-\frac{2}{3}&\frac{2}{3}\\ ~\frac{2}{3}&~\frac{2}{3}&\frac{1}{3}\\ -\frac{2}{3}&~\frac{1}{3}&\frac{2}{3} \end{pmatrix} $

Problem 6

Show that the following symmetric matrices are positive definite and calculate their eigenvalues. Do your results agree with Theorem 5.7.1?

(a) $A=\mattwotwo{2}{1}{1}{2}$	(b) $A=\mattwotwo{~~2}{-1}{-1}{~~2}$
(c) $A=\mattwotwo{3}{1}{1}{3}$	(d) $A=\mattwotwo{4}{1}{1}{4}$

*(e) Is the matrix $A=\begin{pmatrix} 1&1&0\\ 1&1&0\\ 0&0&2 \end{pmatrix}$ positive definite?

Problem 6 (cont.)

(a) $A=\mattwotwo{2}{1}{1}{2}$

Problem 7

What song would a collaborative filtering system recommend to me, using the data from section 5.3.2, if I have made the following ratings:

a '5' for Song 1, and a '4' for Song 10;
a '1' for each of Song 1, Song 7 and Song 8.

Song	User ratings
	U1	U2	U3	U4	U5	U6	U7	U8
Song 1	0	5	3	0	1	0	1	1
Song 2	0	0	0	0	0	5	3	2
Song 3	5	4	4	5	4	0	3	3
Song 4	5	5	3	3	4	4	3	2
Song 5	0	0	0	0	3	0	4	3
Song 6	0	0	0	0	4	5	0	2
Song 7	0	0	5	2	3	3	0	1
Song 8	5	3	3	2	4	2	1	1
Song 9	0	0	0	5	1	3	1	0
Song 10	0	4	4	3	4	1	0	3

Problem 7 (cont.)

$$R=\begin{pmatrix} 0&5&3&0&1&0&1&1\\ 0&0&0&0&0&5&3&2\\ 5&4&4&5&4&0&3&3\\ 5&5&3&3&4&4&3&2\\ 0&0&0&0&3&0&4&3\\ 0&0&0&0&4&5&0&2\\ 0&0&5&2&3&3&0&1\\ 5&3&3&2&4&2&1&1\\ 0&0&0&5&1&3&1&0\\ 0&4&4&3&4&1&0&3 \end{pmatrix}$$

1. $\mathbf x = (5,0,0,0,0,0,0,0,0,4)^T$

2. $\mathbf x = (1,0,0,0,0,0,1,1,0,0)^T$

Python
code
💻 ➡️
Python online


import numpy as np

R = np.array([
	[0, 5, 3, 0, 1, 0, 1, 1],
	[0, 0, 0, 0, 0, 5, 3, 2],
	[5, 4, 4, 5, 5, 0, 3, 3],
	[5, 5, 3, 3, 4, 4, 3, 2],
	[0, 0, 0, 0, 3, 0, 4, 3],
	[0, 0, 0, 0, 4, 5, 0, 2],
	[0, 0, 5, 2, 3, 3, 0, 1],
	[5, 3, 3, 2, 4, 2, 1, 1],
	[0, 0, 0, 5, 1, 3, 1, 0],
	[0, 4, 4, 3, 4, 1, 0, 3]
])
# Change the next line for the other vector
X = np.array([[5],[0],[0],[0],[0],[0],[0],[0],[0],[4]])

normsR = np.linalg.norm(R, axis=0)
Q = R / normsR

normsX = np.linalg.norm(X, axis=0)
x_n = X / normsX

y = Q.T @ x_n

rating = R @ (1/np.sum(y) * y)

print( "Rating:\n", rating )


0 5 3 0 1 0 1 1
0 0 0 0 0 5 3 2
5 4 4 5 5 0 3 3
5 5 3 3 4 4 3 2
0 0 0 0 3 0 4 3
0 0 0 0 4 5 0 2
0 0 5 2 3 3 0 1
5 3 3 2 4 2 1 1
0 0 0 5 1 3 1 0
0 4 4 3 4 1 0 3

Problem 8

My friend's new ice-cream company is developing 10 flavours, and has employed 8 tasters to rate them. Their ratings are shown below.

If I rate the Vanilla a '5' and the Lemon Sorbet a '3', which three flavours would you recommend I try next?

Flavour	Tester ratings
	T1	T2	T3	T4	T5	T6	T7	T8
Chocolate	5	5	5	3	3	5	3	2
Vanilla	4	1	2	5	5	4	2	1
Strawberry	1	4	2	1	1	5	4	2
Rum and Raisin	4	5	2	3	3	2	3	4
Salted Caramel	2	4	5	3	2	2	3	3
Pistachio	1	1	1	2	5	1	3	3
Cookies and Cream	3	5	2	4	5	2	1	5
Bubblegum	1	5	4	1	5	3	1	4
Lemon Sorbet	4	4	2	5	5	2	3	2
Mandarine Sorbet	1	2	5	4	4	5	5	2

Problem 8 (cont.)

\[R = \begin{pmatrix} 5 & 5 & 5 & 3 & 3 & 5 & 3 & 2 \\ 4 & 1 & 2 & 5 & 5 & 4 & 2 & 1 \\ 1 & 4 & 2 & 1 & 1 & 5 & 4 & 2 \\ 4 & 5 & 2 & 3 & 3 & 2 & 3 & 4 \\ 2 & 4 & 5 & 3 & 2 & 2 & 3 & 3 \\ 1 & 1 & 1 & 2 & 5 & 1 & 3 & 3 \\ 3 & 5 & 2 & 4 & 5 & 2 & 1 & 5 \\ 1 & 5 & 4 & 1 & 5 & 3 & 1 & 4 \\ 4 & 4 & 2 & 5 & 5 & 2 & 3 & 2 \\ 1 & 2 & 5 & 4 & 4 & 5 & 5 & 2 \end{pmatrix} \]

$\mathbf x = (0,5,0,0,0,0,0,3,0,0)^T$

Python
code
💻 ➡️
Python online


import numpy as np

R = np.array([
	[5, 5, 5, 3, 3, 5, 3, 2],
	[4, 1, 2, 5, 5, 4, 2, 1],
	[1, 4, 2, 1, 1, 5, 4, 2],
	[4, 5, 2, 3, 3, 2, 3, 4],
	[2, 4, 5, 3, 2, 2, 3, 3],
	[1, 1, 1, 2, 5, 1, 3, 3],
	[3, 5, 2, 4, 5, 2, 1, 5],
	[1, 5, 4, 1, 5, 3, 1, 4],
	[4, 4, 2, 5, 5, 2, 3, 2],
	[1, 2, 5, 4, 4, 5, 5, 2]
])

X = np.array([[0],[5],[0],[0],[0],[0],[0],[3],[0],[0]])

normsR = np.linalg.norm(R, axis=0)
Q = R / normsR

normsX = np.linalg.norm(X, axis=0)
x_n = X / normsX

y = Q.T @ x_n

rating = R @ (1/np.sum(y) * y)

print( "Rating:\n", rating )


5 5 5 3 3 5 3 2
4 1 2 5 5 4 2 1
1 4 2 1 1 5 4 2
4 5 2 3 3 2 3 4
2 4 5 3 2 2 3 3
1 1 1 2 5 1 3 3
3 5 2 4 5 2 1 5
1 5 4 1 5 3 1 4
4 4 2 5 5 2 3 2
1 2 5 4 4 5 5 2

Problem 9

Determine least-squares lines of best fit for the following data, to match the proposed data models

$x_i$	1	2	3	4	5	6	7	8	9	10
$y_i$	9.876	9.862	9.833	9.465	7.635	6.714	3.559	-0.397	-7.424	-19.429

1. Linear model, $y=mx+c$

2. Cubic model, $y=ax^3+bx^2+cx+d;$ or

3. Hyperbolic sinusoidal model $$y=a\cosh\left(\frac{x}{2}\right)+b\sinh\left(\frac{x}{2}\right)+c$$

Python code 💻


import numpy as np

# Data
x = np.array([1,2,3,4,5,6,7,8,9,10], dtype=float)
y = np.array(
	[9.876, 9.862, 9.833, 9.465, 7.635, 6.714, 3.559, -0.397, -7.424, -19.429],
	dtype=float)

y_col = y.reshape(-1, 1) # vector column

# --------- 1. Linear model: y = a1*x + a0 ---------
A_lin = np.column_stack([x, np.ones_like(x)])
coef_lin = np.linalg.inv(A_lin.T @ A_lin) @ (A_lin.T @ y_col)

# --------- 2. Cubic model: y = a3*x^3 + a2*x^2 + a1*x + a0 ---------
A_cubic = np.column_stack([x**3, x**2, x, np.ones_like(x)])
coef_cubic = np.linalg.inv(A_cubic.T @ A_cubic) @ (A_cubic.T @ y_col)

# --------- 3. Hyperbolic model: y = a*cosh(x/2) + b*sinh(x/2) + c ---------
A_hyper = np.column_stack([np.cosh(x/2), np.sinh(x/2), np.ones_like(x)])
coef_hyper = np.linalg.inv(A_hyper.T @ A_hyper) @ (A_hyper.T @ y_col)

# Print results
print("Linear coefficients [a1, a0]:\n", coef_lin)
print("Cubic coefficients [a3, a2, a1, a0]:\n", coef_cubic)
print("Hyperbolic coefficients [a, b, c]:\n", coef_hyper)