Birla Institute of Technology & Science, Pilani

Work Integrated Learning Programmes Division

Second Semester 2024-2025

Mid-Semester Test
EC-2 Regular

Course No. : AIMLCZC416

Course Title : Mathematical Foundations for Machine Learning
Nature of Exam : Closed Book
Weightage : 30% No. of Pages = 2
Duration : 2 Hours No. of Questions = 6
Date of Exam : 28-06-2025, AN
Note to Students:
1. Please follow all the Instructions to Candidates given on the cover page of the answer book.
2. All parts of a question should be answered consecutively. Each answer should start from a fresh page.
3. Assumptions made if any, should be stated clearly at the beginning of your answer.
4. Notation used here are as per the text book.

1 3 −1 4 1
Q1. Consider the matrices 𝐴 = [ 2 6 −2 8 ] and 𝑏 = [2].
3 9 −3 12 3

(a) Find the Echelon form of A and hence find the rank of A. [2M]

1 3 −1 4 1 3 −1 4
R2 ⟵ -2R1 + R2
[ 2 6 −2 8 ] ≈ [ 0 0 0 0 ]
3 9 −3 12 0 0 0 0 R3 ⟵ -3R1 + R3
[1M]

Since in the row echelon form there are only one non zero row, rank of A = 1. [1M]

(b) Find all the solutions of AX = b. [2M]

1 3 −1 4 1 1 3 −1 4 1
The augmented matrix = [ 2 6 −2 8 2 ] ≈ [ 0 0 0 0 0 ] [1M]
3 9 −3 12 3 0 0 0 0 0
Converting back to equation we have
𝑥1 + 3𝑥2 − 𝑥3 + 4𝑥4 =1. Since 𝑥2 , 𝑥3 , 𝑥4 are free variables, the solutions are
1 − 3𝑟 + 𝑠 − 4𝑡
{[ 𝑟 ] : 𝑟, 𝑠, 𝑡 ∈ 𝑅}. [1M]
𝑠
𝑡
1
(c) Suppose if the matrix 𝑏 = [2] what can you conclude about the linear system of equations
2
AX = b, Where A is the same matrix given in (a). [1M]
1 3 −1 4 1
The Echelon form of the augmented matrix = [ 0 0 0 0 0]
0 0 0 0 −1
Since the last row implies 0 = -1, the system has no solution. [1M]
 𝑚 1 1
Q2. Consider the matrix 𝐴 = [ 1 𝑚 1 ], where m ≥ 2, a positive integer.
1 1 𝑚
(a) Prove that A is a positive definite matrix. [3M]
𝑥1
Let 𝑥 = [ 2 ] be any non-zero vector in R3.
𝑥
𝑥3
𝑚 1 1 𝑥1 𝑚𝑥1 + 𝑥2 + 𝑥3
𝑇
Then 𝑥 𝐴𝑥 = 1 𝑥2 𝑥3 ( [ 1 𝑚 1 ] [ 2 ])
[𝑥 ] 𝑥 [1𝑀] = 1 𝑥2 𝑥3 [𝑥1 + 𝑚𝑥2 + 𝑥3 ]=
[𝑥 ]
1 1 𝑚 𝑥3 𝑥1 + 𝑥2 + 𝑚𝑥3

= 𝑚(𝑥1 2 + 𝑥2 2 + 𝑥3 2 ) + 2𝑥1 𝑥2 + 2𝑥1 𝑥3 + 2𝑥2 𝑥3

= (𝑥1 + 𝑥2 )2 + (𝑥1 + 𝑥3 )2 + (𝑥2 + 𝑥3 )2 + (𝑚 − 2)(𝑥1 2 + 𝑥2 2 + 𝑥3 2 )[1M]
Since at least one 𝑥𝑖 ≠ 0 𝑎𝑛𝑑 𝑚 ≥ 2, 𝑥 𝑇 𝐴𝑥 > 0. [1M]

(b) Using this matrix A, find an inner product on R3. [1M]

𝑥1 𝑦1
Suppose 𝑥 = [𝑥2 ] , 𝑦 = [𝑦2 ] are any two vectors in R3.
𝑥3 𝑦3
𝑚 1 1 𝑦1
< 𝑥, 𝑦 >𝐴 = 𝑥 𝐴𝑦 = [𝑥1 𝑥2 𝑥3 ] ( [ 1 𝑚 1 ] [𝑦2 ])
𝑇

1 1 𝑚 𝑦3
𝑚𝑦1 + 𝑦2 + 𝑦3
= [𝑥1 𝑥2 𝑥3 ] [𝑦1 + 𝑚𝑦2 + 𝑦3 ]
𝑦1 + 𝑦2 + 𝑚𝑦3
= 𝑚(𝑥1 𝑦1 + 𝑥2 𝑦2 + 𝑥3 𝑦3 ) + 𝑥1 𝑦2 + 𝑥1 𝑦3 + 𝑥2 𝑦1 + 𝑥2 𝑦3 + 𝑥3 𝑦1 + 𝑥3 𝑦2

1
(c) Using the inner product defined in (b) find the norm of the vector [−1]. [1M]
−2
1 𝑚 1 1 1 𝑚−3
||[−1]|| = [1 − 1 − 2] ( [ 1 𝑚 1 ] [−1]) = [1 − 1 − 2] [−𝑚 − 1]
−2 1 1 𝑚 −2 −2𝑚
= 6𝑚 − 2 [1M]

0 1 2 0
Q3. (a) Is the set {[1] ,
[0] , [1] , [0] } a basis for R4. [3M]
2 0 0 0
0 3 0 4
0 1 2 0
Forming a matrix with columns as given vectors: [1 0 1 0]
2 0 0 0
0 3 0 4
1 0 1 0 1 0 1 0
≈ [0 1 2 0] 𝑅1 ↔ 𝑅2 ≈ [0 1 2 0 ] 𝑅3 ← −2𝑅1 + 𝑅3
2 0 0 0 0 0 −2 0
0 3 0 4 0 3 0 4
1 0 1 0 1 0 1 0
≈ [0 1 2 0 ] 𝑅4 ← −3𝑅2 + 𝑅4 ≈ [0 1 2 0 ] 𝑅4 ← −3𝑅3 + 𝑅4 [1M]
0 0 −2 0 0 0 −2 0
0 0 −6 4 0 0 0 4

Since each column in the echelon form has pivot element the system of equations
 0 1 2 0 0
a[ ] + 𝑏 [ ] + 𝑐 [ ] + 𝑑 [ ] = [0] has only a trivial solution. Hence the given vectors are
1 0 1 0
2 0 0 0 0
0 3 0 4 0
linearly independent. [1M]
Since the dimension of R4 is 4 and the given set consists of 4 elements which are linearly
independent, the given set is a basis of R4. [1M]

1
(b) Write [0] as a linear combination of the elements given in (a). [2M]
0
0
0 1 2 0 1 1 0 1 0 0
The augmented matrix = [ 1 0 1 0 0] ≈ [0 1 2 0 1] 𝑅1 ↔ 𝑅2
2 0 0 0 0 2 0 0 0 0
0 3 0 4 0 0 3 0 4 0
1 0 1 0 0
≈ [0 1 2 0 1 ] 𝑅3 ← −2𝑅1 + 𝑅3
0 0 −2 0 0
0 3 0 4 0
1 0 1 0 0 1 0 1 0 0
≈ [ 0 1 2 0 1 ] 𝑅4 ← −3𝑅2 + 𝑅4 ≈ [ 0 1 2 0 1 ] 𝑅4 ← −3𝑅3 + 𝑅4
0 0 −2 0 0 0 0 −2 0 0
0 0 −6 4 − 3 0 0 0 4 −3
[1M]

Converting back into equations we get

a+c=0
b – 2c = 1
-2c = 0
4 d = -3 implies d = -3/4
c = 0 implies b =1 and a = 0.
1 0 1 2 0
Therefore [0] = 0 [1] + 1 [0] + 0 [1] + (− 4) [0] [1M]
3
0 2 0 0 0
0 0 3 0 4
Note: If they guess a, b, c, and d without going through the steps, give full 2 marks.

−1 0
Q4. (a) Suppose 𝐴 = [0 −1]. Find the diagonal matrix D and the invertible matrix P
2 −2
Such that (𝐴𝑇 𝐴)𝑃 = 𝑃𝐷 [4M]
−1 0
−1 0 2 5 −4
𝐴𝑇 𝐴 = [ ] [0 −1] = [ ]
0 −1 −2 −4 5
2 −2
The characteristic equation is (5 − λ)2 = 16
The eigen values are 1, 9. [1M]
The eigen vector corresponding to 1 is obtained by solving the equations
4 −4 𝑥 0 1
[ ] [ ] = [ ]. ie 4𝑥 − 4𝑦 = 0. One solution is [ ]. [1/2M]
−4 4 𝑦 0 1

The eigen vector corresponding to 9 is obtained by solving the equations

−4 −4 𝑥 0 1
[ ] [𝑦] = [ ]. ie −4𝑥 − 4𝑦 = 0. One solution is [ ]. [1/2M]
−4 −4 0 −1
1 0 1 1
Hence the matrix 𝐷 = [ ] and the matrix 𝑃 = [ ].
0 9 1 −1
 (b) Why for any mxn matrix A, ATA is always diagonalizable [1M].

(𝐴𝑇 𝐴)𝑇 = 𝐴𝑇 (𝐴𝑇 )𝑇 = 𝐴𝑇 𝐴.

Hence ATA is symmetric and we know that every symmetric matrix is diagonalizable. [1M]

Q5. Consider the function 𝑓(𝑥, 𝑦, 𝑧) = 𝑥 2 + 𝑦 2 + 𝑧 2 + 3𝑥𝑦𝑧.

(a) Find ∇f(x,y,z) at (1, -1, 2). [2M]
2𝑥 + 3𝑦𝑧
𝜵𝒇(𝑥, 𝑦, 𝑧) = [2𝑦 + 3𝑥𝑧 ] [1M]
2𝑧 + 3𝑥𝑦

−4
𝜵𝒇(1, −1, 2) = [ 4 ] [1M]
1
1
(b) Is the vector [−1] orthogonal to ∇f(x,y,z) at (1, -1, 2). Why? [1M]
0
1 −4
The inner product < [−1] , [ 4 ] > = −4 − 4 = −8 ≠ 0.
0 1
Hence these vectors are not orthogonal. [1M]

(c) Suppose if you further consider x, y, z as functions of t given by x(t) = t, y(t) = t2, z(t) = t3.
𝑑𝑓
Using the chain rule find 𝑑𝑡 . [2M]
𝑑𝑓 𝜕𝑓 𝑑𝑥 𝜕𝑓 𝑑𝑦 𝜕𝑓 𝑑𝑧
= 𝜕𝑥 𝑑𝑡 + 𝜕𝑦 𝑑𝑡 + 𝜕𝑧 𝑑𝑡 [1M]
𝑑𝑡
= (2𝑥 + 3𝑦𝑧)1 + (2𝑦 + 3𝑥𝑧)(2𝑡) + (2𝑧 + 3𝑥𝑦)3 t2
= (2t + 3t5) + (2t2 + 3t4)(2t) + (2t3 + 3t3) 3t2
= 24t5 + 4t3 + 2t [1M]
2
Q6. (a) Find the Taylor polynomial of degree 3 of the function 𝑓(𝑥) = 𝑒 𝑥 about the point 1. [3M]
2
𝑓(𝑥) = 𝑒 𝑥
2
𝑓 (1) (𝑥) = 2𝑥𝑒 𝑥
2 2 2 2
𝑓 (2) (𝑥) = 2𝑥2𝑥𝑒 𝑥 + 2𝑒 𝑥 = 4𝑥 2 𝑒 𝑥 + 2𝑒 𝑥
2 2 2 2 2
𝑓 (3) (𝑥) = 8𝑥𝑒 𝑥 + 4𝑥 2 2𝑥𝑒 𝑥 + 4𝑥𝑒 𝑥 = 8𝑥 3 𝑒 𝑥 + 12𝑥𝑒 𝑥 [1M]
𝑓(1) = 𝑒
𝑓 (1) (1) = 2𝑒
𝑓 (2) (1) = 6𝑒
𝑓 (3) (1) = 20𝑒 [1M]
2
Taylor polynomial of degree 3 of the function 𝑓(𝑥) = 𝑒 𝑥 about the point 1 is
6𝑒 20𝑒
𝑇3 (𝑥) = 𝑒 + 2𝑒(𝑥 − 1) + 2 (𝑥 − 1)2 + 6 (𝑥 − 1)3
10𝑒
= 𝑒 + 2𝑒(𝑥 − 1) + 3𝑒(𝑥 − 1)2 + (𝑥 − 1)3 [1M]
3

(b) Find the Hessian matrix of the function f(x, y) = xy. [2M]

𝑓𝑥𝑥 𝑓𝑥𝑦
Hessian matrix of the function 𝑓(𝑥, 𝑦) = [ ] [1/2 M]
𝑓𝑥𝑦 𝑓𝑦𝑦
𝑓𝑥 = 𝑦, 𝑓𝑦 = 𝑥. [1/2 M]
𝑓𝑥𝑥 = 0 𝑓𝑦𝑦 = 0 and 𝑓𝑥𝑦 = 1
 0 1
Hessian matrix of the function 𝑓(𝑥, 𝑦) = [ ] [1M]
1 0
----------------------------------------------------------------------------------------