Constrained Optimisation: -
In mathematical optimization, constrained optimisation (in some contexts called constraint
optimisation) is the process of optimizing an objective function with respect to some
variables in the presence of constraints on those variables. The objective function is either a
cost function, which is to be minimised or utility function, which is to be maximised.
Condition for Maximisation / Minimisation: -
Second Order Conditions: -
Suppose we want to optimize
z= f(x1, x2)
subject to. c= g(x1, x2)
So, the FOC are:
f1−λg1=0, f2−λg2=0 and C−g(x1, x2)=0
Now,
dz = f1dx1+f2dx2
and dc = g1dx1+g2dx2=0
⇒dx2 = − g 2 dx1
g1
As we know that g1= g1(x1, x2) and g2= g2(x1, x2)
⇒dx2= f(x1,x2)
Now the second order total differentiation of z is
d2z= d(dz)= d(f1dx1+f2dx2)
∂(dz ) ∂(dz )
d2z= .dx1 + . dx2
∂x 1 ∂x 2
¿ ∂[f 1 dx 1+ f 2 dx 2] ∂[f 1 dx 1+ f 2 dx 2]
.dx1 + . dx2
∂x1 ∂ x2
∂ dx 2 ∂ dx 2
=[f11dx1 + f12dx2 + f2 ]dx1 + [f21dx1+f22dx2 + f2 ]dx2
∂x 1 ∂x 2
∂ dx 2 ∂ dx 2
= f11dx12 + f12dx2dx1 + f2 .dx1 + f21dx1dx2+f22dx22 + f2 .dx2
∂x 1 ∂x 2
∂ dx 2 ∂ dx 2
= [f11dx12 + 2f12dx2dx1 + f22dx22] + f2[ .dx1 + .dx2][ f12-f21]
∂x 1 ∂x 2
= [f11dx12 + 2f12dx2dx1 + f22dx22] + f2d(dx2)
= [f11dx12 + 2f12dx2dx1 + f22dx22] + f2d2x2
Now,
dc= g1dx1+g2dx2
d2c= d(dc)= ∂dc/∂x1.dx1+∂dc/∂x2.dx2
¿ ∂[g 1 dx 1+g 2dx 2] ∂[g 1 dx 1+ g 2dx 2]
.dx1 + . dx2
∂ x1 ∂x2
By the same way we can get:
d2c = [g11dx12 + 2g12dx1dx2 + g22dx22] + g2d2x2
As d2c=0 ⇒
d2c = [g11dx12 + 2g12dx1dx2 + g22dx22] + g2d2x2
⇒ d2x2 = − [g11dx12 + 2g12dx1dx2 + g22dx22]/g2
Now substituting the value of d2x2 in the d2z equation we can get:
d2 z= [f11dx12 + 2f12dx2dx1 + f22dx22] − f2/g2 [g11dx12 + 2g12dx1dx2 + g22dx22]
f2 −f 2 f2
d2z = [f11− .g11]dx12 + 2[f12 .g12]dx1dx2 + [f22− .g22]dx22
g2 g2 g2
Now from 1st order condition we get:
f2=λg2 ⇒f2/g2=λ
So,
d2z = [ f11−λg11]dx12 +2[f12−λg12]dx1dx2 + [f22−λg22]dx22
So, for maximising Z⇒ d2z is seminegative definite subject to dg=0
For minimising Z⇒d2z is semipositive definite subject to dg=0
These are the second-order necessary conditions.
Second-order sufficient conditions are:
d2z is negative definite subject to dg=0 for maximization of z
d2z is positive definite subject to dg=0 for minimization of z
Lagrange Method: -
Suppose we want to maximize utility
U = U(x₁, x₂) subject to the budget constraint:
M̄ = P₁x₁ + P₂x₂
So, we are defining a function Z as
Z = U(x₁, x₂) + λ[M̄ − P₁x₁ − P₂x₂]
Now the FOC (First Order Conditions) are:
∂Z/∂λ = M̄ − P₁x₁ − P₂x₂ = 0
∂Z/∂x₁ = U₁ − λP₁ = 0
∂Z/∂x₂ = U₂ − λP₂ = 0
Where U₁ = ∂U/∂x₁ and U₂ = ∂U/∂x₂
Now by solving these equations we can set:
U₁ = λP₁ and U₂ = λP₂
Or, U₁/U₂ = P₁/P₂
So, in general, our objective function is:
Z = f(x₁, x₂)
And the constraint function is:
g = g(x₁, x₂) = c
So we want to optimize our objective function subject to the constraint function.
Our Lagrange function is:
L = f(x₁, x₂) + λ[c − g(x₁, x₂)]
The FOC are:
∂L/∂λ = c − g(x₁, x₂) = 0
∂L/∂x₁ = f₁ − λg₁ = 0
∂L/∂x₂ = f₂ − λg₂ = 0
Where:
f₁ = ∂f/∂x₁, f₂ = ∂f/∂x₂
g₁ = ∂g/∂x₁, g₂ = ∂g/∂x₂
So, by solving we can get:
f₁ = λg₁ and f₂ = λg₂
And c* = g(x₁*, x₂*)
So, f₁/f₂ = g₁/g₂
Example:
Find the optimum value of the function
Z = 2x₁² − 24x₂ + 5
subject to
x₁ − x₂ = 5
Ans: To solve this problem, let us first form the Lagrangian function, which becomes
L = 2x₁² − 24x₂ + 5 + λ(x₁ − x₂ − 5)
where λ is the Lagrange multiplier.
First-order conditions of optimization require:
Lₓ₁ = 4x₁ + λ = 0
Lₓ₂ = −24 − λ = 0
Lλ = x₁ − x₂ − 5 = 0
Here we get three equations with three unknowns: x₁, x₂, and λ.
Solving them we get:
x₁ = 6, x₂ = 1, and λ = −24
Then the optimum value of Z,
Z = 2x₁² − 24x₂ + 5 = 2(6)² − 24(1) + 5 = 72 − 24 + 5 = 53
So, Z = 53
Bordered Hessian: -
Suppose we are considering a quadratic form:
q=au2+2huv+bv2 subject to αu+βv= 0
Now
−α
αu+βv= 0⇒ v=
β.u
So,
−α −α
q=au2 + 2hu( ) + b( )
β.u β.u
α 2 α2 2
=au2−2h .u + b.( ¿ .u
β β2
u2
=(aβ2−2hαβ+bα2) .( ¿.
β2
u2
As ( ¿ >0, so:
β2
q is positive definite if aβ2−2hαβ+bα2> 0
q is negative definite if aβ2−2hαβ+bα2<0
| |
0 α β
Now we can write: aβ2−2hαβ+bα2 = - α a h
β h b
| |
0 α β
2hαβ− aβ2− bα2= α a h
β h b
So,
q is positive definite iff the above determinant < 0
q is negative definite iff the above determinant > 0
Now considering the value of d2z
d2z = ( f11−λg11)dx12 +(f12−λg12)dx1dx2 + (f22−λg22)dx22
subject to g1dx1+g2dx2=0
Now as g1dx1+g2dx2= 0 ⇒dx2= −g1/g2.dx1
So,
d2z = (f11−λg11)dx12 + 2(f12−λg12)dx1(−g1/g2.dx1) + (f22−λg22)(− g1/g2.dx1)2
g1 2
= (f11−λg11)dx12 − 2(f12−λg12) dx1 + (f22−λg22)g12/g22.dx12
g2
=[(f11−λg11)g22 − 2(f12−λg12)g1g2 + (f22−λg22)g12]⋅ dx12/g22
As dx12/g22 >0, the sign of d2z depends on the sign of the expression in the square brackets.
So,
d2z is positive definite iff
[(f11−λg11)g22 − 2(f12−λg12)g1g2 + (f22−λg22)g12] >0
d2z is negative definite iff
[(f11−λg11)g22 − 2(f12−λg12)g1g2 + (f22−λg22)g12] <0
Now,
− [(f11−λg11)g22 − 2(f12−λg12)g1g2 + (f22−λg22)g12] ⇒
| H̄ |=
| |
0 g₁ g₂
g ₁ f ₁₁−λg ₁₁ f ₁₂−λg ₁₂
g ₂ f ₁₂−λg ₁₂ f ₂₂−λg ₂₂
d2z is positive definite iff | H̄ |<0
So,
d2z is negative definite iff | H̄ |>0
Now we know that,
|
∣J∣= −g ₁ f ₁₁−λg ₁₁ f ₁₂−λg ₁₂
|
0 −g ₁ −g ₂
−g ₂ f ₁₂−λg ₁₂ f ₂₂−λg ₂₂
So, multiplying 1st row and 1st column by -1:
∣J∣=
| |
g ₁ f ₁₁−λg ₁₁ f ₁₂−λg ₁₂ = | H̄ |
0 g₁ g₂
g ₂ f ₁₂−λg ₁₂ f ₂₂−λg ₂₂
That is the endogenous-Variable Jacobian is identical with the bordered
Hessian.
| H̄ |≠ 0⇒∣J∣≠ 0
Now from the second-order sufficient condition we can get that
⇒ Implicit function theorem exists
⇒ This practice would be followed when we analyse the comparative statics of the
constrained optimisation problems
n-variable case
Optimize z= f(x1,x2,…,xn)
subject to c= g(x1,x2,…,xn)
FOC:
f1 f2 fn
g 1 = g 2 =⋯= gn =λ
SOC:
|0 g ₁ g ₂ ... gₙ|
|g ₁ f ₁₁−λg ₁₁ f ₂₁− λg ₂₁... fₙ ₁− λgₙ₁|
|g ₂ f ₁₂−λg ₁₂ f ₂₂− λg ₂₂... fₙ ₂− λgₙ₂|
|⋮ ⋮ ⋮ ⋱⋮ |
|gₙ f ₁ₙ−λg ₁ ₙ f ₂ ₙ−λg ₂ ₙ... fₙₙ−λgₙₙ|
= | H̄ |
So, for maximisation, the second-order sufficient conditions are:
∣ H̄ 2∣>0, ∣ H̄ 3∣<0, ∣ H̄ 4∣>0
And for minimisation, the second-order sufficient conditions are:
∣ H̄2∣<0, ∣ H̄3∣<0, ∣ H̄4∣<0
Example:
Optimize z = x(y + 5)
Subject to x + y = 10
To solve this problem, let us first form the Lagrangian function, which becomes
L = x(y + 5) + λ [10 - x - y], where λ is the Lagrangian Multiplier
∂L/∂x = y + 5 - λ = 0
∂L/∂y = x - λ = 0
∂L/∂λ = 10 - x - y = 0
Solving these equations:
x + y = 10
y - x = -5 ⇒ 2y = 5 ⇒ y = 2.5
x = 10 - y = 7.5
λ = x = 7.5
So, x* = 7.5, y* = 2.5, λ* = 7.5
z* = 7.5(2.5 + 5) = 7.5 × 7.5 = 56.25
Second-Order Condition (SOC):
|
| H̄ | = g ₁ f ₁₁−λg ₁₁ f ₁₂−λg ₁₂
|
0 g₁ g₂
g ₂ f ₁₂−λg ₁₂ f ₂₂−λg ₂₂
We know FOC:
y + 5 − λ = 0 ⇒ f₁ = y + 5, g₁ = -1
x − λ = 0 ⇒ f₂ = x, g₂ = -1
So:
f₁₁ = 0, f₁₂ = 1, f₂₂ = 0, f₂₁ = 1
g₁₁ = 0, g₁₂ = 0, g₂₁ = 0, g₂₂ = 0
|
| H̄ | = −1 0
|
0 −1 −1
1
−1 1 0
=2>0
⇒ Maximisation
Envelope Theorem: -
Envelope Theorem states that partial derivative of the maximum (minimum) value function
as a result of change in parameter is equal to partial derivative of the Lagrangian function as a
result of change in parameter. So, if we maximize utility function, i.e.,
max U = U(x1, x2)
subject to M̄ = P₁x₁ + P₂x₂
and if the U* is the maximum value function, i.e., maximum utility then
∂U*/∂M̄ = ∂L/∂M̄
Proof: -
Let us consider the utility maximization problem max U = U(x1, x2)
subject to M̄ = P₁x₁ + P₂x₂
∴ L = U(x1, x2) + λ[M̄ - P₁x₁ - P₂x₂]
So, FOC ⇒ U1 =λP1 or U1 - λP1 = 0
U2 =λP2 or U2 - λP2 = 0
and M̄ - P₁x₁ - P₂x₂= 0
Now solving these FOC we can get
x1* = x1(P1, P2, M̄ )
x2* = x2(P1, P2, M̄ )
considering that SOC is satisfied then substituting these values of x1* and x2* in the utility
function we get
U* = U*[x1*, x2*]
U* = U*[x1(P1, P2, M̄ ); x2( P1, P2, M̄ )]
or U* = U*[ P1, P2, M̄ ]
This is called as the maximum value function/maximized utility function/indirect utility
function.
Now U* = U*((P1, P2, M̄ ))
So, ∂U*/∂M̄ = (∂U*/∂x1*) . (∂x1*/∂M̄ ) + (∂U*/∂x2*) . (∂x2*/∂M̄ )
∂U*/∂M̄ = U1(∂x1*/∂M̄ ) + U2(∂x2*/∂M̄ )
Now we know that
M̄ = P₁x₁ + P₂x₂
or M̄ - P₁x₁ -P₂x₂= 0
So, this equation also satisfies for x1* and x2*
So, M̄ - P1x1* - P2x2* = 0
or 1 - P1(∂x1*/∂M̄ ) - P2(∂x2*/∂ M̄ ) = 0
or 1 - Σ Pj(∂xj*/∂M̄ ) = 0
Now we can write
∂U*/∂M̄ = U1(∂x1*/∂M̄ ) + U2(∂x2*/∂M̄ ) + λ[1 - P1(∂x1*/∂M̄ ) - P2(∂x2*/∂M̄ )]
= [U1- λP1] ∂x1*/∂M̄ + [ U2 - λP2 ]∂x2*/∂M̄ + λ
Or ∂U*/∂M̄ = λ [as from FOC U1 - λP1 = 0
U2 - λP2 = 0]
Now from the Lagrange function we can write
L = U(x1, x2) + λ[M̄ - P₁x₁ - P₂x₂]
or ∂L/∂M̄ = λ
So, we can write
∂U*/∂M̄ = ∂L/∂M̄
=> Envelope theorem
we know that ∂U*/∂M̄ = λ
=> λ is the marginal utility of money.
Example
A consumer wants to maximize utility from buying two goods, x and y, given their
prices and income.
Max U(x, y) = x0.5 y0.5
Subject to the budget constraint:
Px x + Py y = M
We treat income M as the parameter we are interested in (how utility changes with income).
We form the Lagrangian:
L(x, y, λ) = x0.5 y0.5+ λ (M - Px x – Py y)
First-order conditions (FOC):
∂L/∂x = 0.5 x-0.5 y0.5 - λ Px = 0
∂L/∂y = 0.5 x0.5 y-0.5 - λ Py = 0
∂L/∂λ = M - Px x - Py y = 0
Solving these gives the optimal demand:
x* = M / (2Px), y* = M / (2Py)
Putting x* and y* into the utility function:
V(M) = U(x*, y*) = (M / 2Px) 0.5 (M / 2Py)0.5
V(M) = M / 2√(Px Py)
The envelope theorem says:
dV/dM = ∂L/∂M = λ*
From the first-order condition:
0.5 x-0.5 y0.5 = λ Px
λ = 0.5 x-0.5 y0.5 /Px
Substituting optimal x* and y*:
λ* = 1/2√(Px Py)
dV/dM = d/dM [M /(2√(Px Py)] = 1/2√(Px Py) = λ*
The envelope theorem allows us to compute how maximum utility changes with income
without differentiating the optimal values of x and y.
It tells us that the marginal utility of income is simply λ*, the Lagrange multiplier.
