Lagrange Multipliers Basics
R.Y.Qassim
November 2106
1. Introduction
The purpose of this brief technical note is to introduce the
basics of the application of the Lagrange multiplier method
to constrained optimisation problems. This note is based on
[1.2].
2. Constrained optimisation problem
Consider the following constrained optimisation problem.
Optimise y = y(x1, x2,...., xn) , (1)
subject to
1(x1, x2,..., xn) = 0 , (2.1)
..............................
m(x1, x2,..., xn) = 0 . (2.m)
3. Lagrange multiplier method: necessary conditions
The Lagrange multiplier method states that the optimum
occurs at values of the xs that satisfy the following set of
algebraic equations
y/x1 - 1 1/x1 -...- m m/x1 = 0 , (3.1)
..............................................................
y/xn - 1 1/xn -...- m m/xn = 0 . (3.n)
4. Solution procedure
The set of n algebraic equations (2.1)-(2.m) and and the
set of m algebraic equations (3.1)-(3.n), forming a set of
(n+m) algebraic equations, needs to be solved for the same
number (n+m) of unkowns: x1*, x2*,..., xn* and 1*, 2*,..., m*.
The asterisk on the xs na yje s indicate the values of the
xs and the s at the optimum value of the function y.
5. Application example
Consider the following example, which will be solved in a
sequence of steps.
1) Minimise f(x,y,z) = x2 + y2 + z2 ,
subject to
g(x,y,z) = x2 + y2 - z2 = 0 ,
h(x,y,z) = x 2z 3 = 0 .
2) Define the auxiliary function
L(x,y,z,,) = f(x,y,z) + g(x,y,z) + h (x,y,z)
= x2 + y2 + z2 + [x2 + y2 z2] + [x 2z 3] .
3) Solve the following system of algebraic equations.
L/x (x,y,z,,) = 2x + 2x + = 2 (1 + )x + = 0 , (5.1)
L/y (x,y,z,,) = 2x + 2y = 2(1 + )y = 0 , (5.2)
L/z (x,y,z,,) = 2z - 2z - 2 = 2(1-)z - 2 = 0 , (5.3)
L/ (x,y,z,,) = x2 + y2 z2 = 0 , (5.4)
L/ (x,y,z,,) = x 2z 3 = 0 . (5.5)
3) From equation (5.2), we have either y = 0 or = -1.
Case = -1. Then, the remaining equations reduce to
= 0 , (5.1)
4z - 2 = 0 , (5.3)
x2 + y2 z2 = 0 , (5.4)
x 2z 3 0 . (5.5)
Then, we have
(5.1) = 0, (5.3) z = 0, (5.5) x = 3, (5.4) y2 = -9.
In other words, y is complex, which is inacceptable, and
therefore case =-1 is discarded.
Case y = 0. Then, the remaining equations reduce to
2 (1 + ) x + , (5.1)
2 (1 - ) z - 2 , (5.3)
x2 z2 = 0 , (5.4)
x 2z 3 = 0 . (5.5)
From equation (5.4), we have
z = +/- x .
Subcase y = 0, z = x. Then, remaining equations reduce to
2 (1 + ) x + = 0 , (5.1)
2 (1 -) x - 2 = 0 , (5.3)
x 2x 3 = 0 . (5.4)
Then, the solution of this subcase may be written as
(x, y, z, , ) = (1, 0, -1, -3, -12).
Subcase y = 0, z = -x. Then, the remaining equations
reduce to
2 (1 + ) x + = 0 , (5.1)
-2 (1 - ) x - 2 = 0 , (5.3)
x + 2x 3 = 0 . (5.5)
Then the solution of this subcase may be written as
(x, y, z, , ) = (1, 0, -1, -1/3, -4/3).
4) We conclude that, there are two candidate solutions for
the maximum and the minimum: (3, 0, -3) and (1,0, -1),
correspoding to the objective function values of 32 and 2,
respectively; i.e., the maximum and the minimum values of
the objective function, respectively.
6. Lagrange multiplier method: sufficient conditions
For a function L (x1, x2,...,xn, 1, 2,..., m) to have a
constrained minimum (maximum)is that each root of the
polynomiaL zi, defined by the following determinantal be
positive (negative)
L11 z L12... L1n g11 g21... gm1
L21 L22-z... L2n, g12 g21...gm2
. =0 ,
(6.1)
Ln1 Ln2..., Lnn-z g1n g2n...gmn
g11 g12...g1n 0 0...0
g21 g22...g2n 0 0...0
gm1 gm2...gmn 0 0...0
where
Lij = 2L/xixj (x*, *) , (6.2)
gij = gi/xj(x*) . (6.3)
Equation (6.1), on expansion, leads to an (n-m) order
plynomial iz. If some of the roots are positive while the
others are negative, then the point x* is not am extreme
point.
7.1. Application example
Solve the following constrained maximisation problem.
Maximise f(x1, x2) = x12 x2 ,
subject to
2 x12 + 2x1 x2 = A0 = 24 .
1) The Lagrangian function is
L(x1, x2, ) = 2 x12 x2 + (2 x12 + 2 x1 x2 A0) .
2) The necessary conditions for the optimum of L may be
written as
L/x1 = 2 x1 x2 + 4 x1 + 2 x2 = 0 , (7.1)
L/x2 = x12 + 2 x1 = 0 , (7.2)
L/ = 2 x12 + 2 x1 x2 A0 . (7.3)
3) The solution of the algebraic system of equations (7.1)-
(7.3) may be written as
x1* = (A0/6)1/2 ,
x2* = (2A0/3)1/2 ,
* = -(A0/24) ,
4) Then the maximum value of f is given by
f* = (A03/54) ,
and putting A0 = 24, then the optimum solution may be
written as
x1* = 2, x2* = 4, * = -1, f* = 16 .
5) In order to verify that this s solution corresponds to the
maximum of f, we apply the sifficiency condition of Eq.
(6.1).
5) In this case, we have
L11 = 2L/x12 (x*, *) = 2 x2* + 4 * = 4 ,
L12 = 2L/x1x2 (x*, *) = L21 = 2 x1* + 2 * = 2 ,
L22 = 2L/x22 (x*, *) = 0 ,
g11 = g1/x1 (x*, *) = 4 x1* + 2 x2 = 16 ,
g12 = g1/x2 (x*, *) = 2 x1* .
6) Thus, Eq. (6.1) becomes
4 - z 2 16
2 0 z 4
16 4 0
i.e.,
2722 + 1923 = 0 ,
And
z = - 12/17 .
7) Since the value of z is negative, then the point (x 1*, x2*)
corresonds to a maximum of f.
References
[1] Rao, S.S. Engineering Optimization. Theory and Practice.
Fourth Edition, Johhn wiley, 2009.
[2] Hancock, H. theory of Maxima and Minima, FB&c Ltd.,
2016.
