Lecture 5 – Optimization with equality constraints

Constrained optimization
The idea of constrained optimisation is that the choice of one
variable often affects the amount of another variable that can
be used
Eg if a firm employs more labour, this may affect the amount of
capital it can afford to rent if it is restricted (constrained) by
how much it can spend on inputs
when a consumer maximizes utility (the ‘objective function’), x
and y must be affordable – income provides the constraint.
When a government sets expenditure levels it faces
constraints set by its income from taxes
Any optimal (eg profit maximising, cost minimising) quantities
obtained when under a constraint may be different from the
quantities that might be achieved if the agent were
unconstrained
2
 Binding and non-binding constraints

A constraint is binding if at the optimum the constraint function

holds with equality (sometimes called an equality constraint)
giving a boundary solution
somewhere on the constraint itself
Otherwise the constraint is non-binding or slack (sometimes
called an inequality constraint)
If the constraint is binding we can use the Lagrangean
technique

3
Often we can use our economic understanding to tell us if a
constraint is binding
– Example: a non-satiated consumer will always spend all
her income so the budget constraint will be satisfied with
equality
But in general we do not know whether a constraint will be
binding ( = , > or < )

In this case we use a technique which is related to the

Lagrangean, but which is slightly more general called linear
programming, or in the case of non-linear inequality constraints,
non-linear programming or Kuhn-Tucker programming after
its main inventors

4
 Objectives and constraints - example
A firm chooses output x to maximize a profit function
π = -x2 + 10x - 6
Because of a staff shortage, it cannot produce an output higher than x = 4
What are the objective and constraint functions?
The objective function: π = -x2 + 10x-6
The constraint: x ≤ 4

x
4
5
Note that without the constraint the optimum is x = 5
So the constraint is binding (but a constraint of, say, x ≤ 6 would not be)
– In general it is not always easy to see this

x
4 5
6
 Constrained optimization with two variables and one
constraint
The problem is:

To get the solution we have to write the Lagrangean:

where is a new variable

The candidates to the solution are the stationary points of
the lagrangean, i.e. all points that satisfy the following
system of equations:
y
Using the implicit function theorem

𝑓 1 ( 𝑥 ∗ , 𝑦 ∗) 𝑔1 ( 𝑥∗ , 𝑦 ∗ )
=
𝑓 2( 𝑥 , 𝑦 ) 𝑔2 ( 𝑥 , 𝑦 )
∗ ∗ ∗ ∗

𝑓 1( 𝑥∗ , 𝑦∗ ) 𝑓 2 ( 𝑥∗ , 𝑦∗ )
=
𝑔1 ( 𝑥 , 𝑦 ) 𝑔2 ( 𝑥∗ , 𝑦 ∗ )
∗ ∗

x
Using we get
Using we get
Moreover the solution has to satisfy the constraint
Then the solution has to satisfy the following three equations:

These equations are the derivatives of the Lagrangean

with respect to and to be zero

The first two are know as the first order conditions
Letand be continuously differentiable functions of two variables
defined on the set S,
be a number.
Suppose that is an interior point of S that solves the problem
subject to
Suppose also that

Then there is a unique number such thatis a stationary point of

the Lagrangean

That is, satisfies the first-order conditions.

and .
 Procedure for the solution

1. Find all stationary points of the Lagrangean

2. Find all points (x, y) that satisfy and
3. If the set S has boundary points, find all boundary points
that satisfy
4. The points you have found at which is largest are the
maximizers of

11
 Example

where and is defined for

1.
We solve:

The value of the objective function at the stationary point is:

2. then no values s.t.

3. The boundary points of the set on which the objective

function is defined is the set of points with either or . At
every such point the value of objectve function is 0

4. Then the solution of the problem is

13
 Interpretation of λ

the value of the Lagrange multiplier at the solution of the

problem is equal to the rate of change in the maximal value of
the objective function as the constraint is relaxed.
Example:

solution is so the maximized value of the objective function is .

Its derivative respect to is
Now consider the Lagrangean

The FOC is .
Then and satisfy FOC and the constraint. Note that is equal
to the derivative of the maximized value of the fct w.r.t.
Conditions under which a stationary point is a local optimum

Borderd Hessian of the Lagrangean

Suppose that it exists a value such that is a stationary
point of the Lagrangean.
To check if it is a local maximum
1) Compute the bordered Hessian at the values , i.e.
2) Compute its determinant, i.e.
3) If is a local maximizer

Note, if is a local minimizer

17
 Example

We simplify the problem using a log transformation

FOC are

The solution is

18
Borderd Hessian of the Lagrangean is

The determinant is , then the solution is a local maximizer

19
 Conditions under which a stationary point is a
global optimum
• Suppose that f and g are continuously differentiable
functions defined on an open convex subset S of two-
dimensional space and
• suppose that there exists a number λ* such that (x*, y*)
is an interior point of S that is a stationary point of the
Lagrangean
L(x, y) = f (x, y) − λ*(g(x, y) − c).
• Suppose further that g(x*, y*) = c.
• Then if L is concave then (x*, y*) solves the problem
maxx,y f (x, y) subject to g(x, y) = c
 Example
Consider the previous example

We found that the solution of the FOC is a local maximizer.

Is it a global maximizer?
For a global maximizer we need that lagrangean is concave

Given that constraint is linear we need to check the objective

function
The hessian of the objective function is

21
and
Then the hessian is negative definite, so the objective
function is strictly concave, and the point is a global
maximum.

22
 Optimization with equality constraints: n variables, m
constraints

Let f and g1, ..., gm be continuously differentiable functions of

n variables defined on the set S,
let cj for j = 1, ..., m be numbers, and suppose that x* is an
interior point of S that solves the problem:
maxx f (x) subject to gj(x) = cj for j = 1,...,m
Suppose also that (∂gj/∂xi)(x*) 0 for all i
Then there are unique numbers λ1, ..., λm such that x* is a
stationary point of the Lagrangean function L defined by
L(x) = f (x) − ∑j=1mλj(gj(x) − cj).

That is, x* satisfies the first-order conditions:

L'i(x*) = f i'(x*) − ∑j=1mλj(∂gj/∂xi)(x*) = 0 for i = 1,...,n.

In addition, gj(x*) = cj for j = 1, ..., m.

Conditions under which necessary conditions are sufficient

Suppose that f and gj for j = 1, ..., m are continuously

differentiable functions defined on an open convex
subset S of n-dimensional space and let x* ∈ S be an
interior stationary point of the Lagrangean:

L(x) = f (x) − ∑j=1mλ*j(gj(x) − cj).

Suppose further that gj(x*) = cj for j = 1, ..., m.

Then if L is concave then x* solves the constrained

maximization problem
 Interpretation of λ

Consider the problem

maxx f (x) subject to gj(x) = cj for j = 1,...,m,
Let x*(c) be the solution of this problem, where c is the vector
(c1, ..., cm) and let f *(c) = f (x*(c)).
Then we have
f *j'(c) = λj(c) for j = 1,...,m,
where λj is the value of the Lagrange multiplier on the jth
constraint at the solution of the problem.
 Interpretation of λ

The value of the Lagrange multiplier on the jth constraint at the

solution of the problem is equal to the rate of change in the
maximal value of the objective function as the jth constraint
is relaxed.
If the jth constraint arises because of a limit on the amount of
some resource, then we refer to λj(c) as the shadow price
of the jth resource.
 Quasi-concave functions

f(x″) = f(λx + (1-λ)x‘ ) ≥ min(f(x), f(x’) ) quasi concave

f(x″) = f(λx + (1-λ)x‘ ) > min(f(x), f(x’) ) strictly quasi
concave

Note these conditions hold even if f(x)=f(x’)

A concave function is also quasi-concave, but the opposite is
not true

If f(x) > f(x’) and f(λx + (1-λ)x‘ ) > f(x’) the function is
explicitly quasi concave

28
 f(x″) = f(λx + (1-λ)x’ ) ≤ max(f(x), f(x’) )
Note that a convex function is also quasi-convex
The bottom left picture shows that the opposite is not true

y
y

29
 The importance of concavity and quasi-concavity
Consider the problem
maxx f (x) subject to gj(x) = cj for j = 1,...,m
and let be a stationary point of the lagrangean

If
1. f(x) is explicitly quasi-concave
2. The constrained set is convex
3. then is a global maximum

If
4. f(x) is strictly quasi-concave
5. The constrained set is convex
6. then is the unique global maximum
30
 Convex sets.
A convex set, X, is such that for any two elements of the set, x
and x’ any convex combination of them is also a member of
the set.

x'

More formally, X is convex if for all x and x’ ε X, and 0 ≤λ≤1, x″ = λx + (1-λ)x‘ ε X.

Sometimes X is described as strictly convex if for any 0 < λ <1, x″ is in the interior of
X (i.e. not on the edges)
e.g. convex but not strictly convex

31
 Convex sets.
If, for any two points in the set S, the line segment
connecting these two points lie entirely in S, then S is a
convex set.

x2 p1x1+p2x2 ≤ m x2 U(x)≥ U

x1 x
32 1
Non-Convex sets.

U
x2

U(x)≤ U
x1
33
 A different definition of quasi concavity

Let be a multivariate function defined on the set .

is quasi concave if, for any number, the set of points for which
is convex.

For any real number , the set is called the upper level set of
for .

The multivariate function f defined on a convex set is

quasiconcave if every upper level set of is convex. (That is,
is convex for every value of .)
34
Examples
1. .
The upper level set of for is the set of pairs such that
Thus for it the set of point out of a disk of radius a, then
the upper level set is not convex

2. .
The upper level set of f for a is the set of pairs such
that , or.
Thus for the upper level set is empty
for it is the set of points inside a disk of radius .

35
 Checking quasi concavity
To determine whether a twice-differentiable function is quasi
concave or quasi convex, we can examine the determinants
of the bordered Hessians of the function, defined as follows:

We have to compute the determinants of the leading principal

minors
, ,……………

36
If is quasi concave then
if is even and
if is odd

If then if is even and

if is odd
then is strictly quasi concave

If is quasi convex then are negative

If then is strictly quasi convex

for all in the set where function is defined

37
 Envelope theorem: unconstrained problem
Let f(x,r) be a continuously differentiable function where x is
an n-vector of variables and r is a k-vector of parameters.

The maximal value of the function is given by f(x*(r), r)

where x*(r) is the vector of variables that maximize and that
are function of .

Note that we can write f(x*(r), r) as f *(r)

(because in this function only parameters appear)
If the solution of the maximization problem is a continuously
differentiable function of r then:

the change in the maximal value of the function as a

parameter changes is the change caused by the direct
impact of the parameter on the function, holding the value
of x fixed at its optimal value;
the indirect effect, resulting from the change in the optimal
value of x caused by a change in the parameter, is zero
Example

FOC is
then
and
The effect of a change of parameter c on the maximized
value is:

Consider the derivative of the objective function evaluated

at the solution

Evaluating it in we get

40
 Envelope theorem: constrained problems

Let f(x,r) be a continuously differentiable function where x is

an n-vector of variables and r is a k-vector of parameters.
The maximal value of the function is given by f(x*(r), r)
where x*(r) is the vector of variables x that maximize f and that
are function of r.
Note that we can write f(x*(r), r) as f *(r)
Then

where the function L is the Lagrangean of the problem

Example

we solve:

then and
The effect of a change of parameter c on the maximized
value is:

Consider the derivative of the Lagrangean evaluated at the

solution

42