Advanced Macroeconomic Theory

LECTURE NOTES
Nezih Guner
Department of Economics
Pennsylvania State University
March 2006

Walrasian (Competitive) Equilibrium

This course is an introduction to modern macroeconomic theory. Our main

emphasis will be the analysis of resource allocations in dynamic stochastic environments. We will start, however, with a simple environment, and analyze
a nite dimensional (with nite number of goods and agents) static exchange
economy. By an economy we mean a full characterization of: agentss endowments, agentss preferences, and a description of how these agents interact.
In an exchange economy people interact in the market place. They buy and
sell goods taking market prices as given. They do this in order to maximize
their utility. Their choices are constrained by their endowments. If we can nd
a set of prices such that: given these prices people behave optimally, and there
is no excess demand or excess supply of any good, we will say our economy is
in an equilibrium. 1
Consider an exchange economy with i = 1; :::; n consumers and j = 1; :::; m;
commodities. Each individual in this exchange economy is endowed with wi
units of goods, where wi is a m-dimensional vector, i.e.
wi = wi1 ; wi2 ; ::::; wim :
Hence, the total resources w in this economy is also a m-dimensional vector
given by
!
n
n
n
X
X
X
m
2
1
1
2
m
wi :
wi ; ::::;
wi ;
w = w ; w ; ::::; w
=
i=1

i=1

i=1

Individuals have preferences over these goods and will trade with each other to
maximize their well being. We will assume that:
1. Consumers preferences are representable by a utility function ui : X
m
! R; where X is the consumption set.
R+
2. ui 2 C 2 ; i.e. ui is such that it is continuos and the rst and the second
derivatives exist. Here C 2 represents the set of such functions.
3. Dui (x)
0 for all x 2 X; i.e. preferences are strictly monotonic. [A
weaker assumption would be monotonicity, i.e.Dui (x) > 0 for all x 2 X]:
4. ui is strictly concave, i.e. ui ( x + (1
)y) > ui (x) + (1
)ui (y)
for all x; y 2 X, x 6= y; and 2 (0; 1): [A weaker assumption would be
strict quasi concavity, i.e. ui ( x + (1
)y) > ui (y) for all x; y 2 X,
ui (x) ui (y); 2 (0; 1)].
m
5. wi 2 R++
; i = 1; :::; n; i.e. every agent is endowed with a positive amount
of each good.
1 There are several books that cover the materials presented here in a much more detail.
See, for example, Varian (1992), Farmer (1993), and Mas-Collel, Whinston and Green (1995).
General framework that is used here and in subsequent chapters was developed in Arrow
(1951), Debreu (1951), Arrow and Debreu (1954), and McKenzie (1954).

6. kDui (xk )k ! 1 as xk ! x where some component of x; xj = 0: However,

Dui (x) x is bounded for all x in any bounded subset of X:
Assumption #3 implies that people will be on their budget sets; assumption
#4 implies that indierence curves do not have at sections, and #6 implies
that in the case of two goods indierence curves are tangent to the axes, but
never become vertical or horizontal. Hence preferences are represented by nice
indierence curves as shown in Figure 1.

y
x + (1-)y

Figure 1: Strictly Convex Indierence Curves

Remark 1 Note that we adopt the following notation:

If x = (x1 ; :::::; xn ) and y = (y1 ; :::::; yn ) are n-dimensional vectors
x
x
x
x

=
>

y
y
y
y

means
means
means
means

xi = yi for all
xi yi for all
xi yi for all
xi > yi for all

i;
i;
i and x 6= y
i:

Hence,
n
R+
= fx 2 Rn jx

n
0g and R++
= fx 2 Rn jx

0g :

Dui (x) represents the vector of rst derivatives, i.e.

Dui (x) =

@ui (x)
@x1i

@ui (x)
@x2i

::::

@ui (x)
:
@xni

Finally, If x 2 Rn ; then the norm of x is dened as

n
X

kxk =

i=1

x2i

!1=2

Given a set of prices p = (p1 ; p2 ; :::; pm ); consumers in this economy try to

solve the following problem:
(1)
max ui (xi )
xi

subject to
p(xi

wi )

0; and xi 2 X:

(2)

Here xi is the consumption bundle that the consumer is choosing given a price
vector p and consumers endowment vector wi .
Hence, each individual tries to nd the best possible consumption bundle
xi = x1i ; x2i ; :::; xm
and is constrained by the value of his/her available rei
sources. The budget constraint can be written more explicitly as
pxi = p1 x1i + p2 x2i + ::: + pm xm
i

p1 wi1 + p2 wi2 + ::: + pm wim = pwi :

Remark 2 Note that since p and xi are vectors, we should write p0 xi (with p0
representing the transpose of p) to represent an inner product: Here I adopt a
simpler notation, and do not di erentiate between row and column vectors. It
is obvious that we mean an inner product of two vectors when we write pxi :
Given our assumptions, there is a unique interior solution to this problem.
This solution consist of m equations for each consumer:
x1i
x2i

xm
i

= wi1 + fi1 (p; pwi )

= wi2 + fi2 (p; pwi )
:
:
= wim + fim (p; pwi ):

Consumer chooses x1 given initial endowments. All we care, however, is functions fij representing excess demand or excess supply of each good for each
consumer. For each consumer then we will represent the optimal decisions as
xi = wi + fi (p);
where we drop wi from fi as an argument, since it is given, and its value is
known to each consumer.
4

Note that rst order conditions are necessary and su cient to characterize
xi (since we are maximizing a strictly concave function on a convex set, solution
exists and it is unique): Hence, xi is the solution to consumer i0 s problem if and
only if it satises
Dui (xi ) = i p;
(3)
and
pxi = pwi :

(4)

Here i is the Lagrange multiplier associated with the consumers budget constraint. Note that Dui (xi ) = i p is a set of m equations for each i, since there
is one derivative and one price for each good. Figure 2 represents the optimal
choice of a consumer for a two-good case.

Excess
demand
for good 2

x2

w2
x1

w1

Excess supply of good 1

Figure 2: Optimal Choice

Given this set-up what do we know about fi ? We can state the following:
1. It is continuos. This follows from our assumptions on preferences. Note
that in Figure 2, the optimal choice will change continuously as we change
the prices, since the indierence curves are strictly convex.
2. It is bounded below.
3. Its value is zero, i.e.
pfi (p) = 0:
This simply follows from monotonicity. With monotonicity people will be
on their budget sets.
5

4. It is homogenous of degree 0:
fi ( p) = fi (p); for all

> 0:

This implies that only the relative prices matter. Hence, if we multiply
all the prices with a constant the optimal choice does not change.
5. If pn ! p; where some pj = 0; then kfi (pn )k ! 1: This implies that if
the price of a good is zero, its demand will be innite.
Aggregate excess demand function is then given by
f (p) =

n
X

fi (p) =

f 1 (p) =

i=1

n
X

fi1 (p); f 2 (p) =

i=1

n
X

fi2 (p); :::::; f m (p) =

i=1

n
X

fim (p) :

i=1

Given the properties of fi ; it is immediate that f has the following properties

as well:
1. It is continuos.
2. It is bounded below.
3. Its value if zero, i.e.
pf (p) = 0:
4. It is homogenous of degree 0:
fi ( p) = f (p); for all

> 0:

5. If pn ! p; where some pj = 0; then kf (pn )k ! 1:

Since only the relative prices matter, we can normalize prices. This will
reduce the number of prices that we have to nd by one. One normalization is
to restrict prices to be in the unit simplex dened as:
8
9
m
<
=
X
m
j
pj = 1 :
(p) = p 2 R++
:
;
j=1

Since pf (p) = 0; if all but one market is in equilibrium (i.e. has excess
demand of zero), the remaining market must be in equilibrium as well. This is
called the Walras Law, and allows us to focus on m 1 markets rather than m.
Then, the fundamental question is if we can nd prices such that all markets
are in equilibrium, i.e.
f (p ) = 0:
Before going into the details of nding equilibrium prices, lets look at the
consumers problem in more detail. Following is the Lagrangian for the consumers problem.
6

L = ui (xi ) +

i (pwi

pxi ):

Then, we have the following set of FOCs:

Dui (xi ) =

i p:

Hence for each good j and for each individual i we have

@ui (xi )

@xji

ip

The optimal decision for a consumer is therefore characterized by the familiar

condition that for any two goods the marginal rate of substitution must be equal
to the ratio of prices, i.e.
@ui (xi )
@xk
i
@ui (xi )
@xh
i

pk
; for all i; k; h:
ph

We also know that since any two agents face the same prices, their marginal
rates of substitution must be same for any two goods, i.e.
@ui (xi )
@xk
i
@ui (xi )
@xh
i

@uj (xj )
@xk
j
@uj (xj )
@xh
j

; for all i; j; k; h:

We now introduce some denitions that will simplify our exposition.

Denition 3 A consumer is a pair ci

(ui ; wi ):

Denition 4 An exchange economy, E; is an n-tuple (c1 ; ::::; cn ):

Denition 5 An allocation is a vector of consumption bundles x = (x1 ; x2 ; :::; xn ) =
m
2
1
2
m
1
:
x11 ; x21 ; :::; xm
1 ; x2 ; x2 ; :::; x2 ; ::::; xn ; xn ; :::; xn
Pn
Pn
Denition 6 An allocation is feasible, if i=1 xi
i=1 wi :
Denition 7 A Walrasian (competitive) equilibrium for E is an allocation x
and a price vector p such that

1. The allocation xi solves agent i0 s problem given p ; i.e. xi maximizes ui

subject to p (xi wi ) 0 for all i:
2. Market clear, i.e.

n
X

xi

i=1

n
X
i=1

wi

(5)

w21
c2
w22
w21
c1
w11

Figure 3: Edgeworth Box

Excess demand for good 1

c2

x
Excess
demand
for good 2

Excess
supply
of good 2

c1
Excess supply of good 1

Figure 4: Walrasian Equilibrium

We will next look at a simple example of an economy with two agents and two
goods. When we have a 2 2 economy, we can represent the total resources of
this economy as a box (like in Figure 3). Such a box is called an Edgeworth Box.
Given an initial endowment point w, we know that the equilibrium consumptions
and prices are then given by the tangency of two indierence curves (such as
point x in Figure 4).
Example 8 Consider the following version of a nite dimensional exchange
economy with two goods and two agents. Utility functions take the form
(
1
x1i
x2i
; for i = 1
ui (xi ) =
;
1
x1i
x2i
; for i = 2
where ;
2 (0; 1): Endowments are given by w1 = (1; 0) and w2 = (0; 1):
To nd demand functions of agents 1 and 2 for goods 1 and 2 (as functions of
prices and endowments), we rst need to set-up the optimization problem for an
agent. For agent-1, demand for goods 1 and 2 are chosen to solve the following
problem:
1
max
x11
x21
;
1 ; x2
x
f 1 1g
subject to
p1 x11 + p2 x21 = p1 :
Let 1 be the Lagrange multiplier associated with this budget constraint. Then,
it is easy to see that FOCs are given by
x11

x21

1p

and
(1

) x11

x21

1p

After dividing these two equations and rearranging terms, we arrive at

x21 =

p1
p2

x11 ;

which is then can be substituted in the budget constraint:

p1 x11 + p2

p1
p2

x11 = p1 :

Hence,
x11 +

x11 = 1;

or
x11 = ; x21 = (1

10

p1
:
p2

Similarly, for the second consumer we will get

p2
p1

x12 =
Market clearing condition for good 1 is

x11 + x12 = 1;
hence
f 1 (p) = x11 + x12

1:

Then, f (p) = 0 implies

+
or

p2
=1
p1

p2
1
=
p1

As a normalization, let p2 = 1

p1 ; then
p1

p1

Our equilibrium prices are given by

p1 =

b
and p2 = 1
a+b

b
1 a+b b
1 a
=
=
:
a+b
1 a+b
1 a+b

Hence, optimal consumption decisions are:

x11 = ; x21 = (1
Now let, for example,

p1
= (1
p2

) 1 1 a+b
a = (1
1 a+b

= 0:5; then we have

f 1 (p1 )

= a+b
=

0:5

p1
p1
p1

p1
1 p1
= 0:5
p1
1 p1
= 0:5
p1
1
0:5 p
=
p1

1
0:5
1
p1

Figure 5 shows how f 1 (p1 ) looks like. Since it crosses the horizontal line, there
is an equilibrium price. Furthermore, this price is unique.
11

E x ces s Deman d Func tion

4

3.5

2.5

1.5

0.5

-0.5

-1
0

0.2

0.4

0.6

0.8

1.2

1.4

1.6

1.8

Figure 5: Excess Demand Function

What we would like to know is whether solutions to f (p) = 0 exists in our

more general set-up. The following proposition states that such solutions indeed
exist.
Theorem 9 If f : (p) ! (p) is a continuos function that satises the Walrass Law, pf (p) = 0; then there exists p in (p) such that f (p ) = 0:
Proof. See Varian (1992).
We now know that an equilibrium exists. Next, we would like to know if
the equilibrium satises some e ciency conditions. The condition we will use
is called Pareto optimality.
Denition 10 An allocation is Pareto optimal if it is feasible and there is no
other feasible allocation x0 such that uj (x0j )
uj (xj ) for all j and uj (x0j ) >
uj (xj ) for at least one j:
We will rst show that every competitive equilibrium is Pareto optimal.
Theorem 11 (First Welfare Theorem) Every competitive equilibrium is Pareto
Optimal.
Proof. Suppose x is a competitive equilibrium allocation with price vector p:
Suppose that the statement is not true, i.e. there is another feasible allocation
12

x0 that is preferred to x by at least one consumer. Since x0 is feasible, we know

that
n
X

n
X

x0i

i=1

wi :

i=1

If we write this explicitly we have:

x01 + x02 + :::: + xn

w10 + w20 + :::: + wn :

Note that this is a set of m equations:

01
01
x01
1 + x2 + ::: + xn
02
02
x02
1 + x2 + ::: + xn

w101 + w201 + ::: + wn01 ;

w102 + w202 + ::: + wn02 ;
:
:
w10m + w20m + ::: + wn0m :

0m
0m
x0m
1 + x2 + ::: + xn

(6)

For the consumer who strictly prefers x0 ; it must be outside his/her budget set
with p: Otherwise he/she would choose it in the rst place. Then,
px0j > pwj ;
or
2 02
m 0m
1 1
2 2
m m
p1 x01
j + p xj + ::: + p xj > p wj + p wj + ::: + p wj :

For all other consumers i 6= j; we have

px0i = pwi :
Therefore,

n
X

px0i >

i=1

If we write this explicitly we have:

n
X

pwi :

i=1

px01 + px02 + ::: + px0n > pw1 + pw2 + ::: + pwn ;

or
2 02
m 0m
p1 x01
1 + p x1 + ::: + p x1 +
2 02
m 0m
1 01
2 02
m 0m
p1 x01
2 + p x2 + ::: + p x2 + ::: + p xn + p xn + ::: + p xn
> p1 w11 + p2 w12 + ::: + pm w1m +
p1 w21 + p2 w22 + ::: + pm w2m + ::: + p1 wn1 + p2 wn2 + ::: + pm wnm :

rearranging terms we get:

01
01
2
02
02
02
p1 x01
1 + x2 + ::: + xn + p x1 + x2 + ::: + xn + :::
0m
0m
pm [x0m
1 + x2 + ::: + xn ]
> p1 w11 + w21 + ::: + wn1 + p2 w12 + w22 + ::: + wn2 +
pm [w1m + w2m + ::: + wnm ] :

13

(7)

But then we have a contradiction, since if we multiply each of m equations in

(6) with the appropriate price and then sum the right and left side of (6), we
get the opposite of (7).
Next we would like to know if every Pareto optimal allocation is also a
competitive equilibrium. In order to show this we rst need to characterize the
set of Pareto optimal allocations. Remember that an allocation x is Pareto
optimal if it is feasible and if there is no way to reallocate x to make at least
one person strictly better o without making anyone else worse o. Our claim
is that any such allocation must be a solution to the following problem (you can
try to show why this claim is correct):
max
x

subject to

n
X

i ui (xi ); with

i=1

n
X

n
X

= 1:

i=1

n
X

wi

i=1

xi and xi

0:

i=1

This is a planners problem with representing the weights of dierent agents

in planners objective. Note that we have a set of Pareto optimal allocations for
each possible value of : The solutions to this planning problem is characterized
by two sets of equations:
(8)
i Dui (xi ) = ;
where

is the Lagrange multiplier for the planners resource constraint; and

n
X

wi =

n
X

xi ;

(9)

i=1

i=1

which is the feasibility constraint.

Now lets go back to the competitive allocations. Competitive allocations are
characterized by
Dui (xi ) =
where

i p;

(10)

is the Lagrange multiplier for agent i and p is the set of prices; by

p(wi

xi ) = 0;

(11)

which is the budget constraint for individuals i; and by

n
X

wi =

i=1

n
X

xi ;

(12)

i=1

which is the feasibility constraint.

Note that (8) is a set of n m equations, and that (9) is a set of m equations.
Similarly, (10) is a set of m n equations and (12) is a set of m equations.
Furthermore, (9) and (12) are identical. If we set p = (i.e. set prices such
14

that they are equal to the Lagrange multiplier for the constraint for each good),
and i = 1i ; then (8) and (10) are identical as well. Then, whether a Pareto
optimal allocation can be decentralized comes down to whether the social planer
can make sure at prices ; the planners allocation is feasible for each consumer.
This might require redistribution of resources dened by the following transfer
functions
wi ) :
i ( ) = (xi
Theorem 12 (Second Welfare Theorem) Every Pareto Optimal allocations can
be decentralized as a competitive equilibrium, i.e. given a Pareto Optimal allocation x; we can nd a price vector p and transfers i such that: given the initial
endowments these transfers x is a competitive allocation with prices p:
Before we analyze some particular examples, note that the rst order conditions for the planners problem imply
@ui (xi )
i

@xji

for all i; j:

Then, we have
@ui (xi )
@xk
i
@ui (xi )
@xh
i

@uj (xj )
@xk
j
@uj (xj )
@xh
j

and in a Pareto Optimal allocation the marginal rate of substitution between

any two goods must be the same for any two consumers. Figure 6 illustrates the
set of Pareto Optimal allocations for a 2 2 economy. Figure 7 shows the basic
idea behind the second welfare theorem. Given any Pareto optimal allocation
w ; we can nd the prices that support this allocations and move from the
original endowment point w to w using transfers.
Example 13 Consider again the following version of a nite dimensional exchange economy with two goods and two agents. Utility functions take the form
(
1
x1i
x2i
; for i = 1
ui (xi ) =
;
1
1
2
xi
xi
; for i = 2
where ; 2 (0; 1): Endowments are given by wi = (ai ; bi ) for i = 1; 2; with
ai > 0 and bi > 0: To nd demand functions of agents 1 and 2 for goods 1 and 2
(as functions of prices and endowments), we rst need to set-up the optimization
problem for an agent. For agent-1, demand for goods 1 and 2 are chosen to solve
the following problem:
1
max
x11
x21
;
1
2
fx1 ; x1 g
subject to
p1 x11 + p2 x21 = p1 a1 + p2 b1 :

15

c2

c1

Figure 6: Pareto Optimal Allocations

16

c2

w*
w

c1

Figure 7: Second Welfare Theorem

17

Let be the Lagrange multiplier associated with this budget constraint. Then,
it is easy to see that FOCs are given by
x11

x21

= p1 ;

and
) x11

(1

= p2 :

x21

After dividing these two equations you can arrive at

x21 =

p1 1
p2

x11 ;

which then can be used to nd x11 and x21 by substituting it in the budget constraint:
p1 a1 + p2 b1
p1 a1 + p2 b1
; x21 = (1
)
:
x11 =
1
p
p2
Similarly for the second consumer we will get
x12 =

p1 a2 + p2 b2
; x22 = (1
p1

p1 a2 + p2 b2
:
p2

Market clearing conditions are

p1 a1 + p2 b1
p1 a2 + p2 b2
+
= a1 + a2 ;
p1
p1
and
(1

p1 a1 + p2 b1
+ (1
p2

p1 a2 + p2 b2
= b1 + b 2 :
p2

If we use one of these equations, and normalize prices by p2 = 1

get
b1 + b2
p1 =
:
(1
)a1 + b1 + (1
)a2 + b2

p1 ; we will

In order to characterize Pareto optimal allocations rst note that

M RS1 =

x21
; and M RS2 =
x11
1

x22
:
x12

Since
x12 = a1 + a2

x11 and x22 = b1 + b2

x21 ;

we can dene Pareto optimal allocations as those allocations that satisfy

1

x21
=
x11
1

b1 + b2 x21
:
a1 + a2 x11

(13)

Note that equation (13) provides a complete characterization of all Pareto optimal allocations. Any division of the total output between two agents that satises
this is a Pareto optimal allocations.
18

Example 14 Consider a 2x2 exchange economy. There are 2 goods and 2 individuals, and preferences and initial endowments are as follows:
u1 (x11 ; x21 ) = a log(x11 ) + (1

a) log(x21 ); a 2 (0; 1);

u2 (x12 ; x22 ) = b log(x12 ) + (1

b) log(x22 ); b 2 (0; 1);

and
w1 = (w11 ; w12 ) = (1; 1); w2 = (w21 ; w22 ) = (2; 2):
Then, individual 1s problem is
u1 (x11 ; x21 );

max

x11 ;x21

subject to
p1 x11 + p2 x21

p1 w11 + p2 w12 ; x11

0; x21

0:

Langrangian for this problem is:

L(x11 ; x21 ) = a log(x11 ) + (1

a) log(x21 )

[p1 x11 + p2 x21

p1 w11

p2 w12 ];

and the rst order conditions (FOCs) are:

a
x11

x11 :
and

(1

x21 :

a)
x21

1 1

=)

1 2

=)

p =0

p =0

a
=
x11
(1

a)
x21

1 1

p ;

1 2

p :

Hence, we get
x21 =

p1 1 (1 a)
x
:
p2 1 a

substituting into budget constraint, we have

a(p1 w11 + p2 w12 )
a(p1 + p2 )
=
;
p1
p1

x11 =
and
x21 =

(1

a)(p1 w11 + p2 w12 )

(1
=
p2

a)(p1 + p2 )
:
p2

Similarly the second individuals Langrangian is:

L(x12 ; x22 ) = b log(x12 ) + (1
with FOCs:

b
x12

x12 :
and
x22 :

(1

b)
x22

b) log(x22 )

[p1 x12 + p2 x22

2 1

=)

2 2

=)

p =0

p =0

19

b
=
x12
(1

b)
x22

p1 w21

2 1

p ;

2 2

p :

p2 w22 ];

These imply
x22 =

p1 1 (1 b)
x
:
p2 2 b

Again substituting into the budget constraint, we get:

x12 =
and
x22 =

b(2p1 + 2p2 )
b(p1 w21 + p2 w22 )
=
;
1
p
p1
b)(p1 w21 + p2 w22 )
(1
=
2
p

(1

b)(2p1 + 2p2 )
:
p2

Then, the aggregate excess demand for good 1 is:

x11 + x12

3=

(a + 2b)(p1 + p2 )
p1

3:

If we let p1 + p2 = 1; equate the excess demand to zero, and solve two equations
simultaneously, we get
p1 =
and

a + 2b
;
3

(a + 2b)
:
3
Then, plugging these into demand functions of each individual gives us the competitive allocations:
p2 =

x11 =

3(1 a)
3a
and x21 =
;
a + 2b
3 (a + 2b)

x12 =

6b
6(1 b)
and x22 =
:
a + 2b
3 (a + 2b)

and

The social planners problem is given by

max(x1 ;x2 ;x1 ;x2 ) [a log(x11 ) + (1
1 1 2 2

a) log(x21 )] + (1

)[b log(x12 ) + (1

b) log(x22 )];

subject to:
x11 + x12 = 3

x21 + x22 = 3;

and
x11

0;

x21

x12

0;

0;

x22

0:

Then the Langrangian for this problem is:

L(x11 ; x21 ; x12 ; x22 )

[a log(x11 ) + (1 a) log(x21 )] + (1
1 1
2 2
[x1 + x12 3]
[x1 + x22 3];
20

)[b log(x12 ) + (1

b) log(x22 )]

and the FOCs are:

a
=
x11

x11 :

(1 a)
=
x21

x21 :

(1

x12 :

)b
x12

and

(1

x22 :

)(1
x22

=
b)

;
2

Therefore we have
a
(1
=
x11
3
and

)b
;
x11

a(1 a)
(1
)(1 b)
=
:
2
x1
3 x21

Hence,
x11

x12

3 a
; x2 =
)b + a 1
3(1
)b
; x2 =
(1
)b + a 2
(1

3 (1 a)
a) + (1
)(1
3(1
)(1 b)
(1 a) + (1
)(1
(1

b)
b)

;
:

By substituting these into FOCs of the problem, we can nd the values of the
Lagrange multipliers in the planners problem as
1

(1
(1

)b + a
; and
3
a) + (1
)(1
3

b)

Note that it is very easy to show that when

= 13 , the solutions to social
planners problem will be same as the Walrasian equilibrium allocations and 1
and 2 will be same as Walrasian prices. Hence, our competitive equilibrium is
Pareto optimal.
We also know that the Lagrange multipliers for the social planners problem
can be used as competitive equilibrium prices to decentralized any Pareto optimal
allocation. So for any , we should be able to nd prices and transfers that
decentralize a Pareto optimal allocation as a competitive equilibrium.
Let, for example, = 12 : Then, Pareto optimal allocations are:
x11 =

3a
3(1 a)
and x21 =
b+a
(1 a) + (1

21

b)

(14)

for consumer 1, and

x12 =

3b
3(1 b)
and x22 =
b+a
(1 a) + (1

b)

(15)

for consumer 2. The Lagrange multipliers in the planners problem are:

b+a
(1 a) + (1 b)
and p2 =
:
(16)
6
6
Note that above allocations are not the same one we got when = 1/3. Our
claim is that if prices are given by (16); and people can a ord to choose allocations dened in (14) and (15), then they will indeed do so as their optimal
choices. We need rst transfer goods between individuals and change their initial
endowments to above allocations.
The transfer functions (in terms of ) can be dened as:
p1 =

tr1 ( ) =

3 a
)b + a

(1

1 ;

3 (1 a)
a) + (1
)(1

b)

3(1
)(1 b)
(1 a) + (1
)(1

b)

(1

and
tr2 ( ) =

3(1
(1

)b
)b + a

2 ;

First one is the transfer to individual one and the one is transfer to individual
2. Notice that these terms should sum up to zero. One can plug 1/2 for and
nd the transfers for the above case.
Hence, we can now answer the following question. Suppose prices are given
by (16), and individual 1s endowments are:
1+
and
1+

1
2 (1

3 21 a
1
1
2 )b + 2 a

(1

3 12 (1 a)
a) + 12 (1

b)

(1

3a
of good 1,
b+a
3(1 a)
a) + (1

b)

of good 2,

what would be his/her choice in a competitive economy?

We already know that the consumer 1s optimal choice for a given set of
prices is
a(p1 w11 + p2 w12 )
x11 =
:
p1
Then, we have, for example:
x11

3a
a( b+a
6 b+a +

= a
= a

(1 a)+(1 b)
3(1 a)
6
(1 a)+(1 b) )
3a
b+a

3a 3(1 a)
6
+
6
6
a+b
1
6
3a
=
;
2 a+b
a+b
22

which is exactly the planners allocation of good 1 to individual 1.

References
[1] Arrow, K. J. The Role of Securities in the Optimal Allocation of RiskBearing," Review of Economic Studies, 31, 91-96, 1964, (translation of
original 1953 article from French).
[2] Arrow, K. J. G. Debreu, Existence of An Equilibrium For A Competitive
Economy," Econometrica, 22265-290, 1954.
[3] Debreu, G. Theory of Value, Wiley, 1959.
[4] Farmer, Roger E. A. General Equilibrium under Certainty (Chapter 4)
The Macroeconomics of Self-Fullling Prophecies, The MIT Press, Cambridge, MA, 1993.
[5] Mass-Colell, Michael D. Whinston, and Jerry R. Green, Microeconomic Theory, Oxford University Press, 1995.
[6] McKenzie, "On Equilibrium in Grahams Model of World Trade and Other
Competitive Systems, Econometrica, 22, 147-61, 1954.
[7] Varian, Hal R. Exchange (Chapter 17) in Microeconomic Analysis, 3rd
Edition, W.W. Norton and Company, New York, 1992.

23

Exchange Economies with Innitely-Lived Agents

In our static Walrasian economy agents live for a single period. In this section
we will analyze model economies where they live forever. We will assume that
time is discrete and the horizon is innite, t = 0; 1; 2; :::::As in the previous
section there is a nite number of agents indexed by i = 1; :::n: We will assume,
however, there is one consumption good per period, i.e. m = 1 (see Kehoe
(1989) for an analysis with m > 1): This consumption goods is not storable.
Agents have deterministic endowment streams. The endowment stream of agent
1
i is denoted by wi = wti t=0 :
1
Let cit be the consumption of agent i at time t; and let ci = cit t=0 be a
consumption sequence. Agents preferences are given by
U (ci ) =

1
X

t
i
i ui (ct );

t=0

where i 2 (0; 1) is the discount factor of individual i: Hence we assume that

lifetime utility U (ci ) is time separable. Time-t consumption only aects utility
at time-t: We assume that ui 2 C 2 ; strictly increasing, strictly concave, and
satises limc!0 u0 (c) = 1:
We still have to specify how markets work. We will assume that there is a
market at time 0 where agents can buy and sell goods of dierent time periods.
There is a price for every periods good. Hence, each agent can sell his current
and future endowments in this grand time-0 market and buy any amount of
current and future goods he/she wants. The consumer therefore faces a single
budget constraint:
1
1
X
X
pt cit
pt wti :
t=0

t=0

After consumers choose their consumption sequences at this time zero market,
time does not play an explicit role. As time passes transactions (exchange of
goods) that were agreed at time zero take place. We assume that all contracts
that are agreed at time 0 are honored. We call this market arrangements ArrowDebreu markets. We will normalize prices and set p0 = 1:
Denition 15 An Arrow-Debreu equilibrium is a sequence of allocations ci =
1
1
cit t=0 for each i; and a sequence of prices p = fpt gt=0 such that
1. Given p, ci solves agent i0 s maximization problem for each i:
max
i
c

subject to

1
X

1
X

t
i
i ui (ct );

(17)

t=0

pt cit

t=0

1
X
t=0

24

pt wti :

(18)

2. And markets clear

n
X

n
X

cit

i=1

wti for each t:

(19)

i=1

Remark 16 It is immediate
that for consumers maximization problem to exist
P1
it must be case that
p
wti is nite. Otherwise, there is no well dened
t
t=0
solution for the maximization problem.
P1
i
Remark 17 If
t=0 pt wt is nite, then the value of aggregate endowment is
also nite in an Arrow-Debreu equilibrium, since
!
!
1
n
n
1
X
X
X
X
pt
wti =
pt wti :
t=0

i=1

i=1

t=1

If the societys resources are nite in an Arrow-Debreu equilibrium then it

is immediate that any Arrow-Debreu equilibrium is Pareto Optimal.
Proposition 18 Any Arrow-Debreu equilibrium is Pareto Optimal.
Proof. Suppose given an Arrow-Debreu equilibrium allocations cit and prices p;
there is another allocation e
ct that Pareto dominates cit : Then it must be the case
that
1
1
X
X
t
t
i
i
ct )
i ui (e
i ui (ct ) for all i;
t=0

t=0

and there exists a j such that

1
X

t
cjt )
j uj (e

>

t=0

j
t
j uj (ct ):

t=0

Fist note that e

ct is feasible, i.e.
n
X
i=1

e
cjt ,

1
X

e
cit

n
X

wti for all t:

i=1

it must be outside of her budget set given prices p: For

Since agent j prefers
everyone else e
cit must be feasible. Hence,
!
!
!
n
1
n
1
1
n
X
X
X
X
X
X
i
i
i
pt e
ct >
pt wt =
pt
wt ;
i=1

t=1

i=1

t=1

t=0

i=1

which contradicts the fact that e

cit is feasible, since feasibility implies
!
!
!
n
1
1
n
1
n
X
X
X
X
X
X
i
i
i
pt e
ct =
pt
e
ct
pt
wt :
i=1

t=1

t=0

i=1

t=0

i=1

Remark 19 Note that this proof follows exactly the same steps as the proof of
First Welfare Theorem in the last section.
25

How do equilibrium allocations look like? They are characterized by three

sets of equations. The FOCs for the consumer
i
t @ui (ct )
i
i
@ct

pt for each i and each t:

(20)

Equilibrium allocations must also satisfy the budget constraint of each individual
1
1
X
X
pt cit =
pt wti :
t=0

t=0

There is one budget constraint per consumer. Finally, they have to be feasible
n
X

n
X

cit

i=1

wti :

i=1

This feasibility constraint has to hold every period, since the good is perishable.
What does equation (20) tell us about consumer behavior? Note that for
any consumer i; and any two time periods t and t + 1; we have
i
t @ui (ct )
i @cit
i
t+1 @ui (ct+1 )
i
@cit+1

pt
pt+1

Hence,
@ui (cit )
=
@cijt

pt @ui (cit+1 )
:
pt+1 @cijt+1

(21)

This equation, which will appear many times in this course, is the intertemporal
optimization condition for the consumer.
If the consumer allocates his/her resources optimally, the cost of reducing
@ui (cit )
time-t consumption today, @c
; must be equal to the benet of increasing
i
t

time-t + 1 consumption,

@ui (cit+1 )
;
@cit+1

after taking into account the discount factor

pt
; and the relative value of goods between two periods, pt+1
: Discounting makes
pt
future consumption less valuable. If pt+1 is high, benet of moving resources
pt
from t to t + 1 is large, and if pt+1
is low, then the benet of moving resources
from t to t + 1 is small.
Given FOCs for individual i and j we also know that their consumption at
time period will be related by
i

t @ui (ct )
i @cit
t
j

@ui (cjt )
@cjt

Again we can characterize the set of Pareto optimal allocations as solutions

to the following planners problem
n
X
max
1
fcit gt=0 i=1

26

i
i ui (ct )

subject to

n
X

cit =

i=1

n
X

wti ; t = 0; 1; 2::::

i=1

The solution to this problem is characterized by the following FOCs

i

@ui (cit )
=
@cit

t;

for i = 1; 2; :::; n and t = 0; 1; ::::

where t is the Lagrange multiplier on the time-t resource constraint.

Given ; allocations that solves the planners problem is a Pareto optimal
allocation. In order to decentralize the Pareto optimal allocations we use the
Lagrange multipliers t ( ) as prices and transfer resources among consumers
according to:
1
X
i
(
)
=
wti ;
i
t ( ) ct ( )
t=0

cit (

where
) is a Pareto optimal allocation of goods.
We also know that if we can nd an
; such that i ( ) = 0 for all i;
then the t ( ) are the Arrrow Debreu prices and allocations cit ( ) are ArrowDebreu allocations for this economy. The following example illustrates how this
approach can be an easier way to nd Arrow Debreu allocations than solving
the Arrow Debreu equilibrium directly. This method of computing competitive
equilibrium was formulated by Negishi (1960).
Example 20 Let n = 2 and
u1 (ct ) = u2 (ct ) = log(ct );
and
wt1 = wt2 = 1 for all t:
Also let the consumers di er in their discount factors with 1 < 2 : An Arrow
Debreu equilibrium for this economy is characterized by the following tree sets
of equations:
t 1
i
pt ; for i = 1; 2 and t = 0; 1; 2; :::;
i i =
ct
1
X

pt cit =

t=0

and

1
X

pt for i = 1; 2;

t=0

c1t + c2t = 2; for t = 0; 1; 2; :::::

The Pareto optimal allocations are the solutions to the following maximization
problem:
1
1
X
X
t
t
1
2
max
log(c
)
+
1
2
1
t
2 log(ct );
1 2 1
fct ;ct gt=0

t=0

t=0

27

subject to
c1t + c2t = 2 for all t:
Then the Pareto optimal allocations must satisfy the following conditions
t 1
i i i
ct

for i = 1; 2;

and
c1t + c2t = 2 for all t:
Using the FOCs for i = 1; 2 we have
t 1
1 1 1
ct

t 1
2 2 2:
ct

Hence,
t
2 1
t ct .
1

c2t =

Using the resource constraint, we have

2

c1t 1 +

t
2
t
1

= 2:

Then the Pareto optimal allocations are given by

cit =

t
i i

2
t
1 1

t
2 2

for i = 1; 2;

(22)

and the Lagrange multipliers for the planners resource constraint are given by
t

t
1 1

+
2

t
2 2

(23)

Note that to nd t ; we simply used the FOC, i ti c1i = t : Then we can decent
tralize the allocations in (22) using prices in (23) and the following transfers:
1(

1;

2) =

1
X

1
t (ct

1) =

t=0

and
2( 1;

2)

1
X

2
t (ct

1) =

t=0

28

1
;
2

1
:
2

1
2

Note that

is calculated as:

1(

1;

2)

=
=
=
=

1
X

1
t (ct

t=0
1
X

t
1 1

t=0
1
X

t
1 1

t=0
1
X

1)
+
2

t
2 2

+
2

t
2 2

t
1 1

Then, the values of

and

1)

t
2 2

t
1 1

t
1 1

t
1 1

t
2 2

t
2 2

2
1

2(1
2

t
1 1

t
2 2

t=0

t
1 1

2
1)

2(1

that makes
1

2(1

1)

2)

1( 1;

2)

2(1

2)

equal to 0 are

Now rst note that if 1 ( 1 ; 2 ) = 0; then 2 ( 1 ; 2 ) = 0 as well. Second, we

can normalize one of the weights to 1, since in the planners problem all that
matters if the relative weights. Then, the set of weights that will give us the
competitive allocations are
2

= 1 and

1
1

Hence, our claim is that the competitive allocations are

c1t

1
1

1
1

t
1

and
c2t =

2
1
1

t
1

1
2

1
2

t
2
t
1

t
2

t
2

To prove our claim lets go back to competitive allocations. The FOCs for the
consumer i was
t 1
i
pt :
i i =
ct
Lets focus on i = 1; using the FOC for t and t + 1; we get
t 1
1 c1t
t+1 1
1
c1

t+1

29

pt
:
pt+1

Hence,
c1t+1 =

pt
pt+1

1
1 ct :

We can use this relation to write

c1t =

p0
pt

t 1
1 c0 :

Now using this rule in consumer budget constraint

p0 c10 + p1 c11 + ::::: = p1 + p2 + :::::;
we get
c10 p0 +

1 p0

2
1

+ :::: = [p0 + p1 + :::::] ;

or
c10 p0 1 +
Then,
c10
and
c1t =

2
1

+ ::::: = [p0 + p1 + :::::] :

P1

p0
pt

t=0 pt
p0 1 1
1

t 1
1 c0

(1

1)

P1

t=0

pt

p0
t (1
1

p0
pt

1)

P1

t=0

pt

p0

Since a similar rule also determines the consumption behavior for the consumer
2, we have the following market clearing condition for t = 0;
P1
P1
(1
(1
1)
2)
t=0 pt
t=0 pt
+
= 2:
p0
p0
Therefore,

P1

P1
pt + (1
2)
t=0 pt
:
2
Indeed, using the market clearing condition for any period, we can get
P1
t
t
2)
1 ) + 2 (1
1 (1
t=0 pt
pt =
:
2
p0 =

(1

1)

Then,
c10 =

(1

1)

t=0

P1

t=0

pt

p0

and
c1t

2 11
1
1

1
2

2(1
1)
)
+
(1
1

(1
1
2

t
1

t
1

t
2

2)

It is then trivial to check that these are the allocations we nd by solving the
planners problem and setting the transfers to zero. In this example, it is much
easier to calculate Pareto optimal allocations than competitive (Arrow-Debreu)
allocations.
30

Example 21 Consider a simple exchange economy with two consumers, indexed by i = 1; 2; who live forever, and with one perishable consumption good.
Time is discreet and indexed by t = 0; 1; :::::. Each consumer values sequences
1
of consumption goods, ci = cit t=1 according to
U (ci ) =

1
X

ln(cit );

t=0

with

2 (0; 1): The endowment processes are given by

wt1 = 2 and wt2 = 0 if t is even
:
wt1 = 0 and wt2 = 1 if t is odd

wti =

An Arrow-Debreu equilibrium for this economy consists of allocations c1t ; c2t

1
and prices fpt gt=0 such that:

1
t=0

1. Given fpt gt=0 ; the allocations solve:

1
X
max
1
fcit gt=0 t=0

subject to

1
X

pt cit =

t=0

ln(cit );

1
X

pt wti ;

t=0

for i = 1; 2; (note that since we are in an Arrow-Debreu world, agents face

one single budget constraint).
2. Markets clear, i.e.
c1t + c2t = wt1 + wt2 ;
for all t; (note that since good is perishable, the market clearing condition
has to hold in very period).
1

An allocation c1t ; c2t t=0 is Pareto optimal, if it is feasible and there is no

1
other feasible allocation e
c1t ; e
c2t t=0 such that 1) u(e
ci ) u(ci ) for i = 1; 2; and
1
2) u(e
ci ) > u(ci ) for at least one i = 1; 2; (note that an allocation c1t ; c2t t=0 is
feasible if cit 0 for all t and i = 1; 2; and c1t + c2t = wt1 + wt2 for all t):
For this example we can write the social planners problem as
max
1 2 1

fct ;ct gt=0

1
X

ln(c1t ) + (1

t=0

subject to
c1t + c2t = 2 if t is even,
and
c1t + c2t = 1 if t is odd.
31

) ln(c2t ) ;

FOCs for this problem will result in

t

c1t

(1

c2t

(note that given resources in any period, this condition equates MRSs for agent
1 and 2). Hence,
(1
) 1
c2t =
ct :
You can now use the budget constraint to arrive at
c1t = 2 ; and c2t = 2

if t is even,

and
c1t = ; and c2t = 1

if t is odd.

It is also easy to show that Lagrange multiplier is

t
t

if t is even and

We can then use prices pt =

transfer functions as

if t is odd.

if t is even, and pt =

t1 ( ) =

1
X

pt (cit ( )

if t is odd, and nd

wti );

t=0

that can be used to decentralize any Pareto optimal allocation.

2.1

Sequential Equilibrium

Our analysis above was built on Arrow-Debreu markets where all trade take
place at a time-0 market. There were no other market in any other period.
Now we look another possible market arrangement where we have a market
each period. Suppose now trades take place in spot markets that open every
period. Hence, at time t; agents only trade time-t goods in a spot market. Let
qt be the price of the good in this time-t market. If agents can only trade time-t
good at time t; and there no credit arrangements, then this economy would look
like a sequence of static exchange economies.
With spot markets we need a credit mechanism that will allow agents to
move their resources between periods. Therefore, we will assume that there is a
one period credit market that works as follows: each period agents can borrow
et = 1 + ret be the gross interest
or lend in this one period credit market. Let R
rate on time-t borrowing (lending): How does this credit system work? One can
think of a central credit agency that keeps track of people who borrow and lend.
If you want to lend you bring your goods to the agency which gives it to other
people. Next period you can go and receive your goods back (plus the interest)
or bring your goods to pay your debt. We assume everybody honors his/her
contract and there is perfect record keeping.
32

Then, each individual will have a sequence of budget constraints (rather than
a single one as it was the case with Arrow-Debreu markets):
q0 ci0 + l0i
q1 ci1 + l1i

= q0 w0i
= q1 w1i + (1 + re0 )l0i
::::
= qt wti + (1 + ret 1 )lti
::::

qt cit + lti

(24)

Hence, given a sequence of prices, fpt gt=0 and fe

rt gt=0 ; the agent i0 s problem is
max
i
c

1
X

t
i
i ui (ct );

t=0

subject to the sequence of budget constraints dened above.

In the Arrow-Debreu equilibrium, the value of agents resources had to be
nite to make sure we had a well dened solution to agents maximization
problem. How can we make sure that agents resources are nite in this set-up?
Lets begin by rearranging the equations in (24).
Note that for t = 1; we have
q1 ci1 + l1i
l1i

= q1 w1i + (1 + re0 ) q0 w0i

= q1 w1i + (1 + re0 )q0 w0i

q0 ci0
q1 ci1

Similarly for t = 2; we get

q2 ci2 + l2i
l2i

(1 + re0 )q0 ci0 :

= q2 w2i + (1 + re1 )l1i

= q2 w2i + (1 + re1 )q1 w1i + (1 + re1 )(1 + re0 )q0 w0i
q2 ci2 (1 + re1 )q1 ci1 (1 + re1 )(1 + re0 )q0 ci0 :

Working our way to time t we get

qt cit + lti
lti

= qt wti + (1 + ret

= qt wti + (1 + ret

i
1 )lt 1

i
1 )qt 1 wt 1

+ (1 + ret

1 )(1

::::: + (1 + ret 1 ):::::(1 + re0 )q0i w0i

qt cit + (1 + ret 1 )qt 1 cit 1 + (1 + ret 1 )(1
::::: (1 + ret 1 ):::::(1 + re0 )q0i ci0 :

Multiplying both side by

lti
(1 + ret 1 ):::::(1 + re0 )

(1+e
rt

r0 )
1 ):::::(1+e

; we get

+ ret

+ ret

i
1 )qt 1 wt 2

i
1 )qt 1 ct 2

qt wti
(25)
(1 + ret 1 ):::::(1 + re0 )
qt 1 wti 1
+
+ ::: + q0i w0i
(1 + ret 1 ):::::(1 + re0 )
qt 1 cit 1
qt cit
(1 + ret 1 ):::::(1 + re0 ) (1 + ret 1 ):::::(1 + re0 )
:::

q0i ci0 :

33

Note that the right hand side of this equation is nothing but the time-0 present
value of agents resources minus the present value of his/her consumption. Since
in this economy credit arrangements simply move resources between period and
do not add any resources to the agents budget constraint, we need to impose
the following condition
lti
= 0:
t!1 (1 + r
et 1 ):::::(1 + re0 )

(26)

lim

This condition (sometimes called no-Ponzi game condition) guarantees that

agent do not run a game where he/she keeps borrowing more and more and
never pays.
What is the relation between Arrow-Debreu prices and spot prices? Obviously, p0 = q0 = 1, where p0 = 1 is the Arrow-Debreu price of time-0 good. For
any t > 0; we have
pt =

qt
(1 + re0 )(1 + re1 )::::(1 + ret

1)

qt
:
t 1
et )
t=0 (1 + r

(27)

This equation has a very simple interpretation. In an Arrow-Debreu world, you

have to pay pt units of time 0 goods to get 1 unit of time-t goods. In a sequential
market one unit of time-t good costs qt at time t: How much of time 0 goods
you need to be able to pay qt ? Since you can transfer resources form period 0
t
to period t using one-period credit arrangements, you need exactly t 1q(1+e
.
r )
t=0

Then, the equation (25) becomes

t

X
lti
=
pt wt
(1 + ret 1 ):::::(1 + re0 )
t=0

t
X

pt ct :

t=0

Thus the condition (26) simply makes sure that as t ! 1; agents are on their
budget constraint:
1
1
X
X
pt wt =
pt ct ;
t=0

t=0

and if agents resources are nite, then there is a well-dened solution to agents
maximization problem. How can we make sure that (26) is satised? This can
be achieved by placing an upper bound, call it B i ; on how much each agent can
borrow. Note that any level of borrowing limit, even a very large one, will still
preclude the possibility of a Ponzi game.
In this environment given a sequence of budget constraints, the agents problem is characterized by the following set of FOCs:
(cit )
=
@cit

t @u

t qt ;

for cit ; and

t

+ (1 + ret )
34

t+1

= 0;

for lti ; where t is the Lagrange multiplier for time-t budget constraint.
We can combine these two conditions to arrive at
i

qt
or

t @u (ct )
@cit

= (1 + ret )

t+1 @u (ct+1 )
@cit+1

qt+1

@ui (cit )
qt @ui (cit+1 )
= (1 + ret )
:
i
qt+1
@ct
@cit+1

(28)

This is exactly the intertemporal optimization condition we had for ArrowDebreu economy, see equation (21), since using equation (27), we have
@ui (cit )
@cit

=
=

(1 + ret )
pt

pt+1

pt (1 + re0 )(1 + re1 )::::(1 + ret 1 ) @ui (cit+1 )

pt+1 (1 + re0 )(1 + re1 )::::(1 + ret )
@cit+1

@ui (cit+1 )
:
@cit+1

In this economy, the credit balances are recorded in terms of the value of
time-t goods. Time-t resource constraint implies
lti = qt wti

cit ;

indicating the time t value of your credit balance. Next period, the budget
constraint is
i
qt+1 cit+1 + lt+1
= qt wti + (1 + ret )lti ;

and you pay (or receive) to the credit agency a credit in the amount of (1 + ret )lti :
(1+e
r )li
li
Hence, you borrow qtt units of time-t goods and pay back qt+1t t : Then, the
interest rate in terms of physical quantities is
1 + rt =

(1+e
rt )lti
qt+1
lti
qt

(1 + ret )lti qt
(1 + ret )qt
=
:
qt+1 lti
qt+1

(29)

Rather than keeping the track of goods in terms of the value of goods, the credits
could also be recorded in the physical amount of goods. Suppose if you borrow
(or lend) lti units of goods in time t; then at time t + 1 you pay (or receive)
lti (1 + rt ) units of the good.
The spot prices are not necessary then to dene an equilibrium in this environment. Interest rates, which are dened in way, already contain the formation
about the relative value of the good. In this set-up the budget constraint of the
individual is
ci0 + l0i
ci1 + l2i
cit + lti

= w0i
= w1i + (1 + r0 )l0i
::::
= wti + (1 + rt 1 )lti
::::
35

(30)

The intertemporal FOC is given by

@ui (cit+1 )
@ui (cit )
=
(1
+
r
)
:
t
@cit
@cit+1

(31)

Again there is a well-dened relation between interest rates and Arrow-Debreu

prices, since
(1 + ret )qt
pt
1 + rt =
=
:
(32)
qt+1
pt+1

pt
) will appear may times as we move forward in this
This relation (1 + rt = pt+1
course.
We are now ready to dene a sequential equilibrium

Denition 22 A sequential market equilibrium is a sequence of allocations ci =

1
1
cit t=0 and a sequence of lending/borrowing decisions li = lti t=0 for each
1
i; and a sequence of prices r = frt gt=0 and a borrowing constraint for each
individual B i such that
and li solve agent i0 s maximization problem for each i:

1. Given r, ci

max
i
c

1
X

t
i
i ui (ct );

(33)

t=0

subject to
cit + lti
and
2. Markets clear

cit
lti

= wti + (1 + rt )lti

for all i and for all t

(34)

0 for all i and for all t;

B i ; for all i and for all t:
n
X

cit

i=1

and

n
X

wti for each t;

(35)

i=1

lti = 0 for each t:

(36)

It should be rather straightforward that consumption allocations in an ArrowDebreu equilibrium and in a sequential equilibrium are identical.

References
[1] Kehoe, T. Intertemporal General Equilibrium Models, in The Economies
of Missing Markets, Information, and Games, Frank Hahn (ed.), 1989.
[2] Negishi, T. Welfare Economics and Existence of An Equilibrium For a
Competitive Economy," Metroeconomica, 12, 92-97.
36

Stochastic Endowments

Everything was certain in the economies we have analyzed so far. But we all
know that uncertainty is an important element in many economic activities. We
will now extend our previous analysis in to a stochastic environment.
We assume time is discrete and the time horizon is innite, t = 0; 1; 2; :::::There
is again a nite number of agents indexed by i = 1; :::n and one consumption
good per period. This consumption goods is not storable. In contrast to the
previous set-up endowments depend on the state of the economy. The state of
the economy is uncertain. There are, for example, good days and bad days,
sunny days and rainy days. Yet this uncertainty has a well-dened structure.
We will let st denote the state of the economy at time t. It is a stochastic
variable, and we will assume that st can take values from a given nite set S.
For example if S = fgood; badg : Then, each period either st = good or st = bad:
We assume that st follows a Markov process, i.e. probability that st+1 = s0 only
depends on the current state st : We will let
(s0 js) = prob(st+1 = s0 jst = s);
represent the transition probability from st = s to st+1 = s0 : Since this is a
transition probability it must satisfy
X
(s0 js) = 1:
s0 2S

We assume that st is publicly observed. Hence, given any value of s0 ; we can use
transition probabilities to determine the probability of any particular sequence
of states occurring. We will call a possible realization of states up to time t a
history, denoted by st :
st = [st ; st 1 ; ::::; s0 ] :
Given s0 , the probability of a particular history st is
(st jso ) = (st jst

1 ):::::

(s1 js1 ) (s1 jso ):

Figure (8) illustrates possible 3 period histories when st can take two values
from the set S = fh; lg and s1 = l:
Below we will assume that at t = 0; s0 is known. Otherwise we have to
assume a probability distribution over s0 : If s0 is known, then the transition
probabilities characterize the stochastic structure of this economy. We will
analyze Markov processes in more detail below.
We assume that agents endowments depend on st : In particular we assume
that
wti = wi (st );
i.e. agent i0 s endowments is a time invariant function of st : For example, if
S = fl; hg ; then
2 if st = h
wti =
0 if st = l;
37

s 2= l

s2 = [l,1,1]

s 1= l
s0= l

s2 = [h,1,1]

s 2= h
s2= l

s2 = [l,h,1]

s2= h

s2 = [h,h,l]

s1= h

Figure 8: Possible 3 peirod histories

is a well-dened time invariant endowment function.

We assume that there is a time-0 Arrow-Debreu market where agents can buy
and sell not only goods of dierent periods but also goods of dierent histories.
People at time 0 choose a contingent consumption plan
ci = cit (st )

1
t=0

where agent decide his/her consumption for every date and for every possible
realization of the history.
Agents try to maximize the expected value of their utility dened as
"
#
1
X
X
t
i
i t
t
U (c ) =
u(ct (s )) (s js0 ) ;
(37)
t=0

st

where the term st t u(cit (st )) (st js0 ) represents the expected utility of timet consumption plan (given s0 ): U (ci ) is called an expected utility
P function or
sometimes a von-Neuman Morgenstein utility function. We use st to denote
summation over all possibly histories that can happen up to time t: We assume
that u is strictly increasing, strictly concave, and satises limc!0 u0 (c) = 1:
Let pt (st ) be the price of time-t, history st goods at time 0. Then the budget
constraint of the individual is
1 X
X
t=0

st

pt (st )cit (st ) =

1 X
X
t=0

38

st

pt (st )wti (st ):

(38)

The households problem is to maximize P

(37) subject
to (38). Note that for this
1 P
problem to have a well-dened solution t=0 st pt (st )wti (st ) must be nite.

Denition 23 An Arrow-Debreu equilibrium is a sequence of consumption plans

1
ci = cit (st ) t=0 for each i; and a sequence of history dependent prices p =
1
fpt (st )gt=0 such that, given s0 :
1. Given p, ci solves agent i0 s maximization problem for each i:
"
#
1
X
X
t
i
i t
t
U (c ) =
u(ct (s )) (s js0 ) ;
subject to

1 X
X

(39)

st

t=0

pt (st )cit (st ) =

t=0 st

1 X
X

pt (st )wti (st ):

(40)

t=0 st

2. Markets clear
n
X

cit (st )

i=1

n
X

wti (st ) for each t and each st :

(41)

i=1

Pn
Pn
i
Note that when we write i=1 cit (st )
i=1 wt (st ); the consumption det
pend on s while the endowments depend on st : This is ne, since any history st
implies a nal state st . In what follows we will normalize prices by p0 (s0 ) = 1:
The FOCs for the consumer are given by
t 0

u (cit (st )) (st js0 ) =

pt (st ) for all t:

(42)

Note that if p0 (s0 ) = 1; the any agent i

u0 (ci0 (s0 )) =

Again, we can use the FOC for t + 1;

t+1 0

u (cit+1 (st+1 )) (st+1 js0 ) =

pt+1 (st+1 ) for all t;

to arrive at
t 0

u (cit (st )) (st js0 )

=
pt (st )

or
u0 (cit (st )) =

t+1 0

u (cit+1 (st+1 )) (st+1 js0 )

;
pt+1 (st+1 )

pt (st )
(st+1 js0 ) 0 i
u (ct+1 (st+1 ));
t+1
pt+1 (s ) (st js0 )

For any two histories st+1 and st such that st+1 = [st+1 ; st ] ; we have
(st+1 js0 ) = (st+1 jst ) (st js0 ):
39

(43)

Then, the intertemporal FOC becomes

u0 (cit (st )) =

pt (st )
(st+1 jst )u0 (cit+1 (st+1 )):
pt+1 (st+1 )

Consumption of time t good in history st (with current state st ) is related to

the consumption of time t + 1 goods in history st+1 (with last state of st ) in the
usual way.
Note also that time-t; history st consumption of any two agents is related
by
i
u0 (cit (st ))
= j:
(44)
j
u0 (ct (st ))
Remark 24 Note that equation (44) has a very strong consumption insurance
implication: the ratio of marginal utilities of any two agents is constant across
time and states. For example with CRRA utilities
u(c) =

c1
1

with

> 0;

we get
1

cit (st ) = cjt (st )

Hence, time t consumption allocation to distinct agents are constant fractions

of each other, and as a result individual consumption is perfectly correlated
with aggregate consumption. Furthermore, time-t consumption is independent
of time-t endowment of the agent.
Thus we can write agent i0 s consumption as
cit (st ) = u0
where u0
implies

u0 (c1t (st )

indicates the inverse function of the u0 : We know that feasibility

I
X
i=1

cit (st ) =

I
X

u0

u0 (c1t (st )

i
1

i=1

I
X

wti (st ):

i=1

Since the right hand side of this equation only depends on st the left hand side
must also only depend on st ; hence we have
cit (st ) = cit (st ) for all i:
Example 25 Suppose n = 2; and
ui (cit ) = log(cit ) for i = 1; 2:

40

Suppose there are two states, S = f1; 2g : Let s0 = 1; and

prob(st+1 = 1jst = 1) =
Transition probabilities
tions are

22

and

11 ;
21

and prob(st+1 = 2jst = 1) =

12 :

are dened similarly. The endowment funcwt1 = st ;

and
wt2 = 3

st :

Hence, each period there is xed amount of good (i.e. there is no aggregate
uncertainty)
w = wt1 + wt2 = 3 for all t:
The FOC for cit (st ) is
t

1
(st js0 ) =
cit (st )

Since p0 (s0 ) = 1;

1
=
ci0 (s0 )

pt (st ):

Lets focus on t = 0; and t = 1; using equation (43) we have

1
ci1 (s1 )
or

(s1 js0 ) =

1
p1 (s1 );
i
c0 (s0 )

ci0 (s0 )
(s1 js0 ) = p1 (s1 ):
ci1 (s1 )

Here s1 is a history for t = 1: There are two possible histories [1; 1] and [2; 1]:
Since ; (s1 js0 ); and p1 (s1 )_ is same for both agents we have
c20 (s0 )
c10 (s0 )
=
:
c1t (s1 )
c2t (s1 )
Since c10 (s0 ) + c20 (s0 ) = 3; and c11 (s1 ) + c21 (s1 ) = 3; it must be the case that
c10 (s0 ) = c11 (s1 ) = c1 and c20 (s0 ) = c21 (s1 ) = c2 ;
where c1 + c2 = 3: Indeed one can easily show that this will be true for any time
period. Hence, agents consume a xed amount of goods every period. Then,
ci0 (s0 )
(s1 js0 ) = p1 (s1 ):
cit (s1 )
implies
p1 (s1 ) =
41

(s1 js0 ):

Indeed one can easily show that

t

pt (st ) =

(st js0 ):

Then, the lifetime budget constraint for agent 1 implies

X
X
1+
p1 (s1 )w11 (s1 ) +
p2 (s2 )w21 (s2 ) + ::::::
s1

= c1 +

p1 (s1 )c1 +

s1

Given that pt (st ) =

s2

p2 (s2 )c1 + :::::;

s2

(st js0 ); we have

X
X
1+
p1 (s1 )w11 (s1 ) +
p2 (s2 )w21 (s2 ) + ::::::
"

s1

s2

= c1 1 +
=

c1
1

X
s1

(s1 js0 ) +

s2

(s2 js0 ) + :::::

Therefore,
1

c = (1
and
2

c = (1

"

) 1+
"

) 2+

p1 (s

)w11 (s1 )

s1

)w21 (s2 )

)w22 (s2 )

p2 (s

s2

p1 (s

)w12 (s1 )

s1

p2 (s

s2

+ :::::: ;
#

+ :::::: :

Remark 26 With uncertainty sensitivity of agents to risk plays an important

role. Given u; there are two measures of risk aversion:
Given a utility function u; with u0 > 0 and u0
absolute risk aversion is dened as:
CARA =

u00 (c)
:
u0 (c)

Given a utility function u; with u0 > 0 and u0

risk aversion is dened as:
CRRA =

0; the coe cient of

0; the coe cient of relative

u00 (c)c
:
u0 (c)

Note that for a linear utility function both CARA and CRRA are zero.
Agents with linear utility are called risk neutral. Agents with concave utility functions are risk averse, and the curvature of the utility function determines the risk aversion.
42

Note also that if

u(c) =

c1
1

Then,
CRRA =

c
c

)c

= ;

and for this utility function determines both the elasticity of intertemporal substitution and the coe cient of relative risk aversion.

3.1

Asset Pricing

In an Arrow-Debreu equilibrium, there is a well-dened price for each good in

each possible state of the world. Since there is complete set of prices, any asset
which promises to deliver a particular sequence of state contingent goods is
redundant.
This redundancy condition can be used to price any asset. In particular let
1
fd(st )gt=0 be an asset that promises a deliver of d(st ) amount of time-t goods
if the state is st ; where d is a function that maps states into positive real line.
What is the value of this asset? If you want to sell this asset, you have to
make sure that if the state is st at time t you have d(st ) amount of time-t goods.
Then you have to buy d(st ) amount of time-t goods in (time-0) Arrow-Debreu
market. The state st is, however, is not known at time 0. Thus you have to
cover all possible states that can happen. Then the cost of this asset for the
seller is
1 X
X
pt (st )d(st );
P0;0 =
t=0 st

where P0 ;0 indicates that the asset starts payments at time 0, and its price is
measured in terms of the value of the goods at time-0.

Example 27 Let d(st ) = 1 for all st :The asset pays 1 unit in every date and
in every possible state. The cost of this asset is
P0;0 =

1 X
X

pt (st ):

t=0 st

Example 28 Let d(st ) = 1 if t = ; and d(st ) = 0 otherwise. The asset pays

one unit at time : Then the cost of this asset is
X
P0;0 =
pt (st ):
st

Suppose now we take an asset fd(st )gt=0 ; and get rid of the rst payments,
i.e. d(st ) = d(st ) if t > ; and 0 otherwise. The asset pays according to function
d(st ) starting at date + 1: How much this asset is valued at time 0? Obviously

43

it will depend on what the history is up to : Let P

asset, given history s ; in terms of time 0 goods:
P

1
X

;0 (s ) =

;0 (s

) be the price of this

pt (e
st )d(e
st );

t= +1 fe
st :e
s =s g

P
where fest :es =s g indicates that we are only summing over future histories that
has the right s :
What is the value of this asset in terms of time goods? It is simply
P ;0 (s )
;
p (s )
where p (s ) is the price of time- ; history s goods at time-0. Then,
P

1
X

(s ) =

t= +1 fe
st :e
s =s g

pt (e
st )
d(e
st ):
p (s )

Now P ; (s ) indicates the price of this asset, given history s ; in terms of

time- ; history s goods. In what follows we will simply use P (s ).
Given the equation (42),
pt (st )
=
p (s )

u0 (cit (st ))
(st js );
u0 (ci (s ))

represents the price of time t; history st goods in terms of time ; history s

goods. Then P (s ) is
P (s ) =

1
X

t= +1 fe
st :e
s =s g

u0 (cit (st ))
(st js )d(e
st ):
u0 (ci (s ))

This is our asset pricing formula.

Suppose the current history is st : Then, time-t price of an asset that pays
d(st+1 ) tomorrow and nothing else, for example, is given by
Pt (st ) = Et

u0 (cit (st+1 ))
d(st+1 ) ;
u0 (cit (st ))

where Et represents the expectations over st+1 given st :

Similarly, time-t price of an asset that pays d(st+1 ) for every period starting
at t + 1 is
#
" 1
0 i k
X
u
(c
))
(s
k t
t
Pt (st ) = Et
d(sk ) :
u0 (cit (st ))
k=t+1

Lets look at
Pt (st ) = Et

u0 (cit (st+1 ))
d(st+1 ) ;
u0 (cit (st ))
44

more carefully. We can rearrange it to arrive at

u0 (cit (st )) = Et

d(st+1 ) 0 i t+1
u (ct (s )) :
Pt (st )

For an asset that pays d(st+1 ) next period and nothing else,
represents its return. Then
u0 (cit (st )) =

d(st+1 )
Pt (st )

= Rt+1

Et [Rt+1 ] Et u0 (cit (st+1 )) + Covt u0 (cit (st+1 ))Rt+1

or
Et [Rt+1 ] =

1
Et u0 (cit (st+1 ))

u0 (cit (st ))

Covt u0 (cit (st+1 ))Rt+1

What does the covariance term indicate about asset prices? Suppose cit (st+1 )
and d(st+1 ) move together, i.e. the asset pays you a high amount when your
consumption is high. Then, Covt u0 (cit (st+1 ))Rt+1 is negative and Et [Rt+1 ]
must be higher. Hence, if an asset is highly correlated with your consumption,
then its return must be high. You need a high return to hold this asset because
it is risky. If on the other hand cit (st+1 ) and d(st+1 ) move in opposite direction,
the asset is less risky and the return can be lower. The risky asset has to pay a
premium. We call this the risk premium.
Finally note that given
" 1
#
0 i k
X
(s
))
u
(c
k
t
t
d(sk ) :
Pt (st ) = Et
u0 (cit (st ))
= Et

k=t+1
u0 (cit (st+1 ))
d(st+1 )
u0 (cit (st ))

and
t+1

Pt+1 (s

= Et+1
= Et+1

"

1
X

(cit (st+2 ))
d(st+2 ) + :::: ;
u0 (cit (st ))

2u

#
(cit (sk ))
d(sk ) :
u0 (cit (st ))

k t 1u

k=t+2
u0 (cit (st+2 ))
d(st+2 )
u0 (cit (st+1 ))

(cit (st+3 ))
d(st+3 ) + :::: :
u0 (cit (st+1 ))

2u

It is easy to show that

Pt (st ) = Et

u0 (cit (st+1 ))
[d(st+1 ) + Pt+1 (st+1 )] :
u0 (cit (st ))

Then,
u0 (cit (st )) = Et

u0 (cit (st+1 ))

[d(st+1 ) + Pt+1 (st+1 )]

;
Pt (st )

(45)

which is nothing but our familiar intertemporal optimization condition, since

[d(st+1 ) + Pt+1 (st+1 )]
;
Pt (st )
45

is the return on this asset. This should not be surprising since we arrive at these
asset pricing functions from agents optimization problem.
Remark 29 Material in this section follows Ljungqvist and Sargent (2004).
Remark 30 Note that asset pricing equations that we analyze above provides a
relation between prices and quantities, but do not go far in enough to drive an
asset pricing function that maps prices to the fundamentals of the economy in
equilibrium. This is done in Lucas (1978).
Remark 31 Mehra and Prescott (1982) apply Lucas (1978) framework to the
U.S. data and investigate if the risk premium implied by the model is consistent
with the data.

References
[1] Ljungqvist, Lars and Thomas J. Sargent. 2004. Competitive Equilibrium
with Complete Markets (Chapter 8), in Recursive Macroeconomic Theory, MIT Press.
[2] Lucas, Robert E., Jr. "Asset Prices in an Exchange Economy," Econometrica, 46(6), 1978, 1429-1445.
[3] Mehra, Rajnish and Prescott, Edward C., The Equity Premium: A Puzzle,
Journal of Monetary Economics, vol. 15, March 1985, pages 145-161.

46

Overlapping Generations

In the previous section of this course we focused on pure exchange economies

(static and dynamic) and study some fundamental concepts of general equilibrium, such as the Arrow-Debreu and the sequential equilibrium. We also looked
at the basic properties of intertemporal decision making in dynamic, stochastic
environments.
In the rest of this class, we will study two major workhorses of modern
macroeconomics: overlapping generations (OLG) models and the one-sector
growth model. For each of these models, we will present the basic environment,
introduce the equilibrium concepts that we will use, and look at the tools that
we need to analyze such environments. We will start with the OLG models.
This model was developed by Allais (1947), Samuelson (1958) and Diamond
(1965), and is used to study a variety of issues in modern macroeconomics.

4.1

Exchange Economy

Consider the following overlapping generations structure described below. Time

is discrete and the horizon is innite, t = 1; 2; :::: There is one perishable consumption good per period. Each agent lives for two periods. Therefore at each
point in time the economy is populated by two generations, the young and the
old. At each point in time a new generation appears.
GENERATIONS
0
1
2
3
..
.

1
old (initial)
young

..
.

TIME
2
old
young
..
.

old
young
..
.

:
..
.

There are N (t) members of generation t: Let cht (t) and cht (t + 1) be the
consumption of agent h in generation t at time t (when young) and at t + 1
(when old), respectively. Similarly let wth (t) and wth (t + 1) be the endowment
of agent h in generation t at time t (when young) and at t + 1 (when old). At
time 1, there is N (0) old people (the initial old).
We will assume that the preferences of agent h in generation t > 1 is repre2
! R is dierentiable, with
sented by uht (cht (t); cht (ct+1 )); where u : R+
@uht cht (t); cht (t + 1)
@ 2 uht cht (t); cht (t + 1)
> 0 and
< 0 for j = 0; 1:
2
h
@ct (t + j)
@ cht (t + j)

We will also assume that the initial old have preferences represented by a strictly
increasing utility function.
Remark 32 We will also use chyt and chot+1 to denote an agents consumption
h
h
to denote his endowments). Furwhen young and old (similarly wyt
and wot+1
thermore, if it is understood that every generation is identical and all agents in
47

each generation are identical, we will simply refer to the consumption when old
and when young as cy and co (or c1 and c2 ):
Denition 33 A consumption allocation is a sequence
cht 1 (t)

C=

N (t 1)
h=1

; cht (t)

N (t)
h=1

t=1

Denition 34 A consumption allocation C is feasible if, for all t

N (t 1)

N (t)

C(t) =

cht (t)

h=1

{z

Ct (t)

+
|

h=1

N (t)

cht 1 (t)
{z

Ct

1 (t)

h=1

N (t 1)

wth (t)
{z

Yt (t)

+
|

h=1
Yt

wth 1 (t) = Y (t):

{z

1 (t)

Denition 35 A feasible consumption allocation C is e cient, if there does

b such that
not exit any other feasible allocation C
b
8t; C(t)

b > C(t) for some t:

C(t); with C(t)

Denition 36 A consumption allocation CA is Pareto superior to CB ; if

(1) no agent prefers B to A strictly, (2) at least one agent prefers A to B strictly.
Denition 37 A consumption allocation is Pareto optimal, if it is feasible and
if there does not exist another feasible allocation that is Pareto superior to it.
Denition 38 A consumption allocation is symmetric if all members of all
generations consume the same consumption pair,
cht (t) = cjj (s) = c1 ; cht (t+1) = cjj (s+1) = c2 ; for all h and j in generations t or s.
Hence, a symmetric allocation treats all young in all generations and all old in
all generations the same way. If N (t) = N (t + 1) = N; and Y (t) = Y (t + 1) = Y;
the set of symmetric and e cient allocations will be characterized by (see Figure
9):
N c1 + N c2 = Y;
or
c1 + c2 =

4.2

Y
:
N

Competitive Equilibrium

We will look at two dierent market arrangements: Arrow-Debreu markets and

sequential markets (see Kehoe (1989) for a detailed analysis of the AD set-up).

48

c1

Y/N

slope = -1

Y/N

c2

Figure 9: Symmetric and E cient Allocations

4.2.1

Arrow-Debreu Equilibrium

Suppose all agents (born and unborn) participate in a time-1 market where they
can sell their endowments and buy consumption goods. Let p(t) be the price of
time-t goods in this market. Then the problem of agent h in generation t > 1 is
max

h
ch
t (t);ct (t+1)

uht (cht (t); cht (t + 1));

P(1)

subject to
p(t)cht (t) + p(t + 1)cht (t + 1)
cht (t)

p(t)wth (t) + p(t + 1)wth (t + 1)

0; cht (t + 1)

0:

The problem of the initial old is much simpler and given by

max uh0 (ch0 (1))

P(2)

ch
0 (1)

subject to
p(1)ch0 (1)

p(1)w0h (1):

b=
Then, an Arrow-Debreu equilibrium is an allocation C
1

b
cht 1 (t)

N (t 1)
h=1

; b
cht (t)

and a sequence of prices fb

p(t)gt=1 such that given the sequence of prices the
b
elements of C solve each generations maximization problem and the markets
clear.
49

N (t)
h=1

1
t=1

b and a seDenition 39 An AD equilibrium is a consumption allocation C

1
quence of prices fb
p(t)gt=1 such that
1
(1) Given fb
p(t)gt=1 ; consumers maximize their utility by choosing the releb i.e. consumers solve P(1) and P(2).
vant elements of C;
(2) Goods market clears, that is for all t > 1
N (t)

N (t 1)

cht (t) +

h=1

N (t)

cht 1 (t)

h=1

N (t 1)

wth (t) +

h=1

wth 1 (t):

h=1

Given a set of prices fp(t)gt=1 ; an agent of generation t solves the following

problem:
L =

max

h
ch
t (t);ct (t+1)

uht cht (t); cht (t + 1)

p(t)wth (t) + p(t + 1)wth (t + 1)

p(t)cht (t)

p(t + 1)cht (t + 1) :

Using the FOC for this problem,

and

@uht cht (t); cht (t + 1)

@L
=
h
@ct (t)
@cht (t)

p(t) = 0;

@uht cht (t); cht (t + 1)

@L
=
+ 1)
@cht (t)

p(t + 1) = 0;

@cht (t

we can determine the optimal consumption/saving decision:

@uh
t
@ch
t (t)

p(t)
=
p(t + 1)

@uh
t
@ch
t (t+1)

| {z }
M RS

Note that this is again the standard optimality condition that equates the M RS
to the price ratio.
If
uht cht (t); cht (t + 1) = u(cht (t)) + u(cht (t + 1));
this condition gives us the intertemporal optimization condition
u0 (cht (t)) =

p(t)
u0 (cht (t + 1));
p(t + 1)

that we have seen again and again.

50

4.2.2

Sequential Markets

We could also imagine a market economy in which individuals trade their endowments using one period lending and borrowing arrangements to maximize
their lifetime utility, and in which market clearing determines the interest rate.
Let R(t) be the real gross interest rate between t and t + 1; representing how
much time t + 1 goods market is willing to pay for each unit of time t goods.
Then, the consumers problem is given by
max

h
h
ch
t (t);ct (t+1);lt

uht (cht (t); cht (t + 1))

P(3)

subject to
cht (t)

wth (t)

lth
|{z}

lending/b orrowing

cht (t + 1)
cht (t)

lth R(t) + wth (t + 1)

0; cht (t + 1)

0:

Generation 0 (the initial old);

max uh0 (ch0 (1))

P(4)

ch
0 (1)

subject to
ch0 (1)

w0h (1):

Remark 40 Note that lth < 0 represents borrowing.

Remark 41 Note that the old at time t cant borrow since no one would lend
them.
For generations 1, 2,..., we can combine two budget constraints to arrive at
a lifetime budget constraint (represented in Figure 2):
cht (t) +

cht (t + 1)
R(t)

wh (t + 1)
:
wth (t) + t
R(t)
|
{z
}
lifetim e wealth

Note that lending/borrowing does not appear in this budget constraint since
they are simply ways to allocate resources between two periods.
An agents optimal decision will be determined by the solution of the following problem:
L=

max

h
ch
t (t);ct (t+1)

uht cht (t); cht (t + 1) +

wth (t) +

51

wth (t + 1)
R(t)

cht (t)

cht (t + 1)
:
R(t)

Using the FOC for this problem,

@uht cht (t); cht (t + 1)
@L
=
h
@ct (t)
@cht (t)
and

@uht cht (t); cht (t + 1)

@L
=
+ 1)
@cht (t)

@cht (t

= 0;

1
= 0;
R(t)

we can determine the optimal consumption/saving decision (represented by

point C in Figure 2):
R (t) =

@uh
t
@ch
t (t)
@uh
t
@ch
t (t+1)

| {z }
M RS

Note again that if

uht cht (t); cht (t + 1) = u(cht (t)) + u(cht (t + 1));

this condition gives us the intertemporal optimization condition
u0 (cht (t)) = R(t)u0 (cht (t + 1)):

cth (t+1)

cth (t+1)

slope = -R(t)

wth (t+1)

cth (t)

wth (t)

cth (t)

Figure 10: Optimal Decisions

52

Denition 42 A sequential market equilibrium is a sequence of consumption

N (t)

e a sequence of lending/borrowing decisions, e

allocation C;
lth
; and a
h=1 t=1
n
o1
e
sequence of interest rates R(t)
such that
t=1
n
o1
e
(1) given R(t)
; consumers maximize their utility by choosing the relt=1

e i.e. consumers solve P(3) and P(4).

evant elements of C;
(2) loans market clears, that is for all t > 1
N (t)

h=1

e
lth = 0:

Remark 43 Note that for any given exchange economy Arrow-Debreu equilibrium and sequential market equilibrium are equivalent. That is given any
1
b and fb
Arrow-Debreu equilibrium, C
p(t)gt=1 ; nwith pb
> 0 for all t; there is a
o(t)
1
e
e
e > 0 for all t;
corresponding sequential equilibrium, C and R(t)
with R(t)
t=1

b and C
e are identical. Similarly, given any sequential equilibrium, C
b
such nthat C
o1
b
b
and R(t)
; with R(t) > 0 for all t; there is a corresponding AD equilibt=1

1
e and fe
b and C
e are identical.
rium, C
p(t)gt=1 with pe(t) > 0 for all t; such that C

Remark 44 Note that in the denition of the sequential market equilibrium we

ignored the goods market equilibrium. This is because the good market clearing
implies the loan market clearing. To see this note that at any time t; the total
consumption of young is
N (t)

N (t)

cht (t) =

h=1

N (t)

wth (t)

h=1

lth ;

h=1

and the total consumption of old is

N (t 1)

N (t 1)

cht 1 (t)

= R(t

1)

h=1

N (t 1)

lth 1

h=1

wth 1 (t):

h=1

Then
N (t)

h=1

N (t 1)

cht (t)+

h=1

N (t)

cht 1 (t) =

N (t)

wth (t)

h=1

N (t 1)

lth +R(t 1)

h=1

h=1

If goods markets clear we have

N (t)

N (t 1)

lth

= R(t

1)

h=1

h=1

53

lth 1 :

N (t 1)

lth 1 +

h=1

wth 1 (t):

If if the initial old doesnt have any debt to pay, that is if

N (0)

l0h = 0;

h=1

then,
N (t)

lth = 0 for all t > 1:

h=1

Example 45 Consider an OLG economy with identical agents and no population growth. Let
ut (cyt ; cot+1 ) = log(cyt ) + log(cot+1 );
and
[wyt ; wot+1 ] = [3; 1] :
Then
max log(cyt ) + log(cot+1 )

cyt ;cot

subject to
cyt +

cot+1
R(t)

wyt +

wot+1
:
R(t)

With
L = max flog(cyt ) + log(cot+1 ) + (wyt +
cyt ;cot+1

wot+1
R(t)

cyt

cot+1
)
R(t)

Hence, FOCs are

1
cyt
and

1
cot+1

= 0;
1
= 0:
R(t)

Therefore
1
cot+1
cyt +

1 1
! cot+1 = R(t)cyt ;
R(t) cyt

wot+1
wot+1
R(t)cyt
= wyt +
) 2cyt = wyt +
R(t)
R(t)
R(t)

and,
1
wot+1
1
[wyt +
]; cot+1 = [wyt r(t) + wot+1 ]:
2
R(t)
2
Let lt be the savings given by
cyt =

lt = wyt

cyt =

1
wyt
2

1 wot+1
:
2 R(t)

Since in equilibrium lt = 0 (no heterogeneity, hence no lending and borrowing):

R(t) =

wot+1
1
= :
wyt
3
54

Remark 46 Note that although there is no lending or borrowing in equilibrium,

we still have to nd R(t) that makes this behavior optimal for the agents.
wo
simply represents the resources of an agent when
Note that R(t) = R = w
y
he/she is old and young. In this economy, the interest rate is completely determined by endowment structure. Consumption allocations in competitive equilibrium is
C = f(co1 ; cy1 ) ; (co2 ; cy2 ) ; (co3 ; cy3 ) ; :::g = f(3; 1) ; (3; 1); (3; 1); :::g :
Nobody saves in this economy (i.e. autarchy is the equilibrium), since all agents
are identical and they all want to lend when they are young and borrow when
they are old. Note that this allocations is not Pareto optimal, since consumption
allocation
b = f(b
C
co1 ; b
cy1 ) ; (b
co2 ; b
cy2 ) ; (b
co3 ; b
cy3 ) ; :::g = f(2; 2) ; (2; 2); (2; 2); :::g ;

b
is both feasible and Pareto superior to C: Note that all generations prefer C;
since
log(2) + log(2) > log(3) + log(1) for all generations t 1;
and
log(2) > log(1) for the initial old.
Example 47 Consider an economy with agents having similar utility functions
as the previous example
max log(cyt ) + log(cot+1 );

cyt ;cot+1

but let the economy be populated by two types of agents that di er in their endowments
1
1
2
2
wyt
; wot+1
= [1; 1] and wyt
; wot+1
= [2; 1] :
Now type-2 agents want to lend when they are young, and might be able to do
this by o ering an interest to type-1 agents. If we go through the same exercise
as above for each type of agents, we will get
1
lt1 = wyt

c1yt =

1 1
w
2 yt

1
1 wot+1
1
=
2 R(t)
2

1
2R(t)

2
lt1 = wyt

c2yt =

1 2
w
2 yt

2
1 wot+1
=1
2 R(t)

1
:
2R(t)

and

Therefore
lt1 + lt2 = 0 )
and
c1y =

1
2

1
+1
2R(t)

1
2
= 0 ) R(t) = ;
2R(t)
3

w1
1 1
5
[wyt + ot+1 ] = ) lt1 =
2
R(t)
4
55

1
;
4

c1ot+1 = R(t)c1yt =
c2yt =

w2
1 2
7
1
[wyt + ot+1 ] = ) lt1 = ;
2
R(t)
4
4

c2ot+1 = R(t)c2yt =
Remark 48 Note again that R(t) =
ments of young and old.

4.3

52
5
= ;
43
6

2
3

72
7
= :
43
6

simply reects the relative total endow-

Pareto Optimality in OLG models

We will now investigate Pareto optimality in simple OLG setting that we constructed above. We will go through several claims:
Claim 1 If an allocation is PO, then it is e cient.
Claim 2 Suppose that ut (cht (t); cht (t + 1)) is the utility function of person h in
generation t; t 1: Let MRS be dened as
uht1 (cht (t); cht (t + 1))
;
uht2 (cht (t); cht (t + 1))
where uhtj is the partial derivative of utility function with respect to jth
argument. Suppose that h and h0 are two members of generation t: A
feasible allocation that assigns positive 1st and 2nd period consumption
to h and h0 and implies dierent M RS for h and h0 is not PO.
Claim 3 Consider a stationary and symmetric OLG environment with endowments given by (w1 ,w2 ): With strictly convex indierence curves, and with
w1 > 0; the unique equilibrium (i.e. autarchy) is PO if and only if MRS
at the endowment point is greater than equal to 1, i.e.
u1 (w1 ; w2 )
u2 (w1 ; w2 )

1:

1 ;w2 )
Proof Suppose uu12 (w
(w1 ;w2 ) < 1: Then, u1 (w1 ; w2 ) < u2 (w1 ; w2 ); and (c1 =
w1 ; c2 = w2 ) is not Pareto optimal. To see this consider the alternative feasible allocation that gives e
c1 = w1 " to the young and e
c2 = w2 "
to the old in every period. This allocation Pareto dominates c1 = w1 and
c1 = w2 for some " 2 (0; w1 ); that makes u1 (e
c1 ; e
c2 ) = u2 (e
c1 ; e
c2 ):

(w1 ;w2 )
> 1; and suppose c1 = w1 and c2 = w2 is not
Suppose now uu21 (w
1 ;w2 )
Pareto optimal. Then, there must exists an alternative feasible allocation
e
c1t and e
c2t that Pareto dominates c1 = w1 and c2 = w2 : That is, for all t

u(e
c1t ; e
c2t )

56

u(w1 ; w2 );

and for some t

u(e
c1t ; e
c2t ) > u(w1 ; w2 ):

Let t be the rst date such that e

c1t 6= w1 and e
c2t 6= w2 : The only way
e
c1t and e
c2t can be dierent from the autarky and Pareto improving is if
goods are transferred from the young to the old at time t:
Let "t be the amount of goods that the young give to the old at time t:
Then at time t; e
c2t = w2 + "t ; and e
c1t = w1 "t : Now next period the old
(who were young at time t) must receive transfers from the young at time
t + 1: Lets call this transfer "t+1 :

Since u1 (w1 ; w2 ) > u2 (w1 ; w2 ); in order to make generation t as well o as

they were in the autarky, we need "t+1 > "t : Then, by the same argument
"t+2 > "t+1 ; and eventually the required transfer will be more than w1 :
Hence, we cant nd an alternative allocation that Pareto dominates c1 =
w1 ; and c2 = w2 :

4.4

Introducing a government

Suppose now there is a government that can levy taxes and provide subsidies.
Let
h
h
h
t =
t (t); t (t + 1)
be the taxes or subsidies that agent h of generation t faces in his/her lifetime:
The government budget has be balanced, hence
N (t)

N (t 1)
h
t (t)

h=1

h
t 1 (t)

= 0:

h=1

Then, the lifetime budget constraint for an agent will be given by

cht (t) +

cht (t + 1)
R(t)

wth (t)

h
t (t)

h
wth (t + 1)
t (t + 1)
:
R(t)

We can dene an equilibrium as:

Denition 49 A sequential market equilibrium is a sequence of taxes, (eht 1 (t); eht (t))1
t=1 ; a
e a sequence of lending/borrowing decisions,
sequence of consumption allocation C;
n
o1
e
and a sequence of interest rates R(t)
such that
n t=1o1
e
(1) given (eht 1 (t); eht (t))1
; consumers maximize their utilt=1 and R(t)
t=1

e
ity by choosing the relevant elements of C:
(2) the government budget balances, that is for all t > 1
N (t)

h=1

N (t 1)

eht (t)

h=1

57

eht 1 (t) = 0:

e
lth

N (t)

h=1

t=1

(2) loans market clears, i.e. for all t > 1

N (t)

h=1

e
lth = 0:

Example 50 Let [wyt ; wot+1 ] = [2; 1]; 8t and assume the same preference structure as the examples above. Let [ yt ; ot+1 ] = [ 21 ; 12 ]; 8t. Hence, government
taxes young and transfers the proceeds to the old. Then, going trough the problem
of the agents we have
lt = wyt

yt

cyt =

1
(wyt
2

2
1 wot+1
ot+1
=
2
R(t)
2

yt )

1
2

1 + 21
) R(t) = 1:
2R(t)

and

3
:
2
Note that this allocation is Pareto optimal. In contrast, if there were no government
the allocation would not be Pareto optimal. Note also that = 21 is not a random
tax rate for the economy in this example. If we wanted to nd the tax/transfer
rate that maximizes a representative agents lifetime utility, we would solve the
following problem:
max log(wy
) + log(wo + );
cyt = cot+1 =

where we use the fact that autarchy is the competitive equilibrium for this economy. Hence the optima tax rate is given by
1

wy

1
wo +

1
:
2

= 12 ; the interest rate is 1.

Note also that at the optimal tax rate

Now let the government be able to borrow from the young as well (note
that government cannot borrow from the old). Suppose government issues one
period bonds that are sure claims on 1 unit of goods the next period. Then, if
B(t) is the units of bonds sold at time t; at time t + 1 the government needs
B(t) units of time t + 1 goods to be able to pay back his commitments. The
government can achieve this by taxing the young, taxing the old, or by issuing
new bonds at time t + 1:
The government budget is then given by
N (t)

h=1

N (t 1)
h
t (t)

h
t 1 (t)

+ p(t)B(t)

B(t

1) = 0;

h=1

where p(t) is the price of a unit of government bond at time t: The government
budget is in balance if
N (t)

h=1

N (t 1)
h
t (t)

h=1

58

h
t 1 (t)

= 0;

PN (t)
PN (t 1) h
and if h=1 ht (t) + h=1
B(t 1) > 0 and
t 1 (t) < 0; then p(t)B(t)
government needs to issue new bonds, i.e. p(t)B(t) > B(t 1):
Note that with government bonds individual budget constraints become
cht (t) = wth (t)

h
t (t)

lth

p(t)bht ;

and
cht (t + 1) = wth (t)

h
t (t

+ 1) + R(t)lth + bht ;

where bht is the demand for government bonds by agent h of generation t: Note
that an agent demands bht units of government bonds at a unit price of p(t)
when young that will deliver bht units of goods next period. Note also that the
1
:
rate of return on government bonds is p(t)
The lifetime budget constraint for an agent is then given by
cht (t) +

cht (t + 1)
= wth (t)
R(t)

h
t (t)

wth (t)

h
t (t

+ 1)

R(t)

bht p(t)

1
:
R(t)

1
Note an equilibrium exits only if R(t) = p(t)
: Otherwise the return on the
government bond and the return on private lending and borrowing are not the
same and agents can make prots by borrowing from the agents and lending
1
to the government. When R(t) = p(t)
; these arbitrage opportunities are all
exhausted.
1
When R(t) = p(t)
; the budget constraint of the agent h becomes

cht (t) +

cht (t + 1)
= wth (t)
R(t)

h
t (t)

wth (t)

h
t (t

+ 1)

R(t)

Hence, agents simply try to maximize their lifetime utility given their lifetime
resources. They are indierent between lending to the government or lending
to the other agents. All that matters is the total amount that agents want to
bh
t
lend (to the government or to the others), sht = lth + p(t)bht = lth + R(t)
:
In equilibrium,
N (t)

N (t)

cht (t)

h=1

N (t)

wth (t)

h=1

N (t)

N (t)
h
t (t)

h=1

lth

h=1

| {z }

p(t)bht

h=1

=0

and the total amount that agents are willing to lend to government is
N (t)

N (t)

wth (t)

h
t (t)

cht (t) =

h=1

h=1

p(t)bht

| {z }
total lending to
the government

and this must be equal to p(t)B(t):

An equilibrium is now dened as
59

Denition 51 A sequential equilibrium is a sequence of taxes, (eht 1 (t); eht (t))1

t=1 ;
1
e
a sequence of government borrowing, B(t)
; a sequence of consumption alt=1

N (t)
e , a sequence of lending/borrowing decisions
location C
seht h=1
; and a
t=1
n
o1
e
sequence of interest rates R(t)
such that
t=1n
o1
e
(1) given (eht 1 (t); eht (t))1
and
R(t)
; consumers maximize their utilt=1
t=1

e
ity by choosing the relevant elements of C:
(2) the government budget constraint holds, that is for all t
N (t)

h=1

N (t 1)

eht (t) +

eht 1 (t) +

h=1

(2) loans market clears, i.e. for all t

N (t)

h=1

e
B(t)
e
R(t)

e
B(t

1) = 0;

N (t)

X ebh
e
B(t)
t
+
= 0:
e
e
R(t)
h=1 R(t)
{z
}

e
lth +

et (R(t))
e
S

Remark 52 Note that we could introduce an exogenous stream of government

expenditure as well. Suppose the government spends each period G(t): This
resources do not provide any direct utility to the consumers and must be nanced
by taxes or by borrowing. Then the government budget constraint would become
N (t 1)

N (t)

h=1

eht (t) +

h=1

eht 1 (t) +

B(t)
R(t)

B(t

1)

G(t) = 0;

Remark 53 Note again that the goods market equilibrium and government budget constraint implies the loan market equilibrium. The budget constraint of
young agents at time t implies
N (t)

N (t)

cht (t) =

h=1

N (t)

wth (t)

h=1

N (t)
h
t (t)

h=1

sht ;

h=1

and similarly the budget constraint of the old agents implies

N (t 1)

N (t 1)

cht 1 (t) = R(t

1)

h=1

N (t 1)

sht

1+

h=1

N (t 1)

wth 1 (t)

h=1

h
t 1 (t):

h=1

Then
N (t)

N (t 1)

cht (t)

h=1

h=1

N (t)

N (t)

h=1

wth (t)

h=1

cht 1 (t)
N (t)
h
t (t)

N (t 1)

sht

+ R(t

h=1

1)

h=1

60

N (t 1)

sht 1

h=1

N (t 1)

wth 1 (t)

h=1

h
t 1 (t):

If goods markets clear, we have

N (t)

0=

N (t 1)

sht + R(t

1)

h=1

N (t)

sht

h=1

N (t 1)

h
t (t)

h=1

h
t 1 (t):

h=1

From the government budget constraint,

N (t)

N (t 1)

h
t (t)

h=1

h
t 1 (t)

+ p(t)B(t)

B(t

1) = 0;

h=1

we have
N (t)

N (t 1)
h
t (t)

h=1

Then, we have

= p(t)B(t)

B(t

1):

+ p(t)B(t)

B(t

1):

h=1

N (t)

h
t 1 (t)

N (t 1)

sht

= R(t

1)

h=1

sht

h=1

If the initial old and the government at time 0 has no outstanding debt, that is
if,
N (0)
X
sh0 = 0 and B(0) = 0;
h=1

we have

N (1)

sh1 = p(1)B(1);

h=1

and loan markets clear at time t: Then at time 2,

N (2)

N (1)

sh2

= R(1)

h=1

sh1 + p(2)B(2)

B(1)

sh1 + p(2)B(2)

R(1)

h=1
N (1)

= R(1)

N (1)

h=1

sh1

h=1

= p(2)B(2);

and loan markets clear at time 2 as well. Indeed they clear in every period.
Example 54 Suppose that in period 1, the government wishes to borrow 5 units
of time 1 goods and transfer it to the initial old. It will pay o this debt by taxing
the young of generation 2 and will not issue new debt. Let
1
1
2
2
wyt
; wot+1
= [2; 1] ; with Nt1 = 50; and wyt
; wot+1
= [1; 1] ; with Nt2 = 50; 8t:

61

and
uht = chyt chot+1 :
It is straightforward to drive
s1t = 1

1
;
2R(t)

1
2

1
:
2R(t)

and
s2t =
Then,
X

1
2R(t)

sht (R(t)) = 50 1

+ 50

1
2

1
2R(t)

= 75

50
:
R(t)

At t = 1; the loan market equilibrium implies

75

50
R(1)

B(1)
= 0;
R(1)
| {z }
=5

and

R(1) =
Given r1 ;
s11 = 1
and
s21 =

1
2

5
:
7

1
= 0:3;
2R(1)
1
=
2R(1)

0:2:

Then,
25
:
7
Note that type 1 agents at time 1 want to lend 0.3 each or 15 as a whole. 10
units of this is borrowed by type 2 agents, and the remaining 5 is borrowed by
the government. Each bond costs p1 = 75 units of type 1 goods. Since type 1
agents have more resources when young, they are willing to pay a price higher
than one for a unit of type 2 good to the government.
B(1) = 5R(1) =

Example 55 Let
[wyt ; wot+1 ] = [2; 1] ; with Nt = 100:
and
uht = chyt chot+1 :
Suppose government issues bonds in the rst period, and gives the revenue to
the current old. Moreover, the government wants to raise 50 units of period 1
62

goods. After period 1, government issues new bonds at each period to pay the
outstanding claims. Note that
st = 1 =
Hence,

1
;
2R(t)
50
:
R(t)

st = 100

At t = 1;

100

50
= B(1) = 50:
R(1)

and
R(1) = 1 and B(1) = 50:
Then,
1
1
= :
2
2
Hence, young lends government 12 and each old consumes 12 units extra. Next
period the same situation will be repeated by B(2) = 50; and R(2) = 1; etc. Note
that here government takes place of a permanent borrower, and allows agents to
transfer resources from one period to the next.
s1 = 1

1
Remark 56 Since R(t) = p(t)
; we could model government borrowing slightly
di erently. We could simply state that the government wants to borrow some
amount B(t) at time t; and like any other agent in this economy is willing to
pay an interest rate R(t): Then the government budget constraint would be
N (t)

N (t 1)
h
t (t)

h=1

h
t 1 (t)

+ B(t)

R(t)B(t

1) = 0;

h=1

and the loan market equilibrium condition would be

X
X
lth +
bht + B(t) = 0:
h2N (t)

{z

h2N (t)

St (R(t))

Consider again the budget constraint for agent h in generation t: Suppose

B(t) = 0; for all t: Then, the budget constraint is
cht (t) +

cht (t + 1)
= wth (t)
R(t)

h
t (t)

h
wth (t + 1)
t (t + 1)
:
R(t)

Now consider an alternative tax scheme that agent h faces, [eht (t); eht (t + 1)]: As
long as
h
eht (t + 1)
h
t (t + 1)
h
(t)
+
=
e
(t)
+
;
t
t
R(t)
R(t)
the agentss optimal decision will be the same. This is simply because his/her
budget set doesnt change. Then, we can state the following:
63

e
Proposition 57 Consider a sequential equilibrium, (eht 1 (t); eht (t))1
;
t=1 ; B(t)
t=1
n
o1
e and R(t)
e
e = 0 for all t: Then, alternative taxes and transC
; where B(t)
t=1

fers (bht 1 (t); bht (t))1

t=1 that satisfy
bht (t) +

for all h and all t; and

eh (t + 1)
bht (t + 1)
= eht (t) + t
;
R(t)
R(t)

N (t)

h=1

N (t 1)

bht (t) +

h=1

bht 1 (t) = 0;

for all t are equivalent. That is allocations and the prices under (bht 1 (t); bht (t))1
t=1
are identical to allocations and prices under (eht 1 (t); eht (t))1
t=1 :
Indeed, we could state the following (which you should try to prove) two
results as well:

e
Proposition 58 Consider a sequential equilibrium, (eht 1 (t); eht (t))1
;
t=1 ; B(t)
t=1
n
o1
e and R(t)
e
C
: These equilibrium allocations can be duplicated with alternat=1

tive taxes (bht 1 (t); bht (t))1

t=1 that satisfy
N (t 1)

N (t)

h=1

bht (t) +

h=1t 1

bt

1 (t)

= 0; for all t;

b
b
with B(t)
= 0 are equivalent. That is, given (bht 1 (t); bht (t))1
t=1 and B(t) = 0;
e
the equilibrium
o1 allocations and equilibrium interest rate will be identical to C
n
e
:
and R(t)
t=1

e
;
Proposition 59 Consider a sequential equilibrium, (eht 1 (t); eht (t))1
t=1 ; B(t)
t=1
n
o1
e and R(t)
e
C
: Then alternative taxes and transfers (bht 1 (t); bht (t))1
t=1 that
t=1
satisfy
bh (t + 1)
eh (t + 1)
bht (t) + t
= eht (t) + t
for all h and all t,
R(t)
R(t)

are equivalent. That is, corresponding to the alternative taxation patterns, there
is a pattern of government borrowing such that the initial equilibriums consumption allocation and the initial equilibrium gross interest rates constitute an
equilibrium under the alternative taxation pattern.
Note that both of these propositions imply neutrality of government policy.
If taxes are changed in a particular way (i.e. if they change so that each agent
64

faces the same present value of taxes), the government policy has no real eect
(i.e. allocations and the interest rate remain the same). These results are usually
referred as Ricardian Equivalence results. In the next section we will show that
if agents care about others (i.e. they are altruistic), then we can arrive at even
more general neutrality results.

4.5

Intergenerational Linkages

Since government policies we have considered so far redistribute recourses between young and old that are alive at a point in time or between young and
old that are alive in dierent periods, it is very important to ask the following
question: what are the eects of government policies, if generation care about
each others welfare (i.e. they are altruistic)?
To analyze this question, consider the following form of altruism: Generation0 cares about the well-being of generation-1, and generations 1,2,.... are all
selsh. Hence,
uh0 = uh0 ch0 (1); uh1 (ch1 (1); ch1 (2)) ;
where uh1 (ch1 (1); ch1 (2)) is the lifetime utility of generation 1. Hence, generation0 can choose to transfer some of his/her resources to generation-1. Let bh (0)
be the bequest of generation-0 to generation-1. Note that since generation-1 is
selsh, he will simply take bh (0) as an addition to his/her endowments. The
budget constraint faced by generation-0 is now given by
ch0 (1) = woh (1)

h
0 (1)

bh (0);

and those for generation-1 are given by

ch1 (1) = w1h (1)

h
1 (1)

l1h + bh (0);

and
ch1 (2) = w1h (2)

h
1 (2)

+ R(1)l1h :

Now consider policies of the following kind

N (0)

N (1)
h
0 (1)

h=1

h
1 (1)

= 0;

h=1

where resources are transferred between generation 0 and 1 at time 1. Lets

focus on one agent. Suppose there is no heterogeneity, i.e. the autarky is the
competitive equilibrium. Then the utility of generation 0 can be written as
u0 (w0 (1)

b(0) +

0 (1); u1 (w1 (1)

+ b(0)

and the optimality condition for bequest is given by

@u0
@co (1)
| {z }

m arginal cost

@u0 @u1
:
@u1 @c1 (1)
|
{z
}

m arginal b enet

65

1 (1); w1 (2));

c1(1)

C
b
w1 (1) 1 (1)

slope = -1

b
w1(1)

not allowed
w0 (1)

c0 (1)

w0 (1) - 0 (1)

Figure 11: Bequest Decision

Figure 3 shows the trade-o that is faced by generation 0. Given an aftertax endowment point, generation-0 decides how much bequest to leave. We
restrict bequest to be non-negative. Given preferences of generation-0, we can
determine the optimal bequest level. Note that it is possible that bequest motive
is not operative, i.e. b(0) = 0: The bequests will be zero, if
@u0
@u0 @u1
>
:
@co (1)
@u1 @c1 (1)
Now consider changes in taxes and transfers in Figure 3. Note that as long as
taxes and transfer do not change the location of point C, the bequest will be
reduced or increased by generation 0 one to one with taxes. Hence, as long as
bequest motive is operative (i.e. the old choose to leave bequests), the taxes
and transfers that redistribute income between old and young at time 1 has no
eect on real allocations. The taxes and transfers will have real eect only if
they force the old not to leave any bequests.
In Figure 4, the old consume their after tax endowments (point D) and do
not leave any bequests. Indeed they would like to receive transfers from the
young (i.e. choose b(0) < 0); but that is ruled out.
Note that without altruism, only taxes that keep each agents lifetime tax
liabilities constant have no real eect. With altruism government taxes that
transfers resources from the current old to the current young might have no
66

c1(1)

w1 (1) 1 (1)

C
slope = -1

b
w1(1)
w0 (1)

c0 (1)

w0 (1) - 0 (1)

Figure 12: Bequests are Zero

67

real eect, as long as the bequest motive is operative. Furthermore, government borrowing from the current old (which is nanced by taxing the young
in the future) might not have any real eect as well. See Aiyagari (1993) and
Barro (1974) for a detailed discussion of eects of intergenerational linkages on
government policies in OLG models.

4.6

Production

Consider an OLG economy with identical agents who live for two periods. When
young, each agent has 1 unit of labor that he/she supplies inelasticly and earn
wages, wt : Given wt ; each agent decides how much to consume, ct ; and how
much to save, st : When old they rent their savings as capital to the rm which
pays them back 1 + rt+1 for every unit rented. Hence, in this model agents can
store their goods and keep it for the next period as capital. Capital depreciates
at rate 2 (0; 1) after production. Moreover assume that population grows at
rate n, initial population size is N0 ; i.e. Nt = (1 + n)t N0 ; and initial old are
endowed with k1 units of per capita capital at time 1.
Agents discount the future, i.e. consumption when old, at rate : Their
lifetime utility is given by
u(c1t ) + u(c2t+1 );

< 1; u0 > 0; u00 < 0:

There is a large number of rms that has access to the aggregate production
technology represented by
Y = F (K; L);
where Y is output, K is capital used and L is labor used: The production
function F is called a neoclassical production function if it has positive rst and
negative second derivative w.r.t. each argument
@F
@2F
@2F
@F
> 0;
> 0;
<
0;
< 0;
@K
@L
@K 2
@L2
it is constant returns to scale (CRS), i.e.
F ( K; L) = F (K; L); for all

> 0;

and it satises Inada conditions

lim FK = lim FL = 1; and lim FK = lim FL = 0:

K!0

L!0

K!1

L!1

Since the production function is CRS, we can assume that there is only one
representative rm. The rms objective is to maximize prots, i.e. rms
problem is
max Y (K; L) wL rK:
L;K

68

Hence, the FOCs for this problem are

w=

@Y
;
@L

and

@Y
;
@K
which determine competitive input prices.
Moreover, since the production function is CRS, the prots are zero. This
results from the Eulers theorem which states that if f (x; y) is homogenous of
degree one, then
@f
@f
x+
y:
f (x; y) =
@x
@y
r=

Finally, since the production function is CRS, we can represent per capita
output as a function of per capita capital stock, since:
Y = F (K; L) = LF (

K
K
; 1) = Lf ( );
L
L

hence

Y
= y = f (k):
L
Therefore, marginal products of capital and labor can be found as
@Y
K 1
= Lf 0 ( ) = f 0 (k);
@K
L L

and

4.6.1

@Y
K
K
K
= f ( ) + Lf 0 ( )(1) 2 = f (k)
@L
L
L
L

kf 0 (k):

Individuals problem

Each individual in generation-t; t

1; solves

max u(c1t ) + u(c2t+1 );

subject to
c1t + st = wt ;
and
c2t+1

= (1
)st + rt+1 st
= (1 + rt+1
)st :

The FOC for this problem is given by

u0 (c1t ) = (1 + rt+1

69

)u0 (c2t+1 )

Note that this FOC denes implicitly a savings function

st = s(wt ; rt+1 );
which can be analyzed, using Implicit Function Theorem, to determine how
savings change as wt and rt+1 changes.
Theorem 60 (Implicit Function Theorem) Consider an equation of the following form
F (y; x1 ; x2 ; :::::; xm ) = 0;
(46)
that denes an implicit function of y
y = f (x1 ; x2 ; ::::; xm ):
If (a) the function F has continuous partial derivatives Fy ; Fx1 ; ::::; Fxm ; (b)
at a point (y0 ; x10 ; ::::; xm0 ) satisfying (46), Fy is nonzero, then there exists an
m-dimensional neighborhood of (x10 ; ::::; xm0 ); call it N; in which y is an implicit
function of x1 ; :::; xm : This implicit function satises y0 = f (x10 ; ::::; xm0 ): It
also satises (46) for every m-tuple in the neighborhood N; hence giving (46)
the status of an identity. Moreover, the implicit function f is continuous and
has continuous partial derivatives. In the neighborhood N
Fy dy + Fx1 dx1 + :::: + Fxm dxm = 0:
Hence, we can dene
G( ; st ( )) = u0 (wt

st )

and nd
swt =

@st
=
@wt

Gw t
> 0;
G st

srt =

@st
=
@rt

Gr t
7 0:
G st

and

4.6.2

(1 + rt+1 )u0 ((1 + rt+1 )st ) = 0;

Firms

Firms try to maximize prots, and labor and capital markets are competitive,
hence we will have the following FOCs
wt = f (kt )

kt f 0 (kt );

and
rt = f 0 (kt ):

70

4.6.3

Determination of kt

The capital stock next period is given by the total savings this period, i.e.
Kt+1 = Nt s(wt ; rt+1 );
or
(1 + n)kt+1 = s(wt ; rt+1 ) = s(wt (kt ); rt+1 (kt+1 );
kt f 0 (kt ); f 0 (kt+1 )]
:
1+n
Since the last equation relates current and next period levels of per capita
capital it determines how this economys per capita capital stock evolves given
any initial value of per capita capital stock, k0 :
kt+1 =

s [f (kt )

Remark 61 The dynamic properties of fkt gt=1 depends on the derivative

sw (kt )kt f 00 (kt )
dkt+1
;
=
dkt
1 + n sr (kt+1 )f 00 (kt+1 )

Note that if sr > 0; this derivative is positive.

Remark 62 Furthermore, given a steady state level of per capita capital stock
k = kt = kt+1 ; such a steady state will be stable if
0<

sw (k )k f 00 (k )
< 1:
1 + n sr (k )f 00 (k )

Given this set-up we can dene an equilibrium as:

Denition 63 Given a population growth rate n and time-1 capital per worker
k1 ; a sequential competitive equilibrium consists of a sequence of per capita capi1
1
1
tal stock fkt gt=1 ; sequence of prices frt gt=1 ; fwt gt=1 ; and sequence of decisions
1
by the agents, fc21 ; c1t ; c2t+1 ; st gt=1 such that
given prices, agentss solve their maximization problems for all t > 1,
rt and wt are given by rms optimization problem for all t > 1,
1

fkt gt=1 satises for all t > 1

(1 + n)kt+1 = s(wt ; rt+1 ) = s(wt (kt ); rt+1 (kt+1 ):
Remark 64 Note that we could let the rms output be
Y = F (K; L) + (1

)K:

In this case, the rm takes the capital of the old and pays
@F
:
@K
Then sells the production and the undepreciated part of the capital to the consumers. This would not change anything since the decisions of the young and
the old would be the same as above.
R=1+

71

4.6.4

An Example

Consider the following version of the economy outlines above, with n = 0; and
Lt = N for all t. Agents are identical and have the following preferences
u(cyt ; cot+1 ) = log(cyt ) +

log(cot+1 );

except with initial old having the following preferences

u(co1 ) = co1 :
When young, agents can work and save in the form of a physical capital,
when old they cannot work but receive a rental return by renting their capital.
Capital depreciates at rate : There exist a single rm that produces each period
by combining labor inputs from young and capital inputs from old according to
the following technology
Yt = AKta L1t a :
First consider the rms problem
max AKta L1t

wt Lt

Kt ;Lt

rt kt ;

where wt is the wage rate and rt is the user cost of capital. Then, FOCs are
wt = (1

a)AKta Lt

= (1

a)A

Kt
Lt

= (1

a)A

and
rt = aAKta

L1t

a 1

= aA (kt )

Kt
N

= (1

Agentss problem for generations 1,2,3,....

max log(cyt) +

cyt ;cot+1

log(cot+1 )

subject to
cyt + st = wt ;
and
cot+1 = (1 + rt+1

)st :

The life time budget constraint for an agent is then given by

cyt +

cot+1
1 + rt+1

= wt:

FOCs are then given by

1
= ;
cyt
and

1
1
=
cot+1
1 + rt+1
72

a)A (kt ) ;

Then,
cot+1 = cyt (1 + rt+1

):

And, using the lifetime budget constraint we get

cyt + cyt = wt ) cyt =

1
wt ) st = wt cyt =
1+

1
1+

wt =

1+

wt :

For the initial old

max co1 ;
co1

subject to
co1 = (1 + r1

)k1 ;

where k1 is the initial capital stock.

Finally goods market clear
N cyt + N cot + N st = AKta L1t

+ (1

)Kt ; for t

1;

and the savings market clear

N st = N (wt

cyt ) = Kt+1 for t

1:

From savings market clearance we have

st

=
)

1+
1+

wt = kt+1
(1

a)A

kt
lt

= kt+1 ) kt+1 =

1+

(1

a)A (kt ) :

Hence, equilibrium value of fkt gt=1 must satisfy

kt+1 =

1+

(1

a)A (kt ) ; given any k1 :

This economy has two steady states where kt+1 = kt = k (see Figure 13):
One is where kt = 0 for all t: Other one is
1

k=

4.7

1+

(1

a)A (k) ) k =

1+

(1

a)A

Introducing Government

We can introduce dierent government policies into this set-up.

73

450

kt+1

k*

k0

kt

k*

Figure 13: Dynamics of kt

4.7.1

Pay-as-You-Go Social Security System

A pay-as-you-go social security system simply taxes the young and the transfers
those resources to the current old. If d is payments to the social security system
by the current young and the b is the benets received by the current old, then
the contributions and benets are related by
b = (1 + n)d:
Hence, the problem of an agent born at time becomes
max u(c1t ) + u(c2t+1 );
subject to
c1t + st = wt

d;

and
c2t+1

4.7.2

=
=

(1
)st + rt+1 st + (1 + n)d;
(1 + rt+1
)st + (1 + n)d:

Income Taxes

We could also introduce income taxes. Suppose w is the proportional tax

on labor income and k is the proportional tax on capital income. Then, the
74

problem of an agent born at time becomes

max u(c1t ) + u(c2t+1 );
subject to
c1t + st = (1

w )wt ;

and
c2t+1 = st + (1

k )(rt+1

)st ;

where the government taxes the net return on capital.

Suppose government uses this income to nance an exogenous sequence of
1
government spending, fGt gt=1 ; then the government budget is balanced in
period t; if
)st 1 = Gt :
w wt + k (rt
Furthermore, if we allow the government to borrow from the young, then the
government budget at time t is
w wt

k (rt

)st

+ B(t) = Gt + B(t

1)R(t);

where B(t) and B(t 1) are government borrowing at time t and time t + 1;
and R(t) = 1 + rt
: Note that government has to pay young the return they
would get buy holding into their goods and renting to the capital.

4.8

Dynamic E ciency

In OLG economies with production, we should also worry if the economys

savings decision is e cient. In order to look at this issue, consider the following
planners problem
max U = u(c21 ) +

1
X

[u(c1t ) + u(c2t+1 )] ;

t=1

subject to
(1

)kt + f (kt )
k1

(1 + n)kt+1 + c1t +

c2t
;
(1 + n)

> 0; given.

The FOCs associated with this problem are given by

u0 (c2t ) = (1 + n)

1 0

u (c1t );

for c2t ; and by

(1 + n)u0 (c1t

1)

= (1 + f 0 (kt )

for kt :

75

)u0 (c1t );

The rst FOC characterizes the allocation of resources between two people who are alive at time t; while the second FOC characterizes the optimal
accumulation decision.
Note that this problems FOCs imply the following steady state values for
c1 ; c2 ; and k :
u0 (c2 ) = (1 + n) 1 u0 (c1 );
(47)
and
1

+ f 0 (k ) = (1 + n):

(48)

The last equation determines the steady state value of per capita capital stock:
f 0 (k ) = n + :
This relation is called the golden rule of capital accumulation and characterizes
the e cient steady state capital stock.
Note that in this steady state
(1
where c = c1 +

c2
(1+n) :

)k + f (k ) = k + nk + c ;

Hence,
f (k )

nk

k =c ;

and the capital stock that maximize the consumption of a representative agent
at the steady state is given by
dc
= f 0 (k )
dk

= 0:

Therefore,
dc
= f 0 (k ) n
? 0 () f 0 (k ) ? n + ;
dk
hence, if f 0 (k ) > n+ ; i.e. capital stock exceeds the golden rule level, a decrease
in capital stock will increase the steady state level of per capita consumption.
Note that if f 0 (k ) < n + ; the economy is over accumulating capital so that
technology does not return what is necessary to keep the per capita capital stock
constant see Abel et. al. (1989) for further discussions.

References
[1] Abel, Andrew B., Mankiw, N. Gregory, Summers, Lawrence, and Zechauser,
R. J. Assessing Dynamic E ciency: Theory and Evidence, Review of
Economic Studies, 56, pages 1-20, 1989.
[2] Aiyagari, Rao S. Intergenerational Linkages and Government Budget Policies. Federal Reserve Bank of Minneapolis Quarterly Review (Spring
1987): 14-23.
76

[3] Barro, Robert (1974), Are Government Bonds Net Wealth? Journal of
Political Economy, 82, 1095-1117.
[4] Diamond, Douglas (1965), National Dept in a Neo-Classical Growth
Model, American Economic Review, 55, 1126-1150.
[5] Kehoe, T. Intertemporal General Equilibrium Models, in The Economies
of Missing Markets, Information, and Games, Frank Hahn (ed.), 1989.
[6] Samuelson, P. A. (1958), An Exact Consumption-Loan Model of Interest
with or without the Social Contrivance of Money, Journal of Political
Economy, 66, 467-82.

77

Dynamic Programming

Agents live for a nite number of periods in OLG economies. As we have

already pointed out if agents are altruistically linked in an OLG economy, they
will behave as if they live forever. We will now analyze models in which agents
solve an innite horizon problem.
Our starting point will be the planning problem of a single agent who lives
forever. Later we will talk about market allocations. The problems we will
analyze have the following form:
sup

1
X

fxt+1 g1
t=0 t=0

F (xt ; xt+1 );

(SP)

subject to
xt+1 2 (xt ); t = 0; 1; 2; :::;
and
x0 given.
In this set-up, time is discrete and the horizon is innite, t = 0; 1; 2:::: At
time-0, the economy starts with x0 : Given x0 ; the agent chooses x1 from a
feasible set (x0 ): The x0 together with the choice of x1 determine the current
return, denoted by F (x0 ; x1 ): At time-1, the economy starts with x1 and the
agent chooses x2 ; etc. The future is discounted at rate 2 (0; 1):
1
P1Thetproblem is nding an innite feasible sequence fxt+1 gt=0 that maximizes
F (xt ; xt+1 ): In this part of the class we will try to nd and charactert=0
ize solutions to these sequential problems. A sequential problem (SP) will be
characterized by three objects:
A set X such that xt 2 X for all t:
A set of feasible actions that assigns for each x 2 X a subset of X,
X ! X:

A return function that maps any two elements from X into the real line,
F : X X ! R:

5.1

Neoclassical Growth Model

We will start by analyzing the neoclassical growth model. Consider a production

economy (i.e. one in which the good is storable and can be used for consumption
or production). The economy is populated by a large number of identical agents,
each starting his/her life with k0 units of capital. There is no population growth.
We will focus on a planing problem faced by a single representative agent.
Each period this agent can combine the capital stock and his/her labor, denoted
by nt ; to produce a single good, denoted by yt . Imagine this as a backyard
technology. We will later see how/why the solution to this representative agent
problem is identical to the market allocations.
78

The production technology is represented by

yt = F (kt ; nt );
where F is a neoclassical production function. Hence, F (k; n) is continuously
dierentiable, strictly increasing, concave, and CRS, i.e.
Fk (k; n) > 0; Fn (k; n) > 0 for all k; n > 0;
F( e
k+(1

)b
k; n
e+(1

F (e
k; n
e)+(1

)b
n)

)F (b
k; n
b) for all e
k; n
e; b
k; n
b > 0 and

F (k; n) = F ( k; n) for all

> 0:

It also satises F (0; n) = 0 and the following Inada conditions

lim Fk (k; 1) = 1; and lim Fk (k; 1) = 0:

k!0

k!1

The output together with the undepreciated part of the current capital can
be used for consumption, denoted by ct ; or be kept as future capital stock,
denoted by kt+1 : Therefore,
ct + kt+1
where

F (kt ; nt ) + (1

)kt ;

is the depreciation rate. We will let

f (kt ) = F (kt ; nt ) + (1

)kt ;

denote the total resources available at time t:

Finally, the representative agent has the following time separable utility
function
1
X
1
t
U (fct gt=0 ) =
u(ct );
t=0

where u(ct ) is strictly increasing and strictly concave, with limc!0 u0 (c) = 1:
Hence, the problem of a representative agent is
max

fct ;nt ;kt+1 g1

t=0

1
X

u(ct );

t=0

subject to
ct + kt+1
k0
ct

f (kt );

> 0 is given,
0; kt+1 0
and 1

nt

0:

We already know several things about this problem:

79

2 (0; 1);

Since agents do not drive any utility from leisure, they will supply all of
their labor endowment, i.e. nt = 1 for all t: Therefore, f (kt ) = F (kt ; 1) +
(1
)kt :
Since the utility function is strictly increasing, it must be the case that
ct + kt+1 = f (kt ):
Then, we can write this problem as:
max1

fkt+1 gt=0

1
X

u(f (kt )

kt+1 );

t=0

subject to
k0 > 0 is given, and f (kt )

kt+1

0;

where we have substituted ct = f (kt ) kt+1P

: Hence, the problem is to nd an
1
1
innite sequence fkt+1 gt=0 that maximizes t=0 t u(f (kt ) kt+1 ):
Note that the neoclassical growth model ts our general description of a
sequential problem, since we can write:
F (xt ; xt+1 ) = F (kt ; kt+1 ) = u(f (kt )

kt+1 );

and
(xt ) = (kt ) = [0; f (kt )]:
Remark 65 What about the set X?
How can we solve this problem? In order to gain some insight, rst consider
a nite-time-horizon version of this problem, given by:
max1

fkt+1 gt=0

T
X

u(f (kt )

kt+1 );

t=0

subject to
k0 > 0 is given, and f (kt )

kt+1

0:

We can write down the Lagrangian for this problem as:

L=

T
X

[u(f (kt )

kt+1 ) +

t kt+1 ] ;

t=0

where

is the Lagrange multiplier associated with the constraint kt+1

0:

Remark 66 Note that in this problem it will never be optimal to set kt+1 =
f (kt ):

80

The rst order conditions for this problem are given by:
@L
=
@kt+1
and

t 0

u (f (kt )

kt+1 ) +

@L
=
@kT +1

u0 (f (kT )

t+1 0

u (f (kt+1 )

kT +1 ) +

kt+2 )f 0 (kt+1 ) = 0;

= 0:

Furthermore, we have the following Kuhn-Tucker conditions:

t kt+1

kt+1

0; t = 1; :::; T;
0; t = 1; :::; T;
0; t = 1; :::; T:

Now we can characterize the solution:

Since u0 (f (kT ) kT +1 ) = u0 (cT ) > 0 and u0 (f (kT ) kT +1 ) = T ; we
know that T > 0: Then, it is the case that kT +1 = 0: Therefore, one
important piece of information in a nite horizon problem is that agents
will consume everything in the last period.
We also know that t = 0 for all t < T: If this was not the case (i.e. if t
was strictly positive for some t), kt+1 must be zero for some t < T: This
can not be optimal.
Then, we have, for t = 1; 2; :::; T

u0 (f (kt ) kt+1 ) = u0 (f (kt+1 ) kt+2 )f 0 (kt+1 ):

|
{z
}
{z
}
|
utility cost of
discounted utility benet
saving more kt+1
of having more kt+1
next period.
This equation is called an Euler equation. It characterizes kt+1 given kt and
kt+2 : Since in this problem k0 > 0 is given and kT +1 is 0, it provides us with a
complete characterization of the optimal path.
Example 67 Consider the following version of the neoclassical growth model
where
u(ct ) = log(ct );
and
f (kt ) = kt ; with

2 (0; 1):

The Euler equation is then given by

u0 (ct ) = u0 (ct+1 )f 0 (kt+1 );
or
1
=
ct

kt+11
:
ct+1
81

Hence,
ct+1 =

kt+11 ct ;

kt+2 ) =

kt+11 (kt

or
(kt+1
Note that at time T

kt+1 ):

1 this equation becomes

(kT

kT +1 ) =

kT

Since kT +1 = 0;
kT =

kT

(kT

(kT

kT ):

kT );

and we can nd kT as a function of kT 1 and work our way back to k1 as a

function of k0 (see Exercise 2.2 in Stokey, Lucas with Prescott (1989)):
Now we will look at an innite horizon problem. Note that in a nite-horizon
problem the terminal condition, kT +1 = 0; gives us with valuable information
to solve our problem. Hence, a sequence that satises the Euler equations
together with this terminal condition provided us with a solution. It turns
out that a similar condition is required for an innite horizon problem. This
condition is called the transversality condition. The following proposition shows
1
that any interior innite sequence fkt+1 gt=0 that satises the Euler equation and
the transversality condition is a solution to SP.
Proposition 68 (Su ciency) Consider
max1

fkt+1 gt=0

1
X

F (kt ; kt+1 );

t=0

subject to
k0 > 0 given; and kt+1

0:

Let F be continuously di erentiable, and let F (x; y) be concave in (x; y) and

1
strictly increasing in x: If kt+1 t=0 satises:
kt+1 > 0; for all t:
F2 (kt ; kt+1 ) + F1 (kt+1 ; kt+2 ) = 0 ; for all t:
limt!1

F1 (kt ; kt+1 )kt = 0;

Then, kt+1

1
t=0

maximizes the objective function.

Remark 69 Note that for one sector growth model where F (kt ; kt+1 ) = u(f (kt )
kt+1 ); the second condition implies
u0 (f (kt )

kt+1 )( 1) + u0 (f (kt+1 )

kt+2 )f 0 (kt+1 ) = 0;

or
u0 (f (kt )

kt+1 ) = u0 (f (kt+1 )
82

kt+2 )f 0 (kt+1 ):

n
o1
Proof : Consider an alternative feasible sequence e
kt+1
: We want to
t=0
show that
T
h
i
X
t
F (kt ; kt+1 ) F (e
D = lim
kt ; e
kt+1 )
0:
T !1

t=0

Since F is concave,
D

lim

T !1

lim

T !1

T
X1

t=0

T
X
t=0

F (kt ; kt+1 )

i
F (e
kt ; e
kt+1 )

e
kt ) + F2 (kt ; kt+1 )(kt+1

F1 (kt ; kt+1 )(kt

Note that k0 = e
k0 = k0 > 0 is given. Then,
3
2
T
X 6
7
t
D
lim f
4F2 (kt ; kt+1 ) + F1 (kt+1 ; kt+2 )5 (kt+1
T !1
{z
}
|
t=0
=0

e
k0 ) +
}

+F1 (k0 ; k1 )(k0

|
{z
=0

i
e
kt+1 ) :

e
kt+1 )

e
kT +1 )g:

F2 (kT ; kT +1 )(kT +1

Since F2 (kt ; kt+1 ) + F1 (kt+1 ; kt+2 ) = 0 for all t;

D

F1 (kT ; kT +1 )(kT

lim

F1 (kT ; kT +1 )kT = 0;

T !1

where we used limT !1

e
kT 0:

e
kT )

lim

T !1

F1 (kt ; kt+1 )kt = 0; and the fact that F1 > 0 and

Remark 70 Note that if f (x) : D ! R is di erentiable, then f is concave if

and only if
Df (x)(y x) f (y) f (x) for all x; y 2 D:
Remark 71 Note that for one sector growth model
F (kt ; kt+1 ) = u(f (kt )

kt+1 ) = u(F (kt ; 1) + (1

)kt

kt+1 );

must be concave in (kt ; kt+1 ):

What is the meaning of the transversality condition, limt!1 t F1 (kt ; kt+1 )kt =
0? It simply states that it can not be optimal to choose a capital sequence such
that the discounted present value of kt , i.e. t F1 (kt ; kt+1 )kt ; is positive as t
goes to innity. This is not optimal, since then the agent is saving too much.

83

5.2

Dynamic Programming: An Introduction

The previous section focused on nding solutions to the following problem

max1

fkt+1 gt=0

1
X

u(f (kt )

kt+1 );

t=0

subject to
k0 > 0 is given, and f (kt )

kt+1

0:
1

We looked at solutions in terms of innite sequences, fkt+1 gt=0 ; and provided

su cient conditions that an optimal sequence must satisfy.
There is an alternative and easier way to attack innite-horizon problems,
called dynamic programming. Lets start again from a nite horizon problem.
Consider the following problem
max1

fkt+1 gt=0

T
X

ln(kt

kt+1 ); with k0 given.

t=0

Suppose now we are at period T 1 and have some capital stock k: Then the
only problem is to choose tomorrows capital stock. Lets denote it with k 0 : Since
I exactly know what I will do with k 0 tomorrow, the problem of choosing k 0 is
simply
fln(k k 0 ) + ln(k 0 )g :
max
0
k

The solution is given by

1
k
or

k0

1
;
k0

k
:
1+

k0 =
We will call this period T

1 policy function:
gT

1 (k)

k
:
1+

It tells us what we will do (i.e. how much we will save if we enter the period
with a capital stock k): Once we know this we can also calculate the maximized
value function for period T 1
VT

=
=

k
)+
1+

k
)
1+
1
(1 + ) ln(k) + ln(
) + ln(
):
1+
1+

ln(k

ln(

It tells us the highest utility level we can reach if we enter the period T 1 with
k: Once we know VT 1 ; however, gT 2 and VT 2 can be determined by
VT

= max
fln(k
0
k

84

k 0 ) + VT

1 (k

)g

Indeed for any period t = 0; ::::; T 1; the problem boils down to nding
value functions and associated policy functions that satisfy
k 0 ) + Vt+1 (k 0 )g :

Vt = max
fln(k
0
k

The key feature of the dynamic programming approach is to split the problem
into several problems that involve today and tomorrow.
Remark 72 Note that VT (k) = ln(k) and gT (k) = 0 are determined trivially.
Remark 73 The dynamic programming approach focuses on nding the policy
functions and the value functions rather than nding the sequences.
Remark 74 As the following example shows a pattern often emerges in Vt and
gt that allows us to use an induction argument.
Example 75 Consider the following problem
max1

fkt+1 gt=0

1
X

ln(ct );

t=0

subject to
k0
kt+1
and kt+1

> 0 is given,
= (kt ct )R; with R
0 and ct 0:

1;

How should we write this as a DP problem? Note that it makes sense to write
the value and policy functions as functions of k: If the agent knows k at period
t; then he/she can determine k 0 (the next periods asset). Again lets start from
period T 1: We know that VT = ln(k): Hence,
VT

= max
ln(k 0
0
k

k0
) + ln(k 0 ) :
R

Then, the FOC for k 0 is

1
k0

Rk
For t = T

1
=) gT
k0

Rk
and VT
2

k
= 2 ln( ) + ln(R):
2

2; we have
VT

= max
ln(k
0
k

k0
k
) + 2 ln( ) + ln(R) :
R
2

Now the FOC for k 0 is

1
Rk

k0

2
=) gT
k0

2Rk
and VT
3
85

k
= 3 ln( ) + 3 ln(R):
3

It is easy to guess now

T

Vt = (T

t + 1) ln(

X
k
)+
i ln(R);
t+1
i=1

and

T t
Rk:
T t+1
Suppose now for 2 ft + 1; :::; T g the value and policy functions are given by
these equations. Then,
(
)
TX
t 1
k0
k
Vt = max
ln(k
) + (T t) ln(
)+
i ln(R) ;
k0
R
T t
i=1
gt =

and the FOC for k 0 is given by

1
k0

Rk

= (T

t)

1
(T t)
or k 0 =
Rk;
k0
(T t + 1)

and
Vt

ln(k

ln(

(T

(T t)
k) + (T
(T t + 1)
k
) + (T
t + 1)

t) ln(

t) ln(

(T

(T

t + 1) ln(

(T

TX
t 1
1
Rk) +
i ln(R);
t + 1)
i=1

k
) + (T
t + 1)

t) ln(R) +

TX
t 1

i ln(R)

i=1

X
k
)+
i ln(R);
t+1
i=1

which completes our backward induction.

In an innite horizon problem, the dynamic programming approach uses the
stationary property of innite horizon problems to nd a solution. Note that
in an innite horizon problem, the problem today and tomorrow are identical.
The only thing that changes from today to tomorrow is the value of the current
capital stock.
In order to gain some further insight, let V (k0 ) be the maximized value
function associated with one sector growth model, i.e.
V (k0 )
s:t: k0

max1

fkt+1 gt=0

1
X

u(f (kt )

kt+1 );

t=0

> 0 is given, and f (kt )

kt+1

0:

The maximized value function is the highest level of discounted utility we can get
starting with some initial level of capital and follow the best possible sequence
of actions. It is a function that maps the set of capital stocks into the real line.
86

Note that we can write this problem as

max

fkt+1 g1
t=0

max

1
X

u(f (kt )

kt+1 )

t=0

0 k1 f (k0 )

fu(f (k0 )

"

k1 ) +

max

kt kt+1 f (kt )

1
X

t 1

u(f (kt )

t=1

kt+1 ) g:

Now if V (k0 ) is the maximized value function, then it also gives us the maximum
value starting with k1 :
Then, we have
V (k0 ) =

max

0 k1 f (k0 )

[u(f (k0 )

k1 ) + V (k1 )] :

But time does not play any particular role here other than indicating today vs.
tomorrow.
Hence, if V is the maximized value function, it should satisfy
=

V (k)
| {z }

max

0 k0 f (k)

[u(f (k)

k 0 ) + V (k 0 )] ;

value function

where k and k 0 denote todays and tomorrows capital stock.

If we can nd V (we do not know yet how, we also do not know yet if a
solution is possible), then we have a complete characterization of the optimal
sequence of capital stock.
For any value of k; we can then dene
g(k)
|{z}

= arg

p olicy function

max

k0 2[0;f (k)]

[u(f (k)

k 0 ) + V (k 0 )] :

Once, we know g(k); we can start from k0 ; and characterize the optimal path.
Hence, the dynamic programing approach focuses on nding functions such as
V and g; rather than nding sequences. Of course the question is: how do we
nd V ?
Sometimes a guess and verify approach works as the following example
demonstrates:
Example 76 Let
u(c) = ln(c);
and
f (k) = k :
Then, we can write the following dynamic programming problem
V (k) =

max [ln(k

0 k0 k

87

k 0 ) + V (k 0 )] :

(49)

Suppose we have the following guess

V (k) = A + B ln(k):
Once we have this guess, we have the following maximization problem
k0 ) +

max [ln(k

0 k0 k

(A + B ln(k 0 ))] :

Note that once we have a functional form for V; this maximization problem is
trivial. Now we can nd the FOC for k 0
1
k0

B
:
k0

Hence,
k 0 = Bk

k 0 B;

or

Bk
:
(50)
1+ B
If V (k) is the true value function that solves equation (49) then if we follow g(k)
dened in equation (50) we should get back V (k); i.e.
k 0 = g(k) =

Bk
)+
1+ B

A + B ln(k) = ln(k

A + B ln(

Bk
)
1+ B

One can arrange these terms to arrive at

B=
and
A=

1
1

1
ln(

) + ln (1

) :

Hence, as we have guessed A and B are constants. Once we have V , we can

also nd the optimal policy function
k 0 = g(k) =

Bk
=
1+ B

k :

Remark 77 Note that this policy rule satises the transversality condition,
since for this problem
lim

t!1

F1 (kt ; kt+1 )kt

=
=
=
=

lim

lim

lim

lim

1
(kt )

t!1

a 1

kt+1
1

a 1

(kt )

t!1

t!1

88

(kt )
1

(kt )

t!1

kt

(kt )

(kt )
a

(kt )
= 0:

(kt )

kt

Remark 78 Check Exercise 2:3 in Stokey, Lucas with Prescott (1989) to see
how you can arrive the guess for V using the policy rule for a nite horizon
problem.
It is obvious that the cases where we can have a nice guess that will turn
out to be true V are rather limited. What can we do, if we have no idea what
V is?
We can still hope that starting from some initial guess V 0 (k) might help.
For this problem, for example, let
V 0 (k) = 0:
Then, we can nd a new function for V; call it V 1 (k); by solving
V 1 (k) =

k 0 ) + V 0 (k 0 ) :

ln(k

max

0 k0 k

Again once we have V 0 (k) = 0; solving the maximization problem in the right
hand side is trivial. Since V 0 (k) = 0; the solution is k 0 = 0: Then we have
V 1 (k) = ln(k ) =

ln(k):

By any chance if we got V 1 (:) = V 0 (:); as we did in the previous example, we

were done.
Lets now use this as a new guess to arrive at V 2 (k) :
V 2 (k) =

max

ln(k

0 k0 k

k 0 ) + V 1 (k 0 ) :

Again, once we have a guess for V on the right hand side, we can write the FOC
for k 0 as
1
=
k
k0
k0
to arrive at
k
k0 =
;
1+
and
V 2 (k) = ln k

k
1+

ln

k
1+

Now we can dene V 3 ; using V 2 ; etc.

Our hope is that this sequences of functions will converge to some function
V , which is the true function. Why should such a procedure work? The hope is
that as we start from some initial guess V 0 and update the function, we reduce
the error in V; hence approaching to the true value.
Note that the dynamic programming problem
V (k) =

max [ln(k

0 k0 k

89

k 0 ) + V (k 0 )] ;

denes an operator on functions, which can be written as

(T V )(k) =

max [ln(k

0 k0 k

k 0 ) + V (k 0 )];
{z
}
T

where (T V ) is a new function given V:

Then, our hope is that the sequence of unctions that dened by this operator
converges to a limit function V
fV n (k)g ! V;
and furthermore,
(T V ) = V;
i.e. V is a xed point of operator T:
We will now try to make things more precise. In particular we will make
clear:
What do we mean by a sequence of functions being converged?
Under what condition does the operator T generates a sequence of functions that converge?
Under what conditions, if fV n (k)g is a sequence of functions with certain
properties, will the limit function V also have those properties?
The next section provides some mathematical preliminaries to make these
claims more precise. Once we have the tools to solve for V; we will do two
things:
show that V provides a solution to SP,
characterize V and g:
Before, going into details, note that we have started this section with the
following sequential problem
sup
fxt+1 g1
t=0

1
X

F (xt ; xt+1 );

(SP)

t=0

subject to
xt+1 2 (xt ); t = 0; 1; 2; :::;
and
x0 given.
Now we are interested in solving the following functional equation (FE) to
nd V;
V (x) = sup [F (x; y) + V (y)] ;
(FE)
y2 (x)

where x is todays state and y is todays choice (and tomorrows state).

90

Mathematical Preliminaries

This section is based on Stokey, Lucas with Prescott (1989), Sundaram (1996),
Rudin (1976). We will proceed in following steps:
We will rst dene a set of objects that we can compare with each other
in a meaningful way.
This will allow us to dene convergence.
Then we will focus on sequences of functions and dene dierent convergence notions for sequences of functions.
We will then look at operators, such as the operator T we dened above; that
generate sequence of functions that converge and T V preserves nice properties of function V .

6.1

Vector and Metric Spaces

Denition 79 A real vector space (or linear space) is a set of elements (vectors)
X together with two operations, addition and scalar multiplication (which are
dened as x + y 2 X and ax 2 X for any two vectors x; y 2 X and for any real
number a) such that for all x; y; z 2 X and a; b 2 R
x+y
(x + y) + z
a(x + y)
(a + b)x
(ab)x
9
1x

=
=
=
=
=
2
=

y + x;
x + (y + z);
ax + ay;
ax + bx;
a(bx);
X; x + = x and 0x = ;
x:

Denition 80 A metric space is a set S; together with a metric (distance function) : S S ! R; such that for all x; y; z 2 S :
(i) (x; y)
0; with
equality if and only if x = y; (ii) (x; y) = (y; x); (iii) (x; z)
(x; y)+ (y; z):
Example 81 The metric space (S; ); S = R2 ; the set of real numbers, and
(x; y) = [(x1 y1 )2 + (x2 y2 )2 ]1=2 ; for all x and y in S is a metric space.
Example 82 The metric space (S; ); where S = R; the set of real numbers,
and (x; y) = jx yj ; for all x and y in S is a metric space.
For vector spaces metrics are dened in a way that measures the distance
between two pints as the distance of their dierences from the zero pint
Denition 83 A normed vector space is a vector space S, together with a norm
k:k : S ! R; such that for all x; y 2 S and a 2 R;
(i) kxk 0; and kxk = 0
i x = ; (ii) kaxk = jaj : kxk ; (iii) kx + yk kxk + kyk :
91

Remark 84 Note that (x; y) = kx

yk :

What about the distance between two functions?

Exercise 85 Let S = C(a; b) be the set of continuous and bounded functions
from [a; b] to R, and for x; y 2 C(a; b) dene (x; y) by
(x; y) = max jx(t)
t2[a;b]

y(t)j :

Then (S; ) is a metric space. We need to show that,

(i) (x; y)

max jx(t)

y(t)j = jx(t )

a t b

y(t )j

0;

where t is the maximizer,

(x; y) = max jx(t)

y(t)j = 0 i x(t) = y(t) 8t 2 [a; b]; i.e. x = y:

a t b

(ii) (x; y) = max jx(t)

y(t)j = max jy(t)

a t b

a t b

x(t)j = (y; x):

and
(iii) (x; z)
(x; z)

max jx(t)

max jx(t)

z(t)j = jx(t )

a t b

jx(t )
a t b

(x; y) + (y; z);

y(t )j + jy(t )

y(t)j + max jy(t)

a t b

z(t )j

z(t )j

z(t)j = (x; y) + (y; z);

where t is the maximizer.

Example 86 Let S be the set of functions from some set A to R. For x; y 2 S;
dene another metric as
(x; y) = sup jx(t)

y(t)j

t2A

6.1.1

Supremum and Inmum

Denition 87 Let A 6= ;

R; set of upper bounds of A is dened as

U (A) = fu 2 R j u

a; for all a 2 Ag ;

and the set of lower bound of A is dened as

L(A) = fu 2 R j u

a; for all a 2 Ag :

Then if U (A) 6= ;; A is bounded above, and if L(A) 6= ;; A is bounded below.

Supremum of A; sup A; is dened as the least upper bound of A and inmum
of A is dened as the greatest lower bound of A: Note that if U (A) = ;; then
supA = 1; and if L(A) = ;; then inf A = 1: Also, max A = A \ U (A);
and min A = A \ L(A): Hence, while sup A and inf A are always dened for any
nonempty set (they could be innite), max A and min A might not always exists.
Theorem 88 Suppose sup A is nite. Then, for any " > 0; there is a(") 2 A
such that a(") > sup A ":
92

6.2

Sequences and Completeness

1

Denition 89 A sequence fxn gn=0 in a metric space S converges to a limit

x 2 S; if for each " > 0; there exists N" such that
(xn , x) < " for all n

N" .

Denition 90 A sequence fxn gn=0 in a metric space S converges to x 2 S; if

(xn ; x) ! 0 as n ! 1:
We will be particularly interested in this class with metric spaces that are
formed by sets of functions with appropriate metric dened on functions.
Example 91 Let C(1; 2) denote the set of continuos functions from [1; 2] to R
with
(x; y) = max jx(t) y(t)j :
t2[a;b]

Let
x1 (t) =

t
2t
nt
; x2 (t) =
; ::::; xn (t) =
; :::::
1+t
2+t
n+t

Figure (14) shows xn (t) =

nt
n+t :

Denition 92 If a sequence contains a convergent subsequence, the limit of the

convergent subsequence is called a limit point of the original sequence.
Theorem 93 A convergent sequence in Rn can have at most one limit point.
That is if fxk g is a sequence converging to a point x 2 Rn ; it cannot converge
to a point y 2 Rn for x 6= y:
Denition 94 A sequence in Rn is bounded if 9M such that kxk k
all k:

M for

Theorem 95 Every convergent sequence in Rn is bounded.

Denition 96 The lim sup of a real valued sequence is the supremum of the set
of limit points, and the lim inf of a real valued sequence is the inmum of the set
of limit points. Given a sequence fxk g ; which might not necessarily converge
and therefore might have many limit points, lim sup and lim inf gives the upper
and lower bounds of the set of limit points.
As an example consider the following sequence,
xk =

1 if k is odd
= f1; 1; 1; 2; 1; 3; 4; 1:::::g ;
if k is even

k
2

where lim sup xk = 1; and lim inf xk = 1: Note that lim sup and lim inf
are themselves limit points of a sequence. Therefore, in order to show that a
sequence converges we can use the following theorem.
93

1.8

1.6

1.4
n= 10
1.2

0.8

0.6
n= 1

0.4
1

1.1

1.2

1.3

1.4

1.5

Figure 14: xn (t) =

94

1.6

nt
n+t

1.7

1.8

1.9

Theorem 97 (Sundaram (1996), 1.18) A sequence in R converges to a limit

point x 2 R if and only if lim sup xk = lim inf xk = x:
Note that the denition of convergence requires a candidate limit point x;
when such a candidate is not available the following criteria is useful.
1

Denition 98 A sequence fxn gn=0 in S is a Cauchy sequence if for each " > 0;
there exists N" such that
(xn , xm ) < " for all n; m

N" :

Theorem 99 Let fxk g be a Cauchy sequence in Rn : Then, i) fxk g is bounded,

ii) fxk g has at most one limit point.
Of course for the Cauchy criteria to be useful we must work in spaces where
the Cauchy criteria implies a limit point within that space.
Denition 100 A metric space (S; ) is complete if every Cauchy sequence in
S converges to an element in S:
Note that in a complete metric space if you can show that a sequence is
Cauchy, you know for sure that the limit will be an element of that set.
Exercise 101 Let X = C(a; b) be the set of continuous functions from [a; b] to
R, and for x; y 2 C(a; b) dene (x; y) by
Z b
(x; y) =
j x(t) y(t) j dt:
a

Lets show that (X; ) is a metric space and is not complete. Since j x(t) y(t) j
0;
Z b
(i) (x; y) =
j x(t) y(t) j dt 0:
a

If x(t) = y(t); then j x(t)

y(t) j= 0; and
Z b
(ii) (x; y) =
j x(t)
a

y(t) j dt = 0:

On the other hand if (x; y) = 0; then x = y: In order to show this let (x; y) = 0;
but x 6= y:Then there must exist a t0 such that x(t0 ) y(t0 ) = " > 0: Since x is
a continuous function, 9 x such that j t0 t j < x implies j x(t0 ) x(t) j< 4" :
We can dene y similarly. Let = minf x ; y g; then for all t such thatj t0 t j
< ; x(t) y(t) > 2" : This follows from
" <j x(t0 )

y(t0 ) j j x(t0 )

Dene S = [a; b] \ [t0

x(t) j + j x(t)

y(t) j + j y(t)

; t0 + ]: Since t0 2 S;
Z
"
j x(t) y(t) j dt >
> 0:
2
S
95

y(t0 ) j :

Note that
(x; y) =

j x(t)

y(t) j dt

j x(t)

y(t) j dt > 0;

which is a contradiction. Finally,

(iii) (x; z)

(x; y) + (y; z);

follows from the triangular inequality for the absolute value. To show that C(a; b)
is not complete, consider fn (t) = tn for t 2 [0; 1]: This sequence converges to
0 if t 2 [0; 1)
1if t = 1;

fn ! f =

which is not an element of C(0; 1): It is easy to show however that this sequence
is Cauchy, since
(fn; fm ) =

(tn

tm+1
m+1

tn+1
n+1

tm )dt =

=
0

1
n+1

1
;
m+1

which can be made arbitrarily small.

Example 102 (Stokey and Lucas, 1989, Exercise 3.3c, page 45) The set
S of all continuous, strictly increasing functions on [a; b] with (x; y) = maxa t b jx(t)
y(t)j is a metric space. We need to show that,
(i) (x; y)

max jx(t)

y(t)j = jx(t )

a t b

y(t )j

0;

where t is the maximizer,

(x; y) = max jx(t)

y(t)j = 0 i x(t) = y(t) 8t 2 [a; b]; i.e. x = y:

a t b

(ii) (x; y) = max jx(t)

y(t)j = max jy(t)

a t b

a t b

x(t)j = (y; x):

and
(iii) (x; z)
(x; z)

max jx(t)

max jx(t)

z(t)j = jx(t )

a t b

jx(t )
a t b

(x; y) + (y; z);

y(t )j + jy(t )

y(t)j + max jy(t)

a t b

where t is the maximizer.

96

z(t )j

z(t )j

z(t)j = (x; y) + (y; z);

Example 103 (Stokey and Lucas, 1989, Exercise 3.6c, page 47) The metric space (S; ) in previous example is not complete. Consider the following
example of a Cauchy sequence in S that converges to a point that is not in S;
xn =

t
n

1
n=1

where t 2 [a; b]:

Each element of this sequence is a continuous and strictly increasing on [a; b]:
Hence, fxn g is contained in S: This is also a Cauchy sequence since,
(x; y) = max jxn (t)
a t b

xm (t)j = max j
a t b

t
n

t
1
j=j
m
n

1
j max jtj
m a t b

1
1
j j + j j max jtj;
n
m a t b
which can be made arbitrarily small by picking n and m large enough. Limit of
this sequence of functions, however, is not in S since,
xn =

t
n

! 0;

which is not strictly increasing. Hence, not all Cauchy sequences in S converges
to a limit in S; therefore the metric space (S; ) is not complete.
Theorem 104 R is complete.
Theorem 105 Rn is compete.

6.3

Three Cs: Closedness, Boundedness and Compactness

of Sets

Denition 106 In a metric space (S; ); the set A

S is closed if whenever
a1 ; a2 ; :::: 2 A and a1 ; a2 ; ::: ! a it follows that a 2 A:
Denition 107 A set A in a metric space (S; ) is bounded if there exists a
number D such that (a; a0 ) D for all a; a0 2 A:
Theorem 108 (Rudin (1976), 2.41) A set S
it is closed and bounded.

6.4

Rn is compact if and only if

Functions

Denition 109 Let S and T be metric spaces, E S, p 2 E; and f maps E

into T: Then f is said to be continuous at p if for every " > 0 there exists > 0
such that
T (f (x);

f (p)) < " for all points x 2 E for which

97

S (x; p)

< :

Denition 110 Let f : S ! T where S Rn and T

Rl : Then f is said to
be continuous at x 2 S if for all sequences fxk g such that xk 2 S for all k; and
xk ! x; it is the case that ff (xk )g ! f (x):
Theorem 111 (Rudin (1976), 4.14) Suppose f is a continuous mapping of
a compact metric space X into a compact metric space Y: Then f (X) is compact.

6.5

Sequences of Functions

6.5.1

Pointwise Convergence

Remember our denition of convergence in a metric space.

1

Denition 112 Let (S; ) be a metric space. A sequence fxn gn=0 in S converges to x 2 S; if for each " > 0; there exists N" such that
(xn , x) < " for all n

N" .

One problem with this denition was that we need a candidate limit point x
to check if the sequence converges. When we have a sequence of functions, such
a candidate naturally arises.
Denition 113 Let ffn g be a sequence of functions dened on a set E
S;
and suppose that the sequence of numbers ffn (x)g converges for every x 2 E:
We can then dene a function by,
f (x) = lim fn (x);
n!1

x 2 E:

(51)

We say that ffn g converges to f pointwise on E; if (51) holds.

The main question that we are interested is whether the important properties
of a sequence of functions are preserved under pointwise convergence. We would
like to know, for example, whether the limiting function will be continuous if
every function in the sequence is continuous.
Example 114 Let for each n; fn : [1; 2] ! R be given by,
fn (x) =

nx
;
n+x

x 2 [1; 2]:

Let x = 32 ; then, see Figure (14),

3
3
3
6
3
f1 ( ) = ; f2 ( ) = ; f3 ( ) = 1;
2
5
2
7
2

3
3n
3
; f2 ( ) =
! :
2
2n + 3
2

Indeed,
f (x) = lim fn (x) = x;
n!1

98

x 2 [1; 2]:

Given that ffn g convergence pointwise to f (x) = x; we can use this limiting
function as our candidate and check if the sequence converges to this function
given a particular metric. Let,
(f; g) = max jf (x)

g(x)j :

x2[1;2]

Then,
(fn ; f ) = max

x2[1;2]

nx
n+x

x = max

x2[1;2]

x2
x2
4
< max
= :
n+x
n
x2[1;2] n

Hence, (fn ; f ) ! 0; as n ! 1; and ffn g ! f .

In this example, the sequence of functions converges pointwise to a nice
function. Moreover, sequence also converges to the limiting function given our
metric. This will not be the case in general. Before going into examples, however, note that convergence implies pointwise convergence.
Theorem 115 Let X = C(a; b) be the set of continuous functions [a; b] ! R,
and for f; g 2 C(a; b) dene (f; g) by,
(f; g) = max j f (x)
a x b

g(x) j :

If f1 ; f2 ; f3 ; : : : be functions in X such that f1 ; f2 ; f3 ; : : : ! f 2 C(a; b); then for

any x 2 [a; b], f1 (x); f2 (x); f3 (x); : : : ! f (x) in R:
The converse of this theorem, as we already pointed out, does not hold.
Example 116 Let for each n; fn : [0; 1] ! R in C(0; 1) be given by,
fn (x) = xn ;

x 2 [0; 1]:

Then,
f1 (x); f2 (x); f3 (x);

! 0; for x 2 [0; 1);

but,
f1 (x); f2 (x); f3 (x);

! 1; for x = 1:

Hence,
f (x) = lim fn (x) =
n!1

0 if x < 1
;
1 if x = 1

which is not a continuous function.

Example 117 Let for each n; fn : [0; 1] ! R in C(0; 1) be given by,
fn (x) =

nx
;
1 + n2 x2

99

x 2 [0; 1]:

0.9

0.8

0.7

0.6

n= 1

0.5

0.4

0.3

0.2

n= 10

0.1

0
0

0.1

0.2

0.3

0.4

0.5

Figure 15: fn (x) = xn

100

0.6

0.7

0.8

0.9

In this case,
f1 (x); f2 (x); f3 (x);
Yet,
(fn ; f ) = max

x2[0;1]

! 0; for all:

nx
1 + n2 x2

1
:
2

0 =

Hence, (fn ; f ) 9 0; as n ! 1; and hence ffn g 9 f with (f; g) = maxa

f (x) g(x) j :

x b

These example suggest that we need a stronger concept of convergence than

the pointwise convergence.
6.5.2

Uniform Convergence

Denition 118 We say that a sequence of functions ffn g, n = 1; 2; 3; : : : converges uniformly on E to a function f , if for every " > 0; there exists an integer
N such that n N implies
jfn (x)

f (x)j

" for all x 2 E:

If ffn g converges uniformly on E, then it is possible, for each "; to nd one

integer N which will work for all x 2 E: Following theorem gives a very useful
characterization for uniform convergence.
Theorem 119 The sequence of functions ffn g, n = 1; 2; 3; : : : converges uniformly on E if and only if for every " > 0; there exists an integer N such that
m N; n N; and x 2 E implies
jfn (x)

fm (x)j

":

Proof. Suppose ffn g converges uniformly on E; and let f be the limit function.
Then there must exist an integer N such that n N; and x 2 E implies
jfn (x)
so that if m

N; n
jfn (x)

f (x)j

"
;
2

N; and x 2 E; then
fm (x)j

jfn (x)

f (x)j + jf (x)

fm (x)j

":

Conversely, suppose the Cauchy condition holds. Then for every x; ffn (x)g
converges to a limit point that we may call f (x): Thus, the sequence ffn g converges pointwise to f on E: We need to show that convergence is also uniform.
Let " > 0 be given, and choose N such that the Cauchy condition holds. Fix n;
and let m ! 1 in
jfn (x) fm (x)j ":
Since fm (x) ! f (x) as m ! 1; this gives
jfn (x)
for every n

f (x)j

";

N and every x 2 E; which completes the proof.

101

Hence if a sequence is Cauchy then it converges pointwise to a limiting

function, and moreover convergence is uniformly.
The following theorem shows that uniform convergence preserves the continuity and boundedness of the elements of a sequence of functions.
Theorem 120 (Uniform Convergence Theorem) Let S
Rn ; let ffn g be a
sequence of functions from S to R such that fn ! f uniformly. If functions fn
are all bounded and continuous, then f is bounded and continuous.
Proof. Boundedness is obvious, since fn (x) ! f (x) for each x; if f is unbounded then functions fn must be unbounded.
To show continuity, x x 2 S; we need to show that for all " > 0; there exists
a > 0 such that y 2 S and kx yk < implies jf (x) f (y)j < ": Let " > 0
be given. For any k;
jf (x)

f (y)j

jf (x)

fk (x)j + jfk (x)

Fix N su ciently large so that for all n

jf (z)
Since fN is continuous, there exists a
kx
Therefore whenever kx
jf (x)

f (y)j

yk <

fk (y)j + jfk (y)

f (y)j :

N;and for all z 2 S;

fn (z)j <

"
;
3

> 0 such that,

=) jfN (x)

fN (y)j <

"
:
3

yk < ; we have,

jf (x) fN (x)j + jfN (x)

" " "
<
+ +
3 3 3
= ":

fN (y)j + jfN (y)

f (y)j

Note that previous two Theorems are building blocks in showing that C(X);
the set of bounded and continuous functions on set X
Rl ; with (f; g) =
supx2X j f (x) g(x) j, is complete [Theorem 3.1 in page 47 in Stokey and
Lucas with Prescott (1989)].
The following theorem provides another characterization of uniform convergence.
Theorem 121 Suppose,
lim fn (x) = f (x) for x 2 E;

n!1

Let
Mn = sup jfn (x)

f (x)j :

x2E

Then fn ! f uniformly on E if and only if Mn ! 0 as n ! 1:

102

Example 122 Let for each n; fn : R+ ! R+ be given by,

fn (x) =

nx; for x
1; for x >

1
n
1
n

Then fn converges pointwise to,

f (x) =

0; for x = 0
:
1; for x > 0

However, fn 9 f uniformly, otherwise f would be continuous.

Next theorem provides another useful way to check uniform convergence.
Theorem 123 Suppose E is compact, and
(i) ffn g is a sequence of continuous functions on E;
(ii) ffn g converges pointwise to a continuous function f on E,
(iii) fn (x) fn+1 (x) for all x 2 E; n = 1; 2; 3; : : :
Then fn ! f uniformly on E:
Following example shows that compactness of the set on which the sequence
is dened crucial in the last theorem.
Example 124 Let for each n; fn : (0; 1) ! R in C(0; 1) be given by,
fn (x) =

1
;
1 + nx

x 2 (0; 1):

Note that fn (x) ! 0 monotonically in (0; 1); yet convergence is not uniform
since,
1
0 = 1:
(fn ; f ) = sup
x2(0;1) 1 + nx
Theorem 125 Let X
Rn ; and let C(X) be the set of bounded continuous
functions f : X ! R with sup metric. Then C(X) is a complete metric space.
Proof. See Stokey and Lucas with Prescott (1989) Theorem 3.1.
Hence, if we have a sequence of functions in C(X) and the sequence satises
the Cauchy criteria, then the limiting function is also C(X):

6.6

Contraction Mappings

We will now analyze operators, such as the operator T we dened above:

(T V )(k) =

max [u(f (k) k 0 ) + V (k 0 )]:

|
{z
}
0 k0 k

103

Denition 126 Let (S; ) be a metric space and T : S ! S be a function

mapping S into itself. T is a contraction mapping (with modulus ) if for some
2 (0; 1);
(T x; T y)
(x; y); for all x; y 2 S:
Theorem 127 (Contraction Mapping or Banach Fixed Point Theorem) If (S; )
is a compete metric space and T : S ! S is a contraction mapping with modulus
; then
(i) T has exactly one xed point V in S:
n
(ii) for any V0 2 S; (T n V0 ; V )
(V0 ; V ), n = 0; 1; 2; 3::::
1

Proof. Choose V0 ; and dene fVn gn=0 ; by Vn+1 = T Vn ; so that V1 = T V0 ;etc.

Then,
(V2 ; V1 ) = (T V1 ; V1 )
(V1 ; Vo );
and operation in the same way will lead to
n

(T Vn+1 ; Vn )

(V1 ; Vo ); n = 0; 1; 2; :::

For any m > n;

(Vm ; Vn )
[

(Vm ; Vm 1 ) + :::: + (Vn+1 ; Vn )

m 1
+ :::: + n+1 + n ] (V1 ; Vo )
n m n
[
+ :::: + + 1] (V1 ; Vo )
n

(V1 ; Vo )

Hence, fVn g is a Cauchy sequence. Since S is complete,

Vn ! V 2 S:
To show that T V = V; for all n; and for all V0 2 S; note that
(T V; V )

(T V; T n V0 ) + (T n V0 ; V )
(V; T n 1 V0 ) + (T n V0 ; V ):

As n ! 1; (T V; V ) ! 0; hence T V = V:
To show that V is unique. Suppose,9Vb 2 S; such that T Vb = Vb and V 6= Vb ;
then
0 < (Vb ; V ) = (T Vb ; T V )
(Vb ; V );
a contradiction. Finally, for any n

(T n V0 ; V ) = (T (T n

1:

V0 ); T V )

(T n

V0 ; V ):

Corollary 128 Let (S; ) be a complete metric space and let T : S ! S be a

contraction mapping with xed point of V 2 S: If S 0 is a closed subset of S and
T (S 0 ) S 0 then V 2 S 0 : If in addition, T (S 0 ) S 00 S 0 ; then V 2 S 00 :
104

The Contraction Mapping Theorem states that if you have a contraction

mapping T on a complete metric space, then you always have a unique xed
point. Furthermore you will get to this xed point starting from any initial
guess in this metric space. The corollary states that if the contraction mapping
T maps closed subset S 0 of S into itself, then the xed point has to be in this
subset. If T maps S 0 into a closed subset S 00 of S 0 ; then the xed point has to
be in S 00 :
The question is how we can check if T is a contraction mapping. The following Theorem provides a nice characterization.
Theorem 129 (Blackwells Su ciency Conditions) Let X Rl and B(X) be
a space of bounded functions f : X ! X with sup norm. Let T : B(X) ! B(X)
satisfy,
i) for all f; g 2 B(X) such that f (x) g(x) for all x 2 X implies
(T f )(x)
a

(T g)(x); for all x 2 X;

ii) 9 2 (0; 1) such that [T (f + a)] (x)

0; and x 2 X:

(T f )(x) + a; for all f 2 B(X),

Proof. See Stokey and Lucas with Prescott (1989) Theorem 3.3.
Consider the operator for the one sector growth model
(T V )(k) =
Let W (k)

max [u(f (k)

0 k0 k

k 0 ) + V (k 0 )] :

V (k); then
(T W )(k)

max [u(f (k)

k 0 ) + W (k 0 )]

max [u(f (k)

k 0 ) + V (k 0 )] = (T V )(k);

0 k0 k

0 k0 k

and
T (V + a)(x)

=
=
=

max [u(f (k)

k 0 ) + V (k 0 ) + a]

max [u(f (k)

k 0 ) + V (k 0 )] + a:

0 k0 k
0 k0 k

(T V )(x) + a:

Exercise 130 Consider the mapping A of n-dimensional space into itself given
by the system of linear equations
y = A(x);
where
yi =

n
X

aij xj + bi ; i = 1; : : : n:

j=1

105

If A is a contraction mapping we can use the method of successive approximations to solve the equation Ax = x: Given
(x; y) = max jxi

yi j ;

1 i n

nd the conditions on a such that A is contraction mapping. First note that

jAx

Ayj = jA(x

y)j = j

max jxj

n
X

j=1
n
X

yj j

1 j n

aij (xj

j=1

yj )j

n
X
j=1

jaij jjxj

yj j

jaij j:

Then,
(Ax; Ay)

max jAxi

Ayi j

1 i n

yi j @ max

max jxi

1 i n

= @ max

1 i n

If

max

1 i n

n
X
j=1

n
X
j=1

1 i n

n
X
j=1

jaij jA

jaij jA (x; y):

jaij j

k < 1;

then, A is contraction mapping with modulus k.

Exercise 131 Let X = (1; 1): Let f : X ! R be given by
f (x) =

a
1
x+
:
2
x

Show that if a 2 (1; 3); then f is a contraction. Find the xed point of f as a
function of a:One needs to check two things: a) f maps X onto X and b) for all
x and x0 from X (f (x); f (x0 )) k (x; x0 ) for some k 2 (0; 1). To check that
f maps X onto X, observe that f 00 (x) = 14 xa3 > 0 for all x 2 X and a 2 (1; 3).
Thus, f is strictly convex which means that it has a unique minimum.pTo nd
that minimum,psolve f 0 (x) = 12 1 xa2 = 0. The solution is x = a > 1.
Then f (x ) = a > 1, which implies that for all a 2 (1; 3), f : X ! X. To
verify that f (x) is actually a contraction, consider
(f (x); f (x0 )) =

1
a
x+
2
x

1 0
a
x + 0
2
x

1
1
2

a
xx0

jx

x0 j :

a
It su ces to show that a 2 (1; 3) implies that 12 1 xx
< 1 for all x; x0 2 X.
0
1
a
For any xed a, the function ga (z)
2 1
z , z 2 X, is decreasing on (1; a),

106

is equal to zero at z = a, and is increasing on (a; +1) with

lim ga (z) =

z!+1

1
2.

Therefore, it is su cient to consider x and x0 such that xx0 2 (1; a). Then,
1
a
< 12 (1 a) and 12 (1 a) < 1 if a 2 (1; 3).
0
2 1
xx0

6.7

Correspondences

We will now have a closer look at the particular operator T in

V (x) = max [F (x; y) + V (y)] :

(FE)

y2 (x)

Obviously the properties of T will depend on the maximization problem

maxy2 (x) [F (x; y) + V (y)] :
Denition 132 A correspondence from X Rl into Y
Rm is a map which
associates with each element x 2 X, a (non-empty) subset (x) Y:
Consider the following dynamic programming problem,
V (x) = sup [F (x; y) + V (y)]
y

s:t: y is feasible given x;

l

where x 2 X R is the beginning of period state variable, y 2 X is the endof-period state variable (or control variable) to be chosen, and F (x; y) is the
current period return function.
Correspondences are used to denote the relationship between the current
state variable, x, and the choice variable, y: A feasibility correspondence, :
X ! X; is used to dene which values of y are feasible given x: We would like to
know how (x) behaves as x changes over X in order to be able to characterize
how the maximizing values of y and value function V (x) behaves over X. Hence,
we need to introduce a notion of continuity for correspondences.
Denition 133
: X ! Y is a compact-valued correspondence if
compact set of Y for each x 2 X:

(x) is a

Denition 134
: X ! Y is a closed-valued correspondence if
closed set of Y for each x 2 X:

(x) is a

Denition 135
: X ! Y is a convex-valued correspondence if
convex set of Y for each x 2 X:

(x) is a

To give an example of a correspondence with these properties consider,

(x) = fy j 0

xg ; where X

Denition 136 Graph of a correspondence

A = f(x; y) j y 2
107

R+ ; and Y

R+ :

(x) is the set A dened as,

(x)g :

Denition 137
set.

: X ! Y is a closed-graph correspondence if A is a closed

Denition 138
set.

: X ! Y is a convex-graph correspondence if A is a convex

Note that a closed-graph correspondence is also closed-valued, and a convexgraph correspondence is also convex-valued. The converses, however, do not
hold.
6.7.1

Lower Hemi-Continuity:

Denition 139 A correspondence : X ! Y is lower hemi-continuous (l.h.c.)

at x, if (x) is non-empty and if, for every y 2 (x) and every sequence xn ! x;
1
there exists N 1 and a sequence fyn gn=N such that yn ! y and yn 2 (xn );
for all n N:
Note that in order to check l.h.c. of a correspondence at x; we rst pick
any point y 2 (x) and a sequence xn ! x: Then, we look for a sequence yn
which is contained in (xn ); and converges to the point y: Since we rst pick
any point y in the image of x; l.h.c. fails if there exists a sudden blow-up in
the correspondence. To give an example, consider the correspondence in Figure
16. It fails to be l.h.c. at x, because there is no sequence fyn g which converges
to y; and contained in (xn ); for all n:

(x)

{xn}

Figure 16: Not l.h.c. at x

108

6.7.2

Upper Hemi-Continuity:

Denition 140 A compact valued correspondence : X ! Y is upper hemicontinuous (u.h.c) at x if (x) is non-empty and if, for every sequence xn ! x
and every sequence fyn g such that yn 2 (xn ) for all n; there exists a convergent
subsequence of fyn g whose limit point y is in (x):
Note that in order to check u.h.c. of a correspondence at x; we rst pick
xn ! x and a sequence yn contained in the images of xn : Then, we look for a
convergent subsequence of yn which converges to a point y in the image of x:
Upper hemi-continuity will fail if there is a sudden collapsein the correspondence. Then, we could pick xn and yn , but fail to nd a point y in the image
of x such that a subsequence of y converges to that point. The correspondence
in Figure 17 is not u.h.c. at x:

y
{yn}

(x)

{xn}

Figure 17: Not u.h.c. at x

Denition 141 A correspondence

both u.h.c. and l.h.c.

6.8

: X ! Y is continuous at x 2 X if it is

The Theorem of Maximum:

Consider the following optimization problem,

sup f (x; y);
y2 (x)

109

where f : X Y ! R is a single valued function, and : X ! Y is a non-empty

correspondence. Lets dene the maximized value function h(x) as,
h(x) = max f (x; y);
y2 (x)

and the set of maximizers G(x) as,

G(x) = fy 2

(x) j f (x; y) = h(x)g :

Theorem 142 Let X Rl and Y

Rm ; let f : X Y ! R be a continuous
function, and let : X ! Y be a compact-valued and continuous correspondence. Then:
(i) the function h(x) is continuous, and
(ii) the correspondence G(x) is nonempty, compact-valued, and u.h.c.
Proof.
Step1: G(x) 6= ;:
Fix x: Since (x) is compact-valued and f (x; ) is continuous, a maximum
exists by the Weierstrass Theorem. Hence, G(x) 6= ;:
Step2: G(x) is compact-valued.
Step2a: G(x) is bounded.
Fix x. Since (x) is a compact-valued, it is also bounded. G(x) is the set of
maximizers and (x) is the feasible set. Therefore, G(x)
(x): Hence, G(x)
is bounded.
Step2b: G(x) is closed.
Fix x. Suppose yn ! y and yn 2 G(x) for all n (hence, every element of yn
is a maximizer given x): We want to show that y 2 G(x).
Since G(x)
(x); yn 2 (x) for all n as well: Since (x) is compactvalued, it is also closed. Therefore, y 2 (x) (hence, y is feasible given x):
Since, h(x) = f (x; yn ) for all n and f is continuous, h(x) = f (x; y): Hence,
y 2 G(x):
Step 3: G(x) is u.h.c.
Fix x: Let xn ! x and choose yn 2 G(xn ) for all n: We want to show that
there exists a subsequence fynk g of fyn g such that fynk g ! y 2 G(x):
Since G(x)
(x); yn 2 (xn ) for all n as well: Since (x) is u.h.c., there
must exist a subsequence fynk g of fyk g such that fynk g ! y 2 (x):
Let z 2 (x) (hence, z is any feasible point given x). Since (x) is also l.h.c.,
there must exists a sequence znk ! z such that znk 2 (xnk ) for all k:
110

Since f (xnk ; ynk )

f (xnk ; znk ) [note that ynk are the elements of G(x);
and znk are the elements of (x)], and f is continuous, we have f (x; y)
f (x; z): Since this holds for any z 2 (x); y 2 G(x): Hence, G(x) is u.h.c.
Step 4: h(x) is continuous.
Fix x: Let fxn g be a sequence converging to x: We want to show that
fh(xn )g ! h(x):
Choose yn 2 G(xn ) for all n (hence, yn is a sequence of maximizers corresponding to each element of xn ). Lets dene h = lim sup h(xn ) and h= lim inf
h(xn ): Note that we need to show that h = h = h(x):
By denition of lim sup; there exists a subsequence fxnk g of fxk g such that
h = lim f (xnk ; ynk ): Note that since yn 2 G(xn ) for all n; ff (xnk ; ynk )g is a
subsequence of fh(xn )g.
Since G(x) is u.h.c., there must exist a convergent subsequence of fynk g ;
call it yj0 ; converging to a limit point y 2 G(x): Hence, h = lim f (xj ; yj0 ) =
f (x; y) = h(x): A similar argument also shows that h = h(x):
Corollary 143 If is compact-valued, continuous, and convex-valued; and f
is continuous and strictly concave in y; then G is single-valued, therefore it is a
continuous function, called g:
Note that in the following problem
(T V )(x) = max [F (x; y) + V (y)] ;
y2 (x)

T V is the maximized value function, and therefore if F (x; y)+ V (y) is continuos
and (x) is compact-value and continuos the T V is also a continuos function.
Furthermore, if F (x; y) + V (y) is continuos and strictly concave and (x) is
compact valued, continuos, and convex valued, then there is a unique maximizer
y and hence G(x) = g(x) is a function.

References
[1] Rudin, W. Principles of Mathematical Analysis, McGraw-Hill, 1974.
[2] Sundaram, R. K. A First Course in Optimization Theory, Cambridge University Press, 1996.
[3] Stokey, N. L. and Lucas, R. E. with Prescott, E. Recursive Methods in
Economic Dynamics, Harvard University Press, 1989.

111

Principal of Optimality

We have started our analysis of innite-horizon problems with the following

sequential problem (SP)
sup

1
X

fxt+1 g1
t=0 t=0

F (xt ; xt+1 );

(SP)

subject to
xt+1 2 (xt ); t = 0; 1; 2; :::;
and
x0 given.
We then argued that the supremum function dened by
V (x0 ) =

1
X

sup

fxt+1 g1
t=0 t=0

F (xt ; xt+1 );

should satisfy the following functional equation (FE);

V (x) = sup [F (x; y) + V (y)] ;

(FE)

y2 (x)

The supremum function V (x0 ) tells us the innite discounted value of fol1
lowing the best sequence fxt+1 gt=0 : Our hope was that rather than nding the
1
best sequence xt+1 t=0 ; we can try to nd the function V (x0 ) as a solution
to the FE. If our conjecture is correct, then the function V that solves FE would
give us the supremum function, i.e. V (x0 ) = V (x0 ):
We will now try to show that:
The supremum function V

has to satisfy FE.

If there is a solution to FE, then it is the supremum function, i.e. solution

V to the FE evaluated at x0 gives the value of the supremum in SP when
the initial value is x0 :
1

A sequence fxt+1 gt=0 attains the supremum in (SP) if and only if it satises
V (xt ) = F (xt ; xt+1 ) + V (xt+1 ); t = 0; 1; 2; 3::::
There is indeed a solution to FE.
In order to prove these claims, lets rst introduce some notation. Let X; be
the possible values of the state. We will let : X ! X denote the feasibility
correspondence. The graph of the feasibility correspondence is then given by
A = f(x; y) 2 X X : y 2 (x)g : We will let F : A ! R be the return function.
Note that F indicates the momentary return of being in a particular state x 2 X;
112

and making a particular choice y out of feasible values (x): Finally we will let
0 be the discount factor.
These are the givens in this problem. In order to gain some better insight,
we will use the one sector growth model as an example.
Example 144 (Stokey, Lucas, and Prescott (1989), Exercise 5.1, p. 103) Consider the dynamic programming problem for the one sector growth model,
V (x) =

max

0 y f (x)

fU [f (x)

y] + V (y)g ;

under the following standard assumption on preferences and technology (where

U : R+ ! R; and f : R+ ! R):
U1 : 0 <

< 1;

U 2 : U is continuous,
U 3 : U is strictly increasing,
U 4 : U is strictly concave,
U 5 : U is continuously di erentiable.
T 1 : f is continuous,
T 2 : f (0) = 0; for some x > 0; x
f (x) < x for all x > x;

f (x)

x for all 0

x; and

T 3 : f is strictly increasing,
T 4 : f is weakly concave,
T 5 : f is continuously di erentiable.
Note that in this example, we can pick X = [0; x] where x is the highest
maintainable capital stock (see Figure 18). F (x; y) = U [f (x) y]; and (x) =
[0; f (x)]:
We will sometimes use the following notation to analyze the one-sector
growth model
V (k) = max
fU [f (k) k 0 ] + V (k 0 )g ;
0
0 k

f (k)

or
V (kt ) =

max

0 kt+1 f (kt )

fU [f (kt )

113

kt+1 ] + V (kt+1 )g :

y=x
y=F(x,1)

xupper

Figure 18: X = [0; x]

In order to dene the supremum function V (x0 ); we will introduce some

1
further notation. We will call any sequence of actions a plan, fxt+1 gt=0 : Obviously we are only interested in the set of feasible plans starting from x0; dened
as
1
(x0 ) = ffxt+1 gt=0 : xt+1 2 (xt ); t = 0; 1; 2; :::g :
Let x = (x0 ; x1 ; :::::::::) 2 (x0 ); be a typical feasible plan.
~

Note that nding a solution to the SP is simply nding the best feasible plan
(or plans if there is more than one). In order to be able to say anything about
the solutions to the SP, we will need two assumptions:
A1:

(x) 6= ; for all x 2 X:

A2: 8x0 2 X and x 2 (x0 )

~

lim

n!1

n
X
t=0

F (xt ; xt+1 ) exists (may be + 1 or

1):

Hence, we want to make sure that i) (x0 ) is not empty, i.e. there is some
feasible plan, ii) we can evaluate (say how good or bad it is) any feasible plan.
For each n = 0; 1; 2; :::: we will dene un : (x0 ) ! R by
un (x) =
~

n
X

F (xt ; xt+1 );

t=0

114

which simply gives us the discounted return from following a feasible plan x
~

from data 0 to date n:

Also let u : (x0 ) ! R

R [ f 1g,
u(x) = lim un (x);
n!1

to be the innite discounted sum of returns from following feasible plan x: Then,
~

the supremum function is simply dened as

V (x0 ) =

sup u(x):
~

x2 (x0 )
~

Remark 145 Given A1 and A2, V (x0 ) is uniquely dened.

For some results below, we will make a more restrictive assumption than A2
and assume that F bounded.
A2x: F : A ! R is bounded.
Then, both V and V will be bounded functions. In particular, if B is a
bound on F; Then,
jV (x0 )j

sup

1
X

x2 (x0 ) t=0

1
X

jF (xt ; xt+1 )j

B=

t=0

B
1

Hence, we can replace supremum with a maximum in SP. Then, by denition

V (x0 ) is the unique function satisfying the following conditions:
V (x0 )

u(x) for all x 2 (x0 );

(SP1)

u(x) + "; for some x 2 (x0 ):

(SP2)

and for any " > 0

V (x0 )

Hence, if jV (x0 )j < 1, then it must give more utility than any other feasible
plan.
We want to show that V (x0 ) satises the FE. We have to make more clear
what we mean by that. We will say that V satises the FE, if
V (x0 )

F (x0 ; y) + V (y) for all y 2 (x0 );

(FE1)

F (x0 ; y) + V (y) + "; for some y 2 (x0 ):

(FE2)

and for any " > 0

V (x0 )

We will rst prove that under A1 and A2x the supremum function V satises FE. We will rst prove, however, a Lemma that is key. This Lemma simply
tells us that you can separate discounted innite sum of returns from any feasible plan into current and future returns. This separation is key in dynamic
programming.
115

Lemma 146 Let X; ; F; and

(x0 ; x1 ; ::::::) = x 2 (x0 )

satisfy A2: Then, for any x0 2 X; and any

u(x) = F (x0 ; x1 ) + u(x0 );

~

where x0 = (x1 ; x2 ; ::::):

~

Proof: Remember
u(x) = lim
~

n!1

n
X

F (xt ; xt+1 ) .

t=0

Then,
u(x)

= F (x0 ; x1 ) +

lim

n!1

n
X

F (xt+1 ; xt+2 )

t=0

= F (x0 ; x1 ) + u(x0 ):
~

Note that for our one sector growth model example, this lemma simply states
that
1
X

U (f (xt ) xt+1 ) = U (f (x0 ) x1 )+

t=0

1
X

U (f (xt+1 )

t=0

{z

xt+2 )
}

the present value of following a feasible plan starting from x1

Theorem 147 Let X; ; F; and satisfy A1 A2x: Then, V satises FE.

Proof : Suppose > 0 (otherwise the result is trivial) and choose x0 : We
know that SP1 and SP2 hold. To establish FE1, let x1 2 (x0 ); and let " > 0 be
given. We know that there exists a feasible plan x0 = (x1 ; x2 ; ::::) 2 (x1 ) such
~

that

u(x0 )

V (x1 )

":

This is true since V (x1 ) is the supremum function. Since (x0 ; x0 ) 2 (x0 ) by
~

the previous Lemma

V (x0 )

u(x) = F (x0 ; x1 ) + u(x0 )

~

F (x0 ; x1 ) + V (x1 )

";

Since x1 is arbitrary, for all y 2 (x0 )

V (x0 )

F (x0 ; y) + V (y)

since " is also arbitrary

V (x0 )

F (x0 ; y) + V (y);

116

";

which establishes F E1:To establish FE2, choose x0 2 X; and " > 0: Then, by
SP2 and the previous Lemma
V (x0 )

u(x) + " = F (x0 ; x1 ) + u(x0 ) + ";

~

or
F (x0 ; x1 ) + u(x0 ) + ";

V (x0 )

since x1 2 (x0 ); FE2 follows.

This theorem showed that the supremum function V satises the FE. The
following theorem provides a converse: If V satises the FE (i.e. if V is a
solution to the FE) and if its bounded, then V is the supremum function (i.e.
V = V ):
Theorem 148 Let X; ; F; and
and satises
lim

n!1

satisfy A1

A2: If V is a solution to the FE

V (xn ) = 0; for all (x0 ; x1 ; :::) 2 (x0 ); and for all x0 2 X;

then V = V :
Proof: Note that if we assumed A2x; boundedness condition is satises trivially. Hence, we could state this theorem as: "Let X; ; F; and satisfy A1 A2:
If V is a solution to the FE, then V = V ": Stating bounded condition explicitly
makes it role more transparent. We need to show that FE1 and FE2 imply SP 1
and SP 2: Note that F E1 implies that for all x 2 (x0 )
~

V (x0 )

F (x0 ; x1 ) + V (x1 )
F (x0 ; x1 ) + F (x1 ; x2 ) + 2 V (x2 )
:
:
un (x) + n+1 V (xn+1 ); n = 1; 2; :::
~

As n ! 1; limn!1

V (xn ) = 0; and hence

V (x0 )

un (x);
~

and SP1 holds, i.e.,

V (x0 )

u(x) for all x 2 (x0 );

~

We now need to show that for any ";

V (x0 )

u(x) + " for some x 2 (x0 ):

~

P1
1
To this end x " > 0; and choose f t gt=1 in R+ such that t=1
Since FE2 holds, we can choose x1 2 (x0 ); x2 2 (x1 ); :::: so that
V (xt )

F (xt ; xt+1 ) + V (xt+1 ) +

117

t+1 ; t

= 0; 1; :::::;

t 1

"
2.

i.e.
V (x0 )

F (x0 ; x1 ) + V (x1 ) +

1;

V (x1 )

F (x1 ; x2 ) + V (x2 ) +

2;

etc. Then, obviously x = (x0 ; x1 ; ::::) 2 (x0 ); and

~

V (x0 )

n
X

F (xt ; xt+1 ) +

n+1

V (xn+1 ) + (

+ ::: +

n+1 )

t=0

un (x) +

n+1

V (xn+1 ) + "=2; n = 1; 2; ::::;

which implies that for all n su ciently large, V (x0 )

un (x) + ": Since " > 0

~

was arbitrary, it follows that SP2 holds.

It is important to note that the proof requires limn!1 n V (xn ) = 0 is
true for all feasible plans. Obvioulsy this is satised if F and as a result V is
bounded. If boundedness s not satised, and if there is a feasible plan that does
not satisfy limn!1 n V (xn ) = 0 for a V; then we can not conclude that V
is the supremum function (even if it satises the FE). It is also important to
understand what this theorem states. It states that if V satises the FE and
it is bounded then it is the supremum function. If the boundedness fails, then
there might be solutions to the FE that are not supremum function.
Example 149 Consider the following consumption-saving decision of an innitely lived agent with initial assets x0 2 X = R: The agent can borrow or lend
at rate 1 + r = R = 1 > 1: Hence, price of borrowing one unit for tomorrow is
: There are no borrowing constraints, i.e.
ct + xt+1

xt ;

Hence, the agents problem is

V (x0 ) =

sup

1
X

fct ;xt+1 g1
t=0 t=0

ct :

s.t.
0

ct xt
xt+1 :
x0 given.

What is the value of V (x0 )? Since agent can borrow as much as he wants it is
obvious that V (x0 ) = 1: Consider now the following FE that corresponds to
this problem
V (x) = sup [x
y + V (y)] :
y

Both V (x) = 1 and Ve (x) = x are solutions to the FE, but Ve does not satisfy
boundedness condition in the previous theorem. Remember that when we solved a
118

SP using a Lagrangian approach, we needed the Transversality Condition (TC).

Hence, if a sequence of actions satisfy the Euler equation, it was the optimal
solution if it also satised the TC. Argue (with some carefully selected sentences)
that the TC and the bounded condition in Theorem 4.3 (page 72 in SLP) serve
the same purpose.
The following two theorems show that a sequence that attains the supremum
in SP must satisfy FE and with bounded returns, any sequence that satises
FE is a solution to SP.
Theorem 150 Let X; ; F; and

satisfy A1

A2: Let x 2 (x0 ) be a feasible

~

plan that attains supremum in (SP) for x0 ; then

V (xt ) = F (xt ; xt+1 ) + V (xt+1 ):

Proof: See Stokey, Lucas and Prescott (1989), Theorem 4.3, page 75).
Theorem 151 Let X; ; F; and

satisfy A1

plan from x0 satisfying

A2: Let x 2 (x0 ) be a feasible

~

V (xt ) = F (xt ; xt+1 ) + V (xt+1 );

with
lim sup

t!1

V (xt )

0;

then x attains supremum in SP for x0 :

~

Proof: See Stokey, Lucas and Prescott (1989), Theorem 4.3, page 76).

7.1

FE with Bounded Returns

Consider the following functional equation for a dynamic programming problem,

V (x) = max [F (x; y) + V (y)] ;
y2 (x)

(1)

under the assumption that F is bounded and < 1:

As before X is the set of all possible values for state variable x; (x) is
the feasibility correspondence, A = f(x; y) 2 X X : y 2 (x)g is the graph of
(x); y is the next period state variable to be chosen from (x); F : A ! R is the
return function, and
0 is the discount factor. We are interested establishing
assumptions on X; (x); F; and which will guarantee that the solution to the
functional equation (1) exists, it is unique, and the solution has some desirable
properties.
Assumption 1: X is a convex subset of Rl : : X ! X is a non-empty,
compact-valued, and continuous correspondence.

119

Assumption 2: F : A ! R is bounded and continuous, and 0 <

< 1:

Note that if F is bounded, the solution the supremum function V will be

also bounded. Therefore, we can try to nd a solution to (1) in the space of
bounded and continuous functions C(X); with metric
(f; g) = sup j f (x)
x2X

g(x) j :

The functional equation in (1) denes an operator on the elements of C(X)

given by,
(T f )(x) = max [F (x; y) + f (y)] :
(2)
y2 (x)

Given any particular f ( ) 2 C(X); we can evaluate [F (x; y) + f (y)] for every
possible value of y and solve the maximization problem for any x: This operation
will give us a new function denoted by (T f )( ): Then, solution to (1) will be
given by a xed point of the operator T: Hence, we are trying to nd a V that
satises V = T (V ):
Given a xed point T (V ) = V 2 C(X); we can characterize the policy
correspondence G(x) given by
G(x) = fy 2 (x) : V (x) = F (x; y) + V (y)g :

(3)

Remark 152 It is very important to understand what the operator (T f ) does.

Given a function f; T f is simply the maximized value function in the following
problem
(T f )(x) = max [F (x; y) + f (y)] :
y2 (x)

Hence, if F (x; y) + f (y) and y 2 (x) satises the properties of the Maximum Theorem, we can say something about the maximized value function
T f:
Beyond the Maximum Theorem, in general we would like to show that
T : S ! S (where S is some function space). Hence, if f 2 S; then
T f 2 S: Then we know that the xed point V; T V = V; will be also an
element of S:
We can also use the Corollary to the Contraction mapping theorem to
further characterize V:
Finally, note that T denes a sequence of functions starting from any
guess f 0 (x); by f 1 = (T f 0 )(x): Sometime we will work directly on this
sequence to be able to say something about where these functions converge:
Following theorem establishes that operator T has a unique xed point, and
G(x) is compact-valued and u.h.c.

120

Theorem 153 Under the Assumptions 1 and 2 on X; (x); F; and ;

(i) T : C(X) ! C(X):
(ii) T has exactly one xed point V in C(X):
n
(iii) For any V0 2 C(X); (T n V0 ; V )
(V0 ; V ) for n = 0; 1; 2; : : :
(iv) G : X ! X is compact-valued and u.h.c.
Sketch of the proof: (i) We need to show that T maps set of bounded and
continuous function into itself. Hence, if we take any function f 2 C(X); and
nd (T f ); then it must be the case that (T f ) 2 C(X): Suppose f ( ) is bounded
and continuous. Then, the objective function F (x; y) + f (y) is continuous
and the feasibility set (x) is compact. Therefore, a maximum exists and this
operator is well dened.
Since F (x; y) + f (y) is bounded, T f is bounded as well. Finally, Since
F (x; y) + f (y) is continuous and (x) is compact-valued and continuous, the
maximized value function T f is continuous by the Theorem of Maximum. Hence,
T : C(X) ! C(X):
We established that V = T V is bounded and continuous. This is done by
showing that T V preserves boundedness and continuity properties of V: This
approach is an important tool in dynamic programming problems.
(ii)-(iii) These two results follow directly from the Contraction Mapping Theorem which states that a contraction mapping T on a complete metric space S
has exactly one xed point in S [SLP, (1989), Theorem 3.2]. Note that the
Contraction Mapping Theorem has two ingredients: The metric space S must
be complete, and the operator T must be a contraction mapping. Since the set
C(X) with uniform metric (f; g) = supx2X j f (x) g(x) j is a complete metric
space [SLP, (1989), Theorem 3.1], and T satises the Blackwells Su ciency
Conditions for a contraction mapping [SLP, (1989), Theorem 3.3], the Contraction Mapping Theorem applies to the problem (1).
Since we are maximizing a continuous function on a compact-valued and
continuous feasibility correspondence, the properties of G(x) follow directly from
the Theorem of Maximum.
Lets now go back to our example of one-sector growth model
V (x) =

max

0 y f (x)

fU [f (x)

y] + V (y)g :

In this problem:
X is convex. Since X = [0; x]; where x is the highest maintainable capital
stock, X is a convex subset of R:
(x) is compact-valued. Fix any x 2 X: Since f (0) = 0 and f (x) x; (x)
is bounded. (x) = [0; f (x)] is obviously closed. Hence, it is compact.
(x) is continuous. It is bounded below and above by continuous functions,
therefore it is continuous.

121

F : A ! R is bounded. This follows from U (0) U (f (x)

F is also continuous since U and f are continuous.

y)

U (f (x)):

2 (0; 1):
Hence this problem satises conditions in Assumption 1 and Assumption
2.
Does T dened by
(T V )(x) =

max

0 y f (x)

fU [f (x)

y] + V (y)g ;

satisfy Blackwells su ciency conditions?

Monotonicity. Let W (y)
(T W )(x)

f (y); then
W
max

fU [f (x)
n
U [f (x)

y] + W (y)g ;
o
f (y)
y] + W

max

fU [f (x)

y] + (W (y) + a)g ;

max

fU [f (x)

y] + W (y)g + a

0 y f (x)

max

0 y f (x)

f )(x):
(T W

Discounting. Given W (ynx) + a;

(T (W + a)(x)

=
=
=

0 y f (x)
0 y f (x)

(T W )(x) + a:

In order to characterize V and G further we need more assumptions on X;

(x); F; and :
Assumption 3: 8y, F ( ; y) is strictly increasing. [F is strictly increasing in
the current state].
Assumption 4:
is monotone, x x0 ) (x)
(x0 ): [Feasibility set is
expanding in the current state].
Theorem 154 Under the Assumptions 1-4 on X; (x); F; and ; V is strictly
increasing.
Sketch of the proof: This result follows directly from the Corollary to
the Contraction Mapping Theorem [Stokey and Lucas, (1989), Theorem 3.2,
Corollary 1] which states that if S 0 is a closed subset of S; and T (S 0 ) S 0 then
V 2 S 0 . If in addition T (S 0 ) S 00 S 0 ; then V 2 S 00 .
122

Let S 0 be the set of all bounded and continuous functions, and S 00 be the
set of all bounded-continuous, and strictly-increasing functions. Then, in order
to apply the Corollary we need to show that T maps bounded and continuous
functions into strictly increasing functions.
To do this, suppose V ( ) is non-decreasing and by using the Assumptions on
F and show that T V is strictly increasing. Let x0 > x; we want to show that
T V (x0 ) > T V (x): Let y be the maximizer when the current state is x; i.e.
(T V )(x) = max [F (x; y) + V (y)] = F (x; y ) + V (y );
y2 (x)

since x0 > x; and F (:; y) is strictly increasing, we have

(T V )(x)

=
<

max [F (x; y) + V (y)] = F (x; y ) + V (y )

y2 (x)

F (x0 ; y ) + V (y );

since y is feasible when the current state is x0 ; we have

(T V )(x)

=
<

max [F (x; y) + V (y)] = F (x; y ) + V (y )

y2 (x)

F (x0 ; y ) + V (y )
max0 [F (x0 ; y) + V (y)] = (T V )(x0 ):

y2 (x )

Again lets go back to our example. We have

F ( ; y) is strictly increasing. Note that U and f are strictly increasing.
Hence, x0 > x implies U (f (x0 ) y) > U (f (x) y):
is monotone. Since f is strictly increasing, x
[0; f (x0 )] or (x)
(x0 ):

x0 implies [0; f (x)]

Therefore this example satises Assumption 3 and Assumption 4. Hence,

the solution must be strictly increasing.
We would also like to establish some concavity properties for V: In order to
do this we need further assumptions on F and :
Assumption 5: F is strictly concave,
F ( (x; y) + (1

)(x0 ; y 0 ))

F (x; y) + (1

for all (x; y), (x0 ; y 0 ) 2 A; and for all

x 6= x0 :
Assumption 6:
and y 0 2 (x0 );

)F (x0 ; y 0 );

2 (0; 1) and the inequality is strict if

is convex. For all

2 [0; 1]; and for all x; x0 2 X; y 2 (x)

y 2 (x) and y 0 2 (x0 ) ) y + (1

123

)y 0 2 ( x + (1

)x0 ):

Theorem 155 Under the Assumptions 1-2 and 5-6 on X; (x); F; and ; V
is strictly concave and G is a continuous and single-valued.
Sketch of the proof: The fact that V is strictly concave follows from the
Corollary to the Contraction Mapping Theorem. The arguments are similar to
those in Theorem 1.2, with S 0 being the set of all bounded, continuous, and
weakly concave functions and S 00 being the set of all bounded, continuous, and
strictly concave functions. The fact that G(x) is a single-valued function, call it
g; follows from the Theorem of Maximum under the additional assumptions of
the strict concavity of the objective function and the convexity of the feasibility
set.
Again we want to show that T V (x) is strictly concave, i.e. if x = x + (1
)x0 ; then
T V (x ) > (T V )(x) + (1
)(T V )(x0 ):
Let y be the maximizer for x and y
(T V )(x) + (1

)(T V )(x0 )

be the maximizer for x0 : Then we have

= F (x; y ) + V (y ) + (1
)F (x0 ; y ) + (1
) V (y )
0
= F (x; y ) + (1
)F (x ; y ) + [ V (y ) + (1
) V (y )]
< F (x ; y + (1
)y ) + (V ( y + (1
)y );

where the last step follows from the strict concavity of F and weak concavity of
V: But then,
(T V )(x) + (1

)(T V )(x0 )

= F (x; y ) + V (y ) + (1
)F (x0 ; y ) + (1
) V (y )
0
= F (x; y ) + (1
)F (x ; y ) + [ V (y ) + (1
) V (y )]
< F (x ; y + (1
)y ) + V ( y + (1
)y )
max [F (x ; y) + V (y)] = T V (x );
y2 (x )

where the last step follows since y is feasible given x :

Again for our example:
F is strictly concave. In order to see that, let x; x0 2 [0; x]; and y 2 (x)
and y 0 2 (x0 ): Let x = x + (1
)x0 2 [0; x]; and y = y + (1
)y 0 :
Note that since f is concave and U is strictly concave, hence
)(x0 ; y 0 )] = U [f ( x + (1

F [ (x; y) + (1

U [ f (x) + (1
= U [ (f (x)
> U [f (x)

)f (x0 )
y) + (1
y] + (1

)x0 )

( y + (1

( y + (1

)y 0 )]

)y 0 )]

)(f (x0 )

y 0 )]

)U [f (x0 )

y 0 ]:

is convex. In order to see that, let x; x0 2 [0; x]; and y 2 (x) and
y 2 (x0 ): Hence, that y f (x) and y 0 f (x0 ): Let x = x + (1
)x0 2
0
[0; x]; and y = y + (1
)y . Note that,
0

f (x ) = f ( x + (1

)x0 )

f (x) + (1

)f (x0 )

) y 2 [0; f (x )] = (x ):
124

y + (1

)y 0 = y

Now we also know that the solution to one sector growth problem is a strictly
concave function V:
The next theorem shows that under concavity restrictions, the policy functions associated with operator T converges uniformly. Hence, if gn (x) have
some nice properties that are preserved under uniform convergence, then g(x)
has those nice properties as well.
Theorem 156 Under Assumptions 1-2 and 5-6, if V0 2 C(X) and fVn ; gn g
are dened by
Vn+1 = T Vn ; n = 0; 1; 2; ::::
and
gn (x) = arg max [F (x; y) + Vn (y)] ; n = 0; 1; 2; ::::
y2 (x)

Then, gn ! g pointwise, if X is compact, then the convergence is uniform.

The nal property of the value function that we are interested in is its
dierentiability. In order to establish that we need one more restriction on F:
Assumption 7: F is continuously dierentiable on the interior of A.
The following result is key to establish the dierentiability of V:
Theorem 157 (Benveniste and Scheinkman) Let X
Rl be a convex set,V :
X ! R be a concave function. Let x0 2 intX and let D be a neighborhood of x0 :
If there is a concave, di erentiable function W : D ! R with W (x0 ) = V (x0 )
and W (x0 ) V (x) for all x 2 D then V is di erentiable at x0 and
Vi (x0 ) = Wi (x0 ); for i = 1; 2; :::; l:
Proof: See Figure 54. This theorem states that if you can nd a function W
for which V is like an envelope, then if W is di erentiable at x0 ; so is V and
their derivatives are identical.
The following theorem shows that for dynamic programming problems such
a function is readily available.
Theorem 158 Under the Assumptions 1-2, and 5-7, if x0 2 intX; g(x0 ) 2
int (x0 ); then V is continuously di erentiable at x0 with,
Vi (x0 ) = Fi (x0 ; g(x0 ));
where Vi and Fi are the derivatives with respect to ith argument.
Sketch of the proof: The proof follows from the Theorem of Benveniste
and Scheinkman [Stokey and Lucas, (1989), Theorem 4.10]. The basic idea is
that for any given x0 ; and any neighborhood D of x0 ; the function W : D ! R
given by,
W (x) = F (x; g(x0 )) + V (g(x0 ));
125

V(x)
W(x)

x0

Figure 19: Benveniste and Scheinkman Theorem

satises the conditions of the Theorem of Benveniste and Scheinkman. Note

that W is concave. Also
W (x0 ) = V (x0 ) = F (x0 ; g(x0 )) + V (g(x0 ));
since for x0 , g(x0 ) is the optimal policy. Finally,
W (x)

= F (x; g(x0 )) + V (g(x0 ))

V (x) = max [F (x; y) + V (y)] :
y

One more time lets go back to our example. We know that U (f (x) y) is a
dierentiable function. Hence the solution V is dierentiable. Remember that
for the special case of
f (k) = k and u(k

k 0 ) = ln(k

k 0 );

we had show that (using a guess and verify approach) the value function took
the form
V (k) = A + B ln(k);
where A and B were constant. Not surprisingly, V (k) = A + B ln(k) is strictly
increasing, strictly concave and dierentiable. Note, however, that u(c) = ln(c)
126

is unbounded. Even if we restrict our attention X = (0; k); it is still unbounded

below. Hence, in order to be able to apply these results we must restrict attention to X = [k; k]: We will later see that for this problem even when X = (0; 1);
the solution V is still well dened.
Note that for any dynamic programming problem that satisfy Assumptions
1-7,
V (x) = max [F (x; y) + V (y)]
y2 (x)

that satisfy Assumptions 1-7, we can write the following FOCs for y
Fy (x; y) +

@V (y)
= 0;
@y

where Fy (x; y) indicates the derivative of the return function with respect to y
and @V@y(y) is the derivative function of the value function with respect to y: We
can also write the following envelope condition
@V (x)
= Fx (x; y):
@x
We can combine these two equations to arrive at
Fy (x; y) + Fx (x; y) = 0:
This is the Euler equation.
Remark 159 Remember that when we solved this problem using a Lagrangian
approach, we got the same Euler equation.
Again for our one sector growth model the FOC for y is
U 0 (f (x)

y)( 1) + V 0 (y) = 0;

and the envelope condition is

V 0 (x) = U 0 (f (x)

y)f 0 (x):

Then, the Euler equation is

U 0 (f (x)

g(x))( 1) + U 0 (f (g(x))

g(g(x)))f 0 (g(x)) = 0;

or
U 0 (f (x)

g(x)) = U 0 (f (g(x))

g(g(x)))f 0 (g(x)):

Again we could use (k, k 0 ) notation to write

U 0 (f (k)

k 0 ) = U 0 (f (k 0 )

k 00 )f 0 (k 0 );

or
U 0 (f (kt )

kt+1 ) = U 0 (f (kt+1 )

kt+2 )f 0 (kt+1 ):

We will next go trough several examples to illustrate:

127

How we can write any SP as a dynamic programming (DP) problem.

The properties of the contraction mappings.
Why does a DP approach provide us with a powerful tool to characterize
solutions to SP problems.
Before going into examples, however, we will rst establish a result that will
be useful when the boundedness condition fails.

7.2

Unbounded Returns

Remember that we establish the principal of optimality with two theorems We

showed that under
A1:

(x) 6= ; for all x 2 X;

and
A2: 8x0 2 X and x 2
~

be+1 or

1);

the supremum function V

FE and if its bounded, i.e. if
lim

n!1

(x0 ); limn!1

Pn

t=0

F (xt ; xt+1 ) exists (may

satises the FE, and if a function satises the

V (xn ) = 0; for all (x0 ; x1 ; :::) 2 (x0 ); and for all x0 2 X;

(B)

then V is the supremum function.

It turns out that even if (B) fails, under certain conditions we can come up
with su cient conditions such that any V that solves the FE is a supremum
function. The basic idea is that even if F is unbounded on X; we might still be
able to some up with a function Vb ; that provides and upper bound for V :
Theorem 160 Suppose X; ; F; and
and V

satisfy A1 and A2, and let (x0 ); u(x)

~

as dened before. Suppose there is a function Vb : X ! R such that

(2) lim

n!1

u(x)
~

nb

V (xn )

(1) T Vb

Vb ;

0; for all x0 2 X; and x 2 (x0 );

~

Vb (x0 ); for all x0 2 X; and x 2 (x0 ):

~

If the function V : X ! R dened by

V (x) = lim (T n Vb )(x)

n!1

is a xed point of T; then V = V :

Proof: See SLP, page 93.

128

This theorem simply states that if you can come up with a function that is
an upper bound on V ; then we can work our way from Vb to V; and V = V
Example 161 Consider the following SP
max

fkt+1 g1
t=0

ln(kt

kt+1 )

s.t.
0

kt+1

kt :

Let X = (0; 1): Then the return function is F (x; y) = F (kt ; kt+1 ) = ln(kt
kt+1 ): It turns out that function
Vb (k) =

a ln(k)

satises all of the conditions of the previous theorem. Not surprisingly, the
xed pint V is what we nd using a guess-and-verify method (see SLP page 95).
Where does Vb (k) come from? It is the maximum utility we would get if we had
every period k; and did not save at all. Then each period we would get
ln(k ) =

ln k;

and our discounted lifetime utility would be

Vb (k) =

ln k(1 +

a ln(k)

+ ::::::) =

In many economic applications it is not hard to nd a function like Vb ; even if

F is not bounded.

129

Examples and Exercises

Exercise 162 Consider

P1 the following problem: choose a sequence fct ; kt+1 g for
t 0 to maximize t=0 t u(ct ) subject to
ct + kt+1

kt ;

and k0 > 0 given. The functions u maps R+ to R and are strictly increasing
and continuous, while 2 (0; 1): Let X be the state space, let F : X X ! R
be the return function, and let : X ! X be the feasible set. Specify X; F , and
for the above problem.
Exercise 163 Consider
the following problem: choose a sequence fct ; kt+1 ; nt g
P1
for t 0 to maximize t=0 t u(ct ; 1 nt ) subject to
ct + kt+1

f (kt ; nt ) + (1

)kt ; 0

nt

1;

and k0 > 0 given. The functions u and f map R+ R+ to R+ and are strictly
increasing and continuous, while 2 (0; 1) and 2 [0; 1]: Let X be the state
space, let F : X X ! R be the return function, and let : X ! X be the
feasible set. Specify X; F , and for the above problem.
Exercise 164 Consider
the following problem: choose a sequence fct ; n1t ; n2t ; k1t+1 g
P1
for t 0 to maximize t=0 t U (ct ; 1 n1t n2t ) subject to
ct = f (kt ; n1t ) and kt+1 = n2t ;

2
to R+ and both are strictly
and k0 > 0 given. The functions U and f map R+
increasing, continuous, and strictly concave, while 2 (0; 1): Let X be the state
space, let F : X X ! R be the return function, and let : X ! X be the
feasible set. Specify these objects for the above problem.

Exercise 165 Consider the following model. There is one person, two inputs
(capital and labor) and two outputs (a consumption
good and an investment
P1
good). The person has preferences given by t=0 t u(ct ); where u : R+ ! R
and ct is date t output of the consumption good. The production function for
output of the date t consumption good is G1 (k1t ; n1t ) and that for output of the
investment good is G2 (k2t ; n2t ); where k1t + k2t
kt and n1t + n2t
1: At
date 0; k0 is given and kt+1 = (1
)kt + G2 (k2t ; n2t ); where 2 (0; 1): Aside
from the discount factor, the components of a sequential problem as dened in
Stokey, Lucas, with Prescott are as follows: X ( a set in which the state variable
lies), : X ! X (a constraint set), and F : X X ! R (a return function).
Describe X; ; and F for the above model.
Exercise 166 Let X = R+ , F (x; y) = 0 for all (x; y); and (x) = [0; x= ],
where
2 (0; 1) and is the discount factor. Here X is the state space, F is
the return function, and is the feasible set. (a) Show that v(x) = x satises
the associated functional equation.(b) Is v(x) = x the maximized value of the
objective in the corresponding sequential problem? Explain.
130

Exercise 167 Consider the following economy populated by a continuum of

identical agents. Time is discrete and the horizon is innite. Each individual
owns a set of trees at time 0, denoted by s0 : In each period t, the individual has
to cop some trees to produce (and consume) fruits. The technology to produce
fruits is denoted by
yt = f ( t ; nt );
where t is the amount of trees cut in period t; and nt is the labor used to
cut trees. Each individual is endowed with 1 unit of time each period. Agents
decide how much to work and how many trees to cut in each period in order to
maximize
1
X
t
u(ct ); with 2 (0; 1);
t=0

where ct is consumption of fruits at time t:

Example 168 In Stokey, Lucas and Prescott (1989), a dynamic programming

problem is dened by
(the discount factor); X (the set in which the state
variable lies), : X ! X (feasibility correspondence), and F : X X ! R (the
return function). Describe X; ; and F for this model.
Set up the dynamic programming problem faced by a representative agent.
Write down the rst order condition to the agents DP problem and interpret it.
Let
f ( t ; nt ) =
and
u(ct ) =

n1t

c1t
1

; with

; with

2 (0; 1);
2 (0; 1):

Show that the sequence of chopped trees will follow the following equation
t+1

1
+

t:

Argue that the relationship between the initial set of trees s0 and the se1
quence of chopped trees f t gt=0 can be used to pin down 0 ; and hence to
1
dene the whole sequence f t gt=0 :
Exercise 169 This problem tries to get you acquainted with the Principal of
Optimality (i.e. Theorems 4.2 and 4.3 in Stokey, Lucas and Prescott 1989)
Consider the consumption-saving decision of an innitely lived agent with initial
assets x0 2 X = R: The agent can borrow or lend at rate 1 + r = R = 1 > 1:
Hence, price of borrowing one unit for tomorrow is : There are no borrowing
constraints, i.e.
ct + xt+1 xt

131

Hence, the agents problem is

V (x0 ) =

sup
fct ;xt+1 g1
t=0

1
X

ct :

t=0

s.t.
0

ct xt
xt+1 :
x0 given.

What is the value of V (x0 )? Consider the following FE that corresponds to this
problem
V (x) = sup [x
y + V (y)] :
y

Show that both V (x) = 1 and Ve (x) = x are solutions to the FE. Does Ve
satisfy boundedness condition in Theorem 4.3 (page 72 in SLP). Remember that
when we solved a SP using a Lagrangian approach, we needed the Transversality
Condition (TC). Hence, if a sequence of actions satisfy the Euler equation, it
was the optimal solution if it also satised the TC. Argue (with some carefully
selected sentences) that the TC and the bounded condition in Theorem 4.3 (page
72 in SLP) serve the same purples.
Example 170 Consider the standard one sector growth model,
k 0 )) + V (k 0 )g:

fU (f (k)
V (k) = max
0
k

By using FOCs for this problem show that if V is strictly concave k 0 = g(k) is
increasing in k: Remember that the FOC for k 0 is
U 0 (f (k)

k 0 )( 1) + V 0 (k 0 ) = 0:

Note that this equation characterizes k 0 given V 0 : Hence, we can use the Implicit
Function Theorem to dene
H(k 0 ; k) = U 0 (f (k)

k 0 )( 1) + V 0 (k 0 ):

Then,
g 0 (k) =

dk 0
=
dk

Hk
=
Hk 0

U 00 (f (k) k 0 )f 0 (k)( 1)
:
U 00 (f (k) k 0 )( 1)( 1) + V 00 (k 0 )

Since U 00 < 0; and V 00 < 0; g 0 (k) > 0: Using FOCs and the Implicit function
theorem to characterize g is a powerful tool that is often used. Note that often
we know if V is strictly increasing or not, or concave or not.
Example 171 Consider the following version of the one sector growth model:
Again each agent start his/her life with k0 ; and lives forever. Agents has to
decide each period how much to work (hence labor supply is endogenous) and
132

how much to save. Each agent has one unit of time. Let nt denote the labor
supply. Then, the production function is now
f (kt ; nt ) = F (kt ; nt ) + (1

)kt ;

and the utility function is

U (ct ; 1

nt ):

Then, the agents problem is

max 1

fkt+1 ;nt gt=0

1
X

U (f (kt ; nt )

kt+1 ; 1

nt ):

t=0

Let write down the DP problem facing an agent. Note that the capital stock is
still the only state variable in this economy (i.e. at each date agents only need
to know kt in order to be able to make decisions on nt and kt+1 ): Then,
V (kt ) = max [U (f (kt ; nt )

kt+1 ; 1

kt+1 ;nt

nt ) + V (kt+1 )] :

We now have two FOCs. One is for kt+1

U1 (f (kt ; nt )

kt+1 ; 1

nt )( 1) + V 0 (kt+1 ) = 0;

and one is for nt

U1 (f (kt ; nt )

kt+1 ; 1

nt )f2 (kt ; nt ) + U2 (f (kt ; nt )

kt+1 ; 1

nt )( 1) = 0;

where Ui and fi indicate the derivative w.r.t. ith argument. The FOC for nt
does not involve V: We will call it a static FOC. Note that it simply states that
U1 (f (kt ; nt )
|

kt+1 ; 1
{z

nt )f2 (kt ; nt ) = U2 (f (kt ; nt ) kt+1 ; 1

} |
{z

m arginal benet of w ork

m argin al cost of w ork

nt ):
}

When we have a static FOC, we can always solve it rst and then focus on the
dynamic FOC. In this example, we can use the FOC for nt to nd the optimal
work decision
nt = N (kt ; kt+1 );
and then substitute it in the FOC for kt+1
U1 (f (kt ; N (kt ; kt+1 ))

kt+1 ; 1

N (kt ; kt+1 ))( 1) + V 0 (kt+1 ) = 0:

Note that now everything is in terms of kt and kt+1 in this equation.

Example 172 Consider the stochastic case of the standard one sector growth
model. Let the output be given by f (k; z) where z is a stochastic technology
shock:
f (k; z) = zF (k; 1) + (1
):

133

A representative agent now sees zt ; and then make a decision on kt+1 : Hence,
a representative agent needs to know both kt and zt in order to be able to decide
on kt+1 : Then, the SP will be
"1
#
X
t
max1 E0
U (f (kt ; zt ) kt+1 ) :
fkt+1 gt=0

t=0

The DP problem is then

V (k; z) =

max

0 k0 f (k;z)

fU [f (k; z)

k 0 ] + E [V (k 0 ; z 0 )]g ;

where z 0 is the value of next period shock which is unknown at the current period
when k 0 is chosen, and E( ) is the expected value operator. Hence, we have two
new features in the stochastic case:
First, the state of the problem now consists of both the current capital stock
k and current shock z: Therefore, in order to be able to analyze how the
solution of this problem behaves over time, we need to keep track of both
k and z.
Second, we need to characterize what we mean by the expression E( ).
Example 173 Consider again the stochastic version of the one-sector growth
model with = 1: Hence,
f (k; z) = zf (k) = F (k; 1):
Furthermore, suppose z can only take values from a nite set Z = fz1 ; z2 ; : : : ; zn g :
In this case a probability distribution over Z is simply an assignment of probabilities ( 1 ; 2 ; : : : ; n ) to each element of Z. Since i s are probabilities,
i

= Pr(z = zi ),

0 8i; and

We can then write the DP problem as

(
V (k; zj ) =

max

0 k0 f (k)zj

U [f (k)zj

k ]+

n
X
i=1

n
X

= 1:

i=1

V (k ; zi )

; for each j:

Example 174 Now let z can only take values from a nite set Z = fz1 ; z2 ; : : : ; zn g ;
but follow a Markov process. Let ij denote the probability that the next periods
shock will be zj given that todays shock is zi . The matrix formed by ij for all
i; j is called a transition matrix.
2
3
:::
11
12
1n
6 21
7
:::
22
2n 7
=6
4
5
:::
n1
n2
n3
134

The dening properties of a transition matrix are:

X
i) ij 0 for all i; j and ii)
ij = 1 for all i:
j

Then, the DP problem is

8
<
V (x; zi ) = max
F (x; y; zi ) +
y2 (x;zi ) :

n
X

V (y; zj )

j=1

ij

9
=
;

Example 175 Finally suppose Z = [z; z]: Then we have to dene a distribution
function for z: Suppose z is iid with
H(a) = prob(z
Then, we can write the DP problem as
(
V (k; z) =

max

U [f (k; z)

0 k0 f (k;z)

k ]+

a):

z
0

[V (k ; z )] dH(z ) ;

On the other hand if the future distribution of z depend on its current realization,
i.e. if
H(ajz) = prob(z 0 ajz);
we will write
V (k; z) =

max

0 k0 f (k;z)

U [f (k; z)

k ]+

z
0

[V (k ; z )] dH(z jz) :

Exercise 176 Consider the following functional equation,

Z
(T w)(x) =
w(x0 )dF (x0 ; x) + g(x):
a) Let C(X) be the set of all bounded and continuous functions on X
Under what conditions on F ( ; x) and g(x), T : C(X) ! C(X) ?
b) Show that T is a contraction mapping.

Rl :

Exercise 177 Consider the following functional equation

(T V )(k) = max fW (k); V (f (k))g ; 0 <

< 1; a > 0:

where W (k) is a continuous, increasing, bounded and strictly concave function dened on [0; k] and f (k) is a continuous, increasing and bounded function
dened on [0; k]: Does T map strictly concave functions into strictly concave
functions? Prove or provide a counter-example.

135

Exercise 178 Prove the following: Operator T dened by

(T V )(x) = max

x
1

; V (x) ; with

2 (0; 1) and x 2 [0; x];

on the set of bounded and continuous functions T is a contraction mapping

Exercise 179 Let f ( ; ) : Rn ! Rn and satisfy Blackwells su cient conditions for a contraction mapping. Here 2 R is a parameter. Let g( ) denote the
xed point of f; namely, the unique solution to x = f (x; ): Prove the following:
If f (x; ) is decreasing in ; then g( ) is decreasing in :
Exercise 180 Let the mapping T be dened by
T (v)(x) =

max fU [f (x)

0 y f (x)

y] + v(y)g

where the function U : R+ ! R and the function f : R+ ! R+ are continuous

and strictly increasing and 2 R: Prove the following.
If v : R+ ! R is continuous, then T (v) is continuous.
If v : R+ ! R is continuous, then T (v) is strictly increasing.
Exercise 181 Consider the following two-sector model of optimal growth. A
social planner seeks to maximize the utility of the representative agent given by
1
X

u(ct ; lt );

t=0

where ct is consumption and of good 1 at t; whereas lt is leisure at t:Sector 1

produces consumption goods using capital k1t and labor n1t according to the production function ct f1 (k1t ; n1t ): Sector 2 produces the capital good according
to the production function kt+1 f2 (k2t ; n2t ): Total employment nt = n1t + n2t
and leisure lt is constraint by the endowment of time l; lt + nt l: The sum of
the amounts of capital used in both sectors cannot exceed the initial capital in
the economy, k1t + k2t kt ; k0 given.
Formulate this problem as a dynamic programming problem.
Consider now another economy that is similar to the previous one except for the fact that capital is sector specic. The economy starts period
t with given amounts of capital k1t and k2t that must be used in sector
1 and 2 respectively. During this period the capital-good sector produces
capital that is specic to each sector according to transformation curve
g(k1t+1 ; k2t+1 )
f2 (k2t ; n2t ): Formulate this problem as a dynamic programming problem.

136

Exercise 182 A rm maximizes the present value of cash ows, with future
earnings being discounted at rate : Income at time t is given by sales pt :qt ;
where pt is the price of good and qt is the quantity produced. The rm behaves
competitively and therefore takes prices as given. It knows that the prices evolve
according to a law of motion given by pt+1 = f (pt ): Total or gross production
depends on the amount of capita, kt ; and labor nt and on the square of the
di erence between current ratio of sales to investment xt and the previous period
ratio. This last feature captures the notion that changes in the ratio of sales to
investment require some reallocation of resources within the rm. It is assumed
that the wage rate is constant and equal to w: Capital depreciates at rate : The
rms problem is
1
X
t
max
[pt qt wnt ] ;
t=0

subject to

qt + xt

"

g kt ; nt ;
kt+1

(1

qt
xt

qt
xt

2
1
1

)kt + xt

pt+1 = f (pt )
q 1
> 0 given.
k0 > 0 and
x 1
We assume that g is bounded, increasing in rst two arguments and decreasing
in last argument. Formulate this problem as a dynamic programming problem.
Exercise 183 A workers instantaneous utility unction u(:) depends on the
amount of market goods consumed c1t and also amount of home produced goods
c2t : In order to acquire market produced goods the worker must allocate some
amount of time l1t to market activities that pay a salary of w: The worker takes
wages as given. It is known that the wages move according to wt+1 = h(wt ):
The quantity of home produced goods depend on the stock of expertise that the
worker has at the beginning of the period, which we label by at : This stock of
expertise depreciates at rate and can be increased by allocating time to nonmarket activities. Hence, the agents problem is
max

1
X

u(c1t ; c2t )

t=0

subject to

at+1

c1t

wt lt ;

c2t

f (at );

(1
l1t + l2t

137

)at + l2t ;
l;

wt+1 = g(wt );
a0 > 0 given.
It is assumed that u and g are bounded and continuous. Formulate this problem
as a dynamic programming problem.
Exercise 184 Consider the problem of choosing a consumption sequence ct to
maximize
1
X
t
(ln ct + ln ct 1 ); 0 < < 1; > 0;
t=0

subject to

ct + kt+1

Akt ; A > 0; 0 <

< 1; ko > 0 and c

given.

Here ct is consumption at t, and kt is capital stock at the beginning of period

t. The current utility function ln ct + ln ct 1 is designed to represent habit
formation in consumption.
1. For formulate the dynamic programming problem of a representative agent.
2. Prove that the solution to this dynamic programming problem is of the form
v(k; c 1 ) = E + F ln k + G ln c 1 where c 1 is the lagged consumption and
E; F; and G are constants.. Prove that the optimal policy function is of
the form ln kt+1 = I + H ln kt where I and H are constants. Give explicit
formulas for E; F; G; H; and I:
Exercise 185 Consider the following optimization problem.:
max

1
X

u(at )

t=0

P1
subject to t=0 at s; and at 0:This problem which is a dynamic programming problem with = 1; is known as the cake-eating problem. We begin with
a cake of size s and we need to allocate this cake to consume in each period of
an innite horizon.
Show that this problem may be written as a dynamic programming problem.
That is describe formally the state and choice spaces, the reward function,
and the feasibility correspondence.
Show that if u : A ! R is increasing and linear, i.e. u(a) = ka; for some
k > 0 then the problem always has at least one solution. Find a solution.
Show that if u : A ! R is increasing and strictly concave, then the problem
has no solution.

138

Exercise 186 Write down the Bellman equation for the following problem.
Consider the problem of a worker who faces a wage o er w every period from
a distribution F (W ) = prob(w
W ): If the worker accepts the o er he commits to work for at wage w forever. If he declines the o er, he can search one
more period and have a new wage o er from the distribution F: If the worker is
unemployed, he receives unemployment compensation c:
Exercise 187 Write down the Bellman equation for the following problem.
Consider the problem of a rm which faces a rm specic productivity shock
st each period. The price p for the rms product and wage rate w are constant over time. Output is a function of employment level nt and st and given
by f (nt ; st ): Upon observing st ; rm decides how much labor to employ for the
current period. Changing the level of employment implies an adjustment cost
that is given by g(nt ; nt 1 ): The shocks that rm faces next period depends on
the value of the shock in current period. Firm has also the option of exit for the
next period.
Exercise 188 Consider the problem of a monopolist introducing a new product.
The monopolist desires to maximize the present-value of his prots. He faces
a constant interest rate r: In each period the monopolist faces the downward
sloping demand curve described by
p = D(q) with D(0) = p; D(1) = p and D0 (q) < 0:
That is, according to demand schedule D the monopolist can sell q units of
output at a price of p: Marginal revenue, or D(q) + qD0 (q); is assumed to be
strictly decreasing in q: Let per unit cost of production be given by c. This cost
declines convexly with production experience, e; according to
c = C(e); with C(0) < p; C(1) > p; C 0 (e) < 0; and C 00 (e) > 0:
Production experience is taken to be the cumulative sum of past production, so
the law of motion governing experience is given by
e0 = e + q:
a)Formulate the monopolist dynamic programming problem. Hint: Cast the
relevant maximization problem in terms of the production experience variable.
b) Compute the rst order condition associated with this problem. Interpret it.
Prove that the decision-rule for q is strictly increasing. c) Consider the problem
of myopic monopolist who maximizes current period prots. Who produces the
most: the myopic or farsighted monopolist?
Exercise 189 Consider the following version of one sector growth model. Economy is populated by identical representative agents who want to maximize
X
t
Eo
u(ct ; nt ); with 0 < < 1;
139

subject to
ct + gt + kt+1

F (kt ; nt );

and
0

nt

1; ct

0; kt+1

0; and k0 given.

Here ct is time t consumption, nt is hours worked, and gt is government spending. Government spending does not create any utility to the consumers. It is
iid and drawn from the following cumulative distribution each period
Pr [gt

a] = G(a):

Agents make decisions after they observe the level of government spending. Assume u1 > 0; u2 < 0; u11 < 0; u22 < 0; u21 < 0; and F1 > 0; F2 > 0; F11 < 0;
F22 < 0: Write down the dynamic programming problem for a representative
agent (or equivalently for a central planner).
1. Drive the FOC for nt : Show that

dnt
dgt

0:

2. Drive the FOC and Euler equation for kt+1 : Is kt+1 increasing or decreasing in gt ?
3. Will your results about the properties of policy function on part c change,
if gt was serially correlated?
Exercise 190 Consider the following savings-consumption problem for an innitely lived agent
"1
#
X
t
2
max U = E0
(a + "t )ct bct
t=0

s:t: at+1
and ao

= Rat + yt
> 0 given

ct

Here yt is time-t realization of a given nonrandom income stream, ct is time-t

consumption, and "t is a test shock. R is the gross interest rate of return on
keeping assets from one period to the next. Let R = 1 :
Write down the value function and FOC for a for this problem.
Assume that
"t = "t

t;

0<

< 1 and

How does the FOC looks like now?

Exercise 191 Assume that
u(c) =

c1
1

140

> 0:

t ~iid

Assume that the gross interest rate Rt is independently and identically distributed and is such that E(Rt1 ) < 1= : Consider the problem of a representative agent who wants to maximize
E

1
X

u(ct ); 0 <

< 1;

t=0

subject to
At+1

Rt (At

ct ); and Ao given,

where At is period t assets. It is assumed that ct must be chosen before Rt is

observed. Show that the optimal policy function takes the form ct = At and
give an explicit formula for : Hint: Consider a value function of the general
form v(A) = BA1 ; for some constant B:
Exercise 192 Consider the following version of one sector growth model where
a representative agent tries to maximize
max

1
X

U (ct ; lt )

t=0

subject to
ct + it = F (kt ht ; lt )
and
kt+1 = (1

(ht ))kt + it (1 + "t )

and
k0 > 0 given
Here ct is consumption and lt is labor supply. Production for a given capital stock kt and labor supply lt is given by F (kt ht ; lt ): The variable ht is the
intensity of factor utilization for capital which is a choice variable. When ht
is high, you get more services form a given capital stock. Using capital stock
more intensely, however, has a cost. It makes capital stock depreciate faster,
i.e. 0 > 0: Moreover, let 0 < < 1 and 00 > 0: Hence in this economy depreciation is not xed, but given by function (h): There is an investment specic
technology shock here. The productivity of the exiting capital at time t; kt ; is
not a ected by this technology shock. When "t is high, however, it is cheaper
to make investment since a given unit of [F (kh; l) c] can create more investment and a higher capital stock next period. Let "t be an autocorrelated random
variable with cumulative distribution ("t+1 j"t ); and let "t 2 Q = [", "]:
Let
U (ct ; lt ) = U (ct

G(lt ));

with U 0 > 0; U 00 < 0; G0 > 0; G00 > 0: Find the marginal rate of substitution between consumption and labor given this particular utility function.
What is the signicance of the MRS that this particular utility function
implies?
141

Write down the dynamic programming problem faced by this consumer.

Write down the FOCs for kt+1 ; ht and lt : Write down also the Euler equation for kt+1 : Interpret each of these FOCs.
Using the FOC for kt+1 ; show that if the value function is concave, then
dkt+1
dkt+1
d"t > 0: Try to interpret d"t :
Exercise 193 Consider the following version of the one sector growth model
max

fct ; kt+1 g1
t=0

1
X

u(ct );

t=0

subject to
ct + kt+1
k0
ct

F (kt ; lt );

> 0 is given,
0; kt+1 0; lt

lbt :

The agent has k0 > 0 units of capital at time 0. The agent also has 2 units of
labor endowment in even periods and 1 units of labor endowment in odd periods,
i.e.
(b
l0 ; b
l1 ; b
l2 ; :::) = (2; 1; 2; 1; ::::);

where b
lt denotes the labor endowment at time t: Formulate the problem of maximizing consumer utility subject to feasibility constraint as a dynamic programming problem.
Exercise 194 Consider the following version of the one sector growth model
with linear utility
1
X
t
ct ;
max 1
fct ; kt+1 gt=0

t=0

subject to

ct + kt+1
k0
ct

f (kt );

> 0 is given,
0; kt+1 0:

Here f (kt ) represents the total amount of goods available at time. The function
f is strictly increasing, strictly concave with f (0) = 0: The representative agent
1
has k0 units of capital at time 0; chooses a sequence of fct ; kt+1 gt=0 to maximize
P
1
t
ct :
t=0
Write down the dynamic programming problem associated with this sequential problem.

142

Write down the FOC, the Envelope condition, and the Euler equation associated with the dynamic programming problem. Using the Euler equation
show that the optimal decision rule for an interior solution is
kt+1 = g(kt ) = (f 0 )
where (f 0 )

=k ;

indicates the inverse of f 0 function.

Suppose kt+1 = g(kt ) is an interior decision for all kt ; nd the maximized

value function V (k0 ):
Do you expect kt+1 = g(kt ) to be the optimal decision for any kt ? Explain.
Describe the optimal decision rule as fully as possible.

143

Deterministic Dynamics

Consider the following sequential problem (SP)

max1

fxt+1 gt=0

1
X

F (xt ; xt+1 );

(SP)

t=0

subject to
xt+1 2 (xt ); t = 0; 1; 2; :::;
and
x0 given,
where F is a bounded return function and is a feasibility correspondence.
We know that (see Chapter 4 of SLP) the maximized value function
V (x0 ) =

max1

fxt+1 gt=0

1
X

F (xt ; xt+1 );

t=0

satises the following functional equation (FE);

V (x) = max [F (x; y) + V (y)] ;
y2 (x)

(FE)

As an example consider the FE for the standard one-sector growth model

V (k) =

max

0 k0 f (x)

fU (f (k)

k 0 ) + V (k 0 )g ;

(52)

where k is the current period capital stock, k 0 is the next period capital stock
to be chosen, f ( ) is the production function, U ( ) is the utility function, and
is the discount factor. Assume that,
1. f : R+ ! R+ ; and U : R+ ! R are continuous, strictly concave, strictly
increasing, and continuously dierentiable.
2.

2 (0; 1)

3. f (0) = 0; limk!0 f 0 (k) = 1; limk!1 f 0 () = 0; and limc!o U 0 (c) = 1:

Under these assumptions, we know that:
1. There exists unique maximum maintainable capital stock k > 0 such that
f (k) = k; and if kt > k then kt+1 f (kt ) < kt : Hence the set of possible
values for the state variable k can be restricted to [0, k]:
2. Functional equation (52) has a unique, bounded and continuous solution
on [0, k]:
3. V is strictly increasing and strictly concave.

144

4. The maximum in (52) is attained at a unique value g(k); and the policy
function g( ) is continuous.
5. Given any k0 ; the sequence kt+1 = g(kt ) denes an optimal path for the
capital stock, i.e. it is a solution for SP.
We want to characterize g( ) in order to understand how the optimal sequence of the capital stock behaves over time. In particular, we would rst like
to nd stationary points of g (the points where g(k) = k); and then try to gure
out if and how the economy moves over time towards these stationary points.
We also know under these assumptions that the solution is everywhere interior, and g(k) is characterized by the rst order and the envelope conditions
(52),
U 0 (f (k) g(k)) = V 0 (g(k));
(53)
and,
V 0 (k) = U 0 [f (k)

g(k)]f 0 (k):

(54)

Claim: The policy function g( ) is increasing.

Proof: Let k and k o be such that k, k o 2 [0; k] and k 0 > k: We want to show
that g(k 0 ) > g(k): Consider the FOCs for k and k 0 ;
U 0 (f (k)
U 0 (f (k 0 )

g(k)) = V 0 (g(k));
g(k 0 )) = V 0 (g(k 0 )):

Take the ratio of these two equalities to get ,

U 0 (f (k)
U 0 (f (k 0 )
Suppose g(k 0 )

g(k))
V 0 (g(k))
=
:
g(k 0 ))
V 0 (g(k 0 ))

g(k): Since k 0 > k; we have f (k 0 ) > f (k); then since U 00 < 0;

U 0 (f (k)
U 0 (f (k 0 )

and since V 00 < 0;

g(k))
> 1;
g(k 0 ))

V 0 (g(k))
V 0 (g(k 0 ))

1;

leading to a contradiction.

9.1

Stationary Points of g( )

Note that k = 0 is a trivial stationary point of g( ); since g(0) = 0: An economy

that start with no capital will be stuck there. In order to characterize the
stationary points, lets evaluate the rst order and envelope conditions at g(k) =
k to get,
U 0 (f (k) k) = V 0 (k);
(55)

145

and
V 0 (k) = U 0 [f (k)

k]f 0 (k):

(56)

These will give us the necessary condition for a stationary point

f 0 (k) =

(57)

Since f ( ) is continuous and strictly increasing, there will be a unique k satisfying (57) given by,
k = (f 0 ) 1 (1= ):
(58)
This is our unique candidate for a stationary point.
We showed that g(k) = k implies (58) (the necessary condition), if we can
also show that (58) implies g(k) = k; we will have a full characterization of the
stationary points of g( ):
In order to get the su cient condition, note that the strict concavity of V
implies that for any k and k 0 ;
[V 0 (k)
[V 0 (k)

V 0 (k 0 )][k
V 0 (k 0 )][k

k0 ]
k0 ]

0; and
= 0 if and only if k = k 0 :

In particular, let k 0 = g(k); then

[V 0 (k)
[V 0 (k)

V 0 (g(k))][k
V 0 (g(k))][k

g(k)]
g(k)]

0; and
= 0 if and only if k = g(k):

By using (53) and (54), we get

V 0 (g(k)) =

U 0 [f (k)

g(k)];

and,
V 0 (k) = U 0 [f (k)

g(k)]f 0 (k):

leading to
[V 0 (k)

V 0 (g(k))] = U 0 [f (k)

g(k)](f 0 (k)

):

Since U 0 ( ) > 0;
[f 0 (k)
[f 0 (k)

1= ][k
1= ][k

g(k)]
0; and
g(k)] = 0 if and only if k = g(k):

Hence, k is a stationary point, since LHS of the inequality is zero at k . Therefore,

k = k , [f 0 (k) 1= ] = 0 , k = g(k):

146

In order to characterize the behavior of the economy out of the steady state,
note that since k is the unique steady state, if k 6= k ;
[f 0 (k)

1= ][k

g(k)] < 0:

Since f ( ) is concave,
f 0 (k)

( ) 1= , k

( )k :

Therefore,
g(k) > (<) k , k < (>) k ;
and g(k) will look like Figure 20.
This analysis shows that: (i) there is a unique steady state k > 0; (ii)
given any k0 > 0; economy will converge to k (global stability), and (iii) the
convergence is monotone.

g(k)
k*

k*

Figure 20: g(k) for one sector growth model

9.1.1

A Non-monotone Example

Consider an economy with two sectors, one producing consumption goods and
the other producing investment goods. The agents have one unit of time. The

147

production of the consumption goods ct require both labor nt ; and capital kt ,

and is given by,
kt
ct = nt f
;
nt
with f ( ) is a neoclassical production function, and limn!0 nf (k=n) = 0:
The production of capital goods require only labor,
kt+1 = 1

nt :

The dynamic programming problem faced by a representative agent is

V (k) = max

U (1

y2[0;1]

y)f

+ V (y) ;

(59)

where k is the current capital stock, and y = 1 n is the next period capital
stock to be chosen. The production function for capital goods imply that y 2
[0; 1]: [Check that V ( ) is strictly increasing and strictly concave, and the policy
function g : [0; 1] ! [0; 1] is continuous.]
Lets nd the stationary points of g( ): In this example, k = 0 is no more a
stationary point. Since capital goods are produced by labor only, an economy
with k = 0 will not be stuck there.
The rst order and envelope conditions for this problem are given by,
U 0 [1

g(k)]f

k
g(k)

V 0 (k) = U 0 [1

k
g(k)

g(k)]f

k
f0
g(k)
1

1
k
g(k)

f0

k
g(k)

= V 0 (g(k));

k
:
g(k)

Setting g(k) = k; we get the necessary condition for a stationary point,

f

k
1

f0

k
1

= 0:

(60)

Under the assumptions on f ( ) there is again a unique k satisfying the necessary

condition.
Since V ( ) is strictly concave we can show that this condition is also su cient. Using,
[V 0 (k)
[V 0 (k)

V 0 (g(k))][k
V 0 (g(k))][k

g(k)]
g(k)]

0; and
= 0 if and only if k = g(k);

k
g(k)

we get,
U 0( ) f 0

k
g(k)

1
1

k
f0
g(k)

k
g(k)

[k g(k)]

hence,
U 0( )

f0

k
g(k)

k
g(k)
148

k
g(k)

[k

g(k)]

0;

0;

and since U 0 ( ) > 0; (60) is also a su cient condition for a stationary point.
We again have a unique stationary point k which is characterized by (60).
But it is obvious that g( ) will not look like Figure 20. If k is near to 0, it is
optimal to spend most of the time to produce capital, hence near g(0) will be
close to one and g( ) will be decreasing.
In order to investigate this possibility further, let
U (c) = c ; and f (z) = z where 0 <

< 1; and 0 <

< 1:

Then the FOC becomes,

U

k (1

k
g(k)

)k

(1

g(k))

k
g(k)

k
g(k)

= V 0 (g(k));

resulting in
(1

(g(k))

(1

) 1

= V 0 (g(k)):

Claim: g(k) is decreasing.

Proof: Take any k and k 0 ; and let k 0 > k: Suppose g(k 0 )
for k and k 0 we have,
k
k0

1 g(k)
1 g(k 0 )

(1

) 1

g(k): From FOCs

V 0 (g(k))
:
V 0 (g(k 0 ))

If g(k 0 ) = g(k); the LHS<1, and RHS=1. If g(k 0 ) > g(k); LHS<1, and
RHS>1. Hence g(k) is decreasing.

10

Euler Equations

Consider the Euler equation for the standard one sector growth model,
U 0 (f (kt )

kt+1 ) = U 0 (f (kt+1 )

kt+2 )f 0 (kt+1 );

(61)

where U ( ) and f ( ) are utility and production functions, is the discount factor,
and kt is the capital stock. We know that given any initial capital stock k0 ; the
sequence of capital stocks fkt g that satises (61) and the transversality condition
is an optimal solution for this problem. Note that (61) is a (non-linear) second
order di erence equation in kt : Hence, analyzing how the optimal capital stock
moves over time amounts to analyzing how this dierence equation behaves. As
we have already shown that, in this example there exists a unique stationary
capital stock given by, f 0 (k ) = 1= ; and it is possible to characterize how
economy moves from any given k0 towards k :
For a general dynamic programming problem (X; ; A; F; ) where,
1. X is a convex subset of Rl ;
2.

: X ! Y is a non-empty, compact-valued, continuous, and convex

correspondence,
149

3. F : A ! R is a bounded, continuous, strictly concave, and continuously

dierentiable on the interior of A;
4.

2 (0; 1);

the Euler equations are given by,

Fy (xt ; xt+1 ) + Fx (xt+1 ; xt+2 ) = 0;

(62)

and the transversality conditions are given by,

lim

t!1

Fx (xt ; xt+1 )xt = 0:

(63)

Hence, any sequence of state variables fxt g satisfying (62) and (63) is a
solution for SP given x0 .
For the one-sector growth model
F (k; k 0 ) = U (f (k)

k 0 );

therefore,
Fy (xt ; xt+1 ) = U 0 (f (kt )

kt+1 )( 1)

and,
Fx (xt+1 ; xt+2 ) = U 0 (f (kt+1 )

kt+2 )f 0 (kt+1 )

which gives us (61). In order to be able to analyze the equations like (61) we
need to study how to solve dierence equations.

10.1

Solving Linear Dierence Equations

Consider a system of rst-order linear dierence equations

xt+1 = Axt ;

(64)

where xt is a (l 1) vector, and A is an (l l) matrix of coe cients. If we had

an initial value for x; then the period-t solution would be xt = At x0 :
There is no loss of generality in analyzing a rst order system of from (64)
since:
1. Any higher order system of linear equations can be written as a rst order
system. Consider a q-th order system given by,
zt+1 = B1 zt + B2 zt

+ : : : + B q zt

This can be written as rst order system as,

2
3 2
B1 B2
Bq 1
zt+1
6
6
7
I
0
0
z
t
6
7 6
6
7=6 0
I
0
6
7 6
4
5 4
0
0
I
zt q+2
|
{z
} |
{z
xt+1

where xt and xt+1 are (lq

Bq
0
0
0

32
76
76
76
76
54
}|

1) vectors and A is a (lq

150

(65)

q+1:

zt
zt

zt

q+1

{z
xt

3
7
7
7
7
5
}

lq) matrix.

2. Any non-homogenous rst order system can be written as a homogenous

system. Consider a non-homogenous system given by,
zt+1 = C + Azt :

(66)

If (I A) is non-singular, the steady-state of (66) is given by z = C + Az;

and therefore, z = (I A) 1 C: Let xt = zt z; and xt+1 = zt+1 z; then
xt+1 = zt+1

z = C + Azt

z = C + Azt

Az = A(zt

z) = Axt :

In order to analyze the behavior of xt = At x0 we need to know how At

evolves.
Remark 195 Note that if l = 1 and a is a scalar, then xt = at x0 ! 0 if jaj < 1:
Note that if A is a diagonal matrix, the it is easy to analyze At . Therefore,
it is a good idea to diagonalize A: To do that lets introduce some denitions:
Denition 196 Given an l l matrix A; an eigenvector e of A is a l 1 vector
that is mapped by A into its scalar multiple. That is Ae = e for some scalar
: If e is an eigenvector, it must solve,
(A

I)e = 0:

Remark 197 Note that an l l matrix is a linear operator in Rl : Then an

eigenvector is almost like a xed point of this operator in the following sense:
Ae scales e by a constant :
Denition 198 (A
chosen so that

I)e = 0 will have non-zero solutions if and only if

det(A

is

I) = 0:

The values of that satisfy this equation are called eigenvalues of A: For each
eigenvalue i ; then (A
I)e = 0 will give the corresponding eigenvector ei :
Denition 199 A square matrix A is diagonalizable, if there exists an invertible matrix P such that P 1 AP is diagonal.
Theorem 200 Let A be an l l matrix. If l eigenvectors of A are linearly
independent, then it is diagonalizable, and B 1 AB = ; where B is the matrix
of eigenvectors, and
is a diagonal matrix with the eigenvalues of A in the
diagonal.
Theorem 201 If l eigenvalues of A are all di erent, then its eigenvectors are
linearly independent, hence A is diagonalizable.

151

10.1.1

Two-dimensional case

Let l = 2; and consider

xt = a11 xt + a12 yt ;
and
yt = a21 xt + a22 yt :
Then,
xt+1
a11 a12
xt
=
yt+1
a21 a22
yt
{z
}| {z }
| {z } |
zt+1

Let 1 and
izable as

2;

with

6=

zt

2;

be the eigenvalues of A: Then, A is diagonal1

=B

AB;

where
b11
b21

B=

b12
b22

with [b11 b21 ]0 and [b12 b22 ]0 are the eigenvectors corresponding to
and
0
1
=
:
0
2

and

2;

Now in order to be able to use ; we want to transform our original system

so that we end with something like zbt+1 = zbt ; where zb denotes the transformed
variable. To this end note that
1

zt+1 = B

1
A(BB 1 )zt = B
| {zAB}(B
| {z }

zt ) = B

zt :

Let zbt = B

zt : Then,

or
Then,

x
bt+1
ybt+1

x
bt+1 =
x
bt =

bt
1x

t
b0
1x

and ybt+1 =
and ybt =

x
bt
ybt

bt :
2y

t
b0 :
2y

We can now recover our original variables using zt = Bb

zt ,
xt
yt

b11
b21

b12
b22

which implies
xt = b11

t
b0
1x

+ b12

152

t
b0
1x
t
b0
2y
t
b0 ;
2y

(67)

and
yt = b21
1

Finally, substituting zbtt = B

x
b0
yb0

where eb denotes elements of B

t
b0
1x

zt for
"
eb
= e11
b21
1

t
b0 :
2y

+ b22

(68)

t = 0; i.e. using,
#
eb12
xt
;
eb22
yt

; we have

x
b0 = eb11 x0 + eb12 y0 and yb0 = eb21 x0 + eb22 y0 :

Then, equations (67) and (68) become:

xt = b11
and
yt = b12

t e
1 (b11 x0

t e
1 (b11 x0

+ eb12 y0 ) + b12

+ eb12 y0 ) + b22

t
2

t
2

eb21 x0 + eb22 y0 ;

eb21 x0 + eb22 y0 :

Now, it is obvious that the behavior of (69) and (70) depend on

If j

1j

< 1 and j

2j

< 1; then xt ! 0 and yt ! 0:

If j
0):

1j

> 1 and j

2j

> 1; then xt !

If j

1j

< 1 and j

2j

> 1; then the system is stable only if

1 and yt !

(69)

(70)
1

and

1 (unless x
b0 = yb0 =

eb21 x0 + eb22 y0 = 0:

(71)

Hence, there is a particular set of initial conditions that put the system
into a stable path (usually referred as a saddle path). In other words,
initial conditions y0 and x0 are not independent of each other.
In this case, i,e, if (71) holds, we have
xt = b11
And

t e
1 (b11 x0

+ eb12 y0 ) and yt = b21

xt =

b11
yt ;
b21

t e
1 (b11 x0

+ eb12 y0 ):

and we know exactly how xt and yt are related for all t > 0:

153

(72)

10.1.2

General Case

If A in xt+1 = Axt is diagonalizable, then

=B

A=B B

AB; or

and we have
xt = B

x0 :

(73)

Let again b represent the elements of B; and eb represent the elements of B

Then,
3
2
b11 t1 b12 t2
b1l tl
t
t
t
6 b21
b22 2
b2l l 7
1
7;
B t=6
5
4
t
t
t
bll l
bl1 1 bl2 2
and,

Therefore,

2 Pl
e
j=1 b1j xj;0
6 Pl e
6
j=1 b2J xj;0
B 1 x0 = 6
4
Pl e
j=1 blj xj;0

7
7
7:
5

Hence for any k; xk;t is given by,

xk;t =

l X
l
X
i=1 j=1

bki tiebij xj;0 =

l
X
i=1

bki

t
i

3 2 Pl Pl e t
b1i bij xj;0
x1;t
Pli=1 Plj=1 e it
6 x2;t 7 6
i=1
j=1 b2i i bij xj;0
6
7=6
4
5 6
4
Pl Pl e t
xl;t
i=1
j=1 bli i bij xj;0
2

0
@

l
X
j=1

7
7
7:
5
1

ebij xj;0 A :

(74)

Note that this exactly equations (69) and (70) we derived for the two dimensional
case. We are interested in the behavior of xt as t ! 1: From (74) we have the
following immediate result.
Theorem 202 If all eigenvalues of A are less then 1 in absolute value, then
xt ! 0 as t ! 1; since ti ! 0 for all i:
Suppose not all of the eigenvalues are less than one in absolute value, but only
m < l of them are less then one in absolute value. Without loss of generality,
suppose that rst m of the eigenvalues are less than one in absolute value and
last l m are greater than one in absolute value. Then it is obvious from (74)
that we need additional restrictions on the initial values:

154

Theorem 203 Suppose j i j < 1 for 1 i

m; and j i j > 1 for i > m:
Pl
limt!1 xt = 0 if and only if xj ;0 satises the restrictions j=1 ebij xj;0 = 0 for
all i > m:
Pl
Proof: (Su ciency) If j=1 ebij xj;0 = 0 for all i > m; then
xk;t =

m
X

bki

i=1

0
1
l
X
t@
ebij xj;0 A ;
i
j=1

and the e ects of unstable

are killed.
Plroots
e
(Necessity) Suppose
j=1 bij xj;0 6= 0 for some i > m: There must exists
some k such that bki 6= 0: For this k; limt!1 xt 6= 0 since
0
1
m
l
X
X
ebij xj;0 A ! 1:
xk;t =
bki ti @
i=1

j=1

Pl
Note that the condition j=1 ebij xj;0 = 0 for all i > m is exactly equation
(71) that we got for the two dimensional case.
This theorem implies that for the stability of the system the initial values
associated with the stable eigenvalues must form a basis for the system. To
see this note that the condition on P
the initial values for the state variables
l
associated with unstable eigenvalues, j=1 bij xj;0 = 0 for all i > m; imply that
the l m initial values associated with unstable eigenvalues are functions of
m initial values associated with stable eigenvalues. In other words, the initial
values associated with the instable eigenvalues should not be exogenous to the
system.
Specically,
l
X
j=1

or,

ebij xj;0 = 0 for all i > m )

2 e
bm+1;m+1 ebm+1;m+2
6 ebm+2;m+1 ebm+2;m+2
6
4
ebl;m+1
ebl;m+2
{z
|

(l m) (l m)

m
X
j=1

ebm+1j xj;0 +

ebm+1;n 32 xm+1;0
ebm+2;n 76 xm+2;0
76
54
ebn;n
xl;0
{z
}|

(l m) 1

l
X

j=m+1

7
7 =
5
}

ebm+1j xj;0 = 0;
2 Pm
e
j=1 bm+1;j xj;0
6 Pm e
6
j=1 bm+2;j xj;0
6
4
Pm e
j=1 bl;j xj;0

Hence, the initial values associated with unstable eigenvalues can not be independent of the initial values associated with stable eigenvalues. The subspace
in Rm formed by the initial values associated with stable eigenvalues is called
the stable manifold of the system.
155

7
7
7:
5

Note that although we studied the linear dierence equations, there is no

reason for (62) to be linear in x: Indeed, most probably these equations will be
nonlinear. If the objective function F is quadratic, however, the Euler equations
will be linear and we can apply the results from the linear dierence equations.
For nonlinear cases we will try to linearize the Euler equations around the steady
state and study the linearized versions.

10.2

Quadratic Return Functions

Suppose F (x; y) is a quadratic function. Then the rst derivatives of F (x; y)

can be written as
Fx (x; y) = F x + Fxx x + Fxy y;
and,
0
Fy (x; y) = F y + Fxy
x + Fyy y;

where F x and F y are l 1 vectors of constants, Fxx , Fyy , and Fxy are l l
0
is the transpose of Fxy . Note that the rst
matrices of constants, and Fxy
derivative of a quadratic function of (x; y) can only have a constant, terms with
x; or terms with y: Hence, for the quadratic case the Euler equations become
0
F y + Fxy
xt + Fyy xt+1 + fF x + Fxx xt+1 + Fxy xt+2 g = 0;

which can be written as

0
F y + F x + Fxy
xt + (Fyy + Fxx )xt+1 + Fxy xt+2 = 0

(75)

To nd the conditions for stationary points, lets evaluate these equations at

x = xt = xt+1 = xt+2 . This gives
0
F y + F x + Fxy
x + (Fyy + Fxx )x + Fxy x = 0

If (Fxy + Fyy + Fxx + Fxy ) is non-singular, then a unique stationary point

will be given by
x=

(Fxy + Fyy + Fxx + Fxy )

(F y + F x ):

Let the deviations from the steady state be denoted by zt = xt

non-singular, then we have
1

Fxy1 Fxy zt +

Fxy1 (Fyy + Fxx )zt+1 + zt+2 = 0:

x: If Fxy is
(76)

Since this is a system of second order linear dierence equation in zt we can

write it as a rst order system,
zt+2
zt+1

J
I

K
0

zt+1
zt

where
J

1
1

Fxy1 (Fyy + Fxx );

0

Fxy1 Fxy ;
156

(77)

and I and 0 are l

l matrices. Let
zt+1
zt

Zt =

and A =

J
I

K
0

Then the dynamics of the system can be represented by

Zt+1 = AZt :

(78)

Hence we need to analyze the eigenvalues A which is a 2l 2l matrix. Following

lemma shows that there is a upper bound in the number of eigenvalues that can
be less than one in absolute value in the quadratic case.
Lemma 204 Assume Fxy and (Fxy + Fyy + Fxx + Fxy ) are non-singular, let
1
A be dened as in (78), then if is a characteristic root of A, so is ( ) .
Proof: If is a characteristic root of A; then (A
I) will be singular, that
is for some stacked vector x 6= 0 with x0 = (x01 ; x02 ) 2 R2l ;
J
I

x1
x2

0
0

which lead the following set of equations,

(J

I)x1 + Kx2 = 0 and x1

x2 = 0:

If Fxy and (Fxy + Fyy + Fxx + Fxy ) are non-singular, one can show that A
will be non-singular (left as an exercise). If A is non-singular, then 6= 0: Then
x1 = x2 ; and the two equations reduces to
2

(K + J
2

implying that (K + J

I)x2 = 0;

I) is singular. Hence,

det(K + J

I) = 0;

and
is the characteristic root of A if and only if it satises this equation.
Using the values of K and J this equation becomes
det(

0
Fxy1 Fxy

Fxy1 (Fyy + Fxx )

I) = 0;

since Fxy is non-singular,

det(

0
Fxy
+

(Fyy + Fxx ) +

Fxy ) = 0:

(79)

Hence is characteristic root of A if and only if (79) holds. Let b = ( ) 1 ;

we need to show that (79) also holds for b: If we replace with b in equation
(79), we get
det(

0
Fxy
+b

(Fyy + Fxx ) + b Fxy ) = 0;

157

or
det(
since (

0
Fxy
+

(Fyy + Fxx ) +

Fxy ) = 0:

) is a constant
2

) det(

0
Fxy
+

(Fyy + Fxx ) +

Fxy ) = 0:

Note that determinant is zero if and only if (2.5) holds. Hence if

A so is ( i ) 1

(80)
is a root of

Previous Lemma implies that the roots of A come in almost reciprocal pairs
and we can have at most l roots smaller than one. Indeed if exactly l of the
eigenvalues are smaller than one in absolute value then we have the following
global stability result.
Theorem 205 Let F : R2l ! R be a strictly concave, quadratic function. Let
(x) = Rl for all x 2 Rl , and 0 < < 1: Assume that Fxy and (Fxy + Fyy +
Fxx + Fxy ) are non-singular, and let x be the unique stationary pint. Assume
A has l characteristic roots less than one in absolute value, then 8x0 2 Rl , there
exits a unique solution fxt g to the optimization problem. This sequence satises
(75) and has limt!1 xt = x:
To gain some intuition about this theorem, let l = 1; then
zt+2
zt+1

c
1

d
0

zt+1
zt

(81)

where c and d are some constants. We know from our analysis of two-dimensional
case that if one of the eigenvalues is less than one in absolute value and the other
one is greater than one in absolute value, then we need the following condition
eb12 z1 + eb22 z0 = 0:

(82)

Note that this restriction implies a relation between z1 and z0 : The basic idea is
that for any z0 ; z1 can be chosen to satisfy this equation. This will be the case
if eb12 6= 0: Suppose not, i.e. eb12 = 0: Then, both (z0 = 0; z1 = 0) and (z0 = 0
and z1 = "); would satisfy this restriction contradicting the fact that there is at
most one sequence that is optimal.

10.3

Linear Approximations

This Theorem gives us a powerful tool for analyzing the stability of dynamic
programming problems with quadratic return functions. What happens when
the return function is not quadratic, and the Euler equations are not linear. The
following theorem deals with this problem and develops a local stability result.
Theorem 206 Suppose the dynamic programing problem (X;
ises
158

; A; F; ) sat-

X is a convex subset of Rl : : X ! X is a non-empty, compact-valued,

and continuous correspondence.
F : A ! R is bounded and continuous, and 0 <

< 1:

F is strictly concave,
F ( (x; y) + (1

)(x0 ; y 0 ))

for all (x; y), (x0 ; y 0 ) 2 A; and for all

if x 6= x0 :
is convex. For all
y 2 (x0 );
0

F (x; y) + (1

)F (x0 ; y 0 );

2 (0; 1) and the inequality is strict

2 [0; 1]; and for all x; x0 2 X; y 2

y 2 (x) and y 0 2 (x0 ) ) y + (1

)y 0 2 ( x + (1

(x) and
)x0 ):

F is continuously di erentiable on the interior of A.

Let x be a stationary point. Assume further that F is twice continuously
di erentiable in a neighborhood N of (x; x): Let Fxx ; Fxy ; and Fyy be the matrices of second derivatives of F evaluated at (x; x); and assume that Fxy and
(Fxy + Fyy + Fxx + Fxy ) are non-singular. Let A be dened as
zt+2
zt+1

=
|

J
I

K
0
{z }

zt+1
zt

1
1
Fxy1 Fxy : Suppose A has l charwhere J =
Fxy1 (Fyy + Fxx ); and K =
acteristic roots less than one in absolute values, then there exists a neighborhood
U of x such that if x0 2 U; then fxt g satises limt!1 xt = x:

Consider the standard one-sector growth model where F (x; y) = U (f (x) y),
the steady state k is characterized by
f 0 (k ) = 1;
and the Euler equation is given by
U 0 (f (kt )

kt+1 ) + f 0 (kt+1 )U 0 (f (kt+1

kt+2 ) = 0:

The linear approximation of this equation around k is given by

0

00

U ( )f 0 (k )(kt k ) + U 00 ( )(kt+1 k ) + f 00 (k )U 0 ( )(kt+1

f 0 (k )U 00 ( )f 0 (k )(kt+1 k )
f 0 (k )U 00 ( )(kt+2 k ):

159

k )+

Substituting the steady state condition f 0 (k ) =

0

U 00 ( )(kt

U 00 ( )(kt+2

k ) + (1 +

)U 00 ( ) + f 00 (k )U 0 ( ) (kt+1

k )

k );

where U ( ) indicate that the function is evaluated at the steady state values.
Hence we have the second order system given by
(kt+2

k )=

(kt

k ) + (1 +

)+

As a rst order system we have,

"
00
0
kt+2 k
(1 + 1 ) + Uf 00 =f
0
=U
=
kt+1 k
1
|
{z

f 00 (k )U 0 ( )
(kt+1
U 00 ( )

kt+1 k
kt k

k ):

Hence A is the matrix which governs the behavior of this linearized system
around k :
We could also derive A using the formulas from previous Theorem,
= U 0 (f (x) y)f 0 (x);
= U 00 (f (x) y)(f 0 (x))2 + U 0 (f (x)
=
U 0 (f (x) y);
= U 00 (f (x) y);
=
U 00 (f (x) y)f 0 (x);

Fx
Fxx
Fy
Fyy
Fyx

y)f 00 (x);

which will give us after evaluating at the steady state

0
Fxy
+ Fyy + Fxx + Fxy

U 00 (f (x)

y)f 0 (x) + U 00 (f (x)

00

U (f (x) y)(f (x)) + U (f (x)

U 00 (f (x) y)f 0 (x)
U 0 f 00

and

Fxy =

y) +
y)f 00 (x)

U 00 ;

which are non-zero (non-singular). We can then nd

J

=
=

1
1+

Fxy1 (Fyy + Fxx ) =

1

f 00 =f 0
;
U 00 =U 0

160

1
2
(U 00 + U 00 (f 0 ) + U 0 f 00 )
U 00

and
1

K=

0
Fxy1 Fxy
=

leading to A
A=

J
I

K
0

"

(1 + 1 ) +
I

f 00 =f 0
U 00 =U 0

In order to analyze the dynamics of this problem we need to nd two

eigenvalues of A: From matrix algebra we know that 1 + 2 =trace(A) and
1 2 = det(A): Under the standard assumptions on preferences and technology
f 00 =f 0
U 00 =U 0 > 0: Hence, both the trace and the determinant are positive. Therefore
both of the eigenvalues must be positive.
Characteristic equation for A is given by
R( ) =

trace(A) + det(A) =

Note that
R(0) =
R(1) =
and
R(1= ) =

(1 +

)+

f 00 =f 0
U 00 =U 0

f 00 =f 0
;
U 00 =U 0
1 f 00 =f 0
:
U 00 =U 0

It is easy to show that and one of the eigenvalues of A is greater than 1

and other is less than one. Hence the system is stable around the steady state.
00
0
Furthermore, as Uf 00 =f
=U 0 becomes larger, the smaller eigenvalue which determines
the convergence of the system becomes closer to one. Hence, the curvature in
the technology speeds up convergence and the curvature in preferences retard
it.
Finally, for this example, we have
kt+1

k =

1 (kt

k );

where 1 is the eigenvalue that is less than 1 in absolute value. Again from our
analysis of the two dimensional case, we know that this should correspond to
kt+1

k =

b11
(kt
b12

k ):

Let b = [b11 b12 ]0 be the eigenvector associated with

"
#
00
=f 0
1
b11
(1 + 1 ) + Uf 00 =U
0
=
b12
1
0
161

1;

then

b11
b12

which indeed implies that

b11 =

1 b12

b11
=
b12

1:

Example 207 SLP (1989), p.157, Exercise 6.7, a-c.

a) Given
n
o
V (k) = max k (1 y) (1 ) + V (y) ;
0 y 1

where

2 (0; 1). The rst order and envelope conditions are given by
k

(1

k0 )

)(1

(1

) 1

( 1) + V 0 (k 0 ) = 0;

and
V 0 (k) =

k0 )

(1

(1

leading to the Euler equation

k

(1

k0 )

)(1

(1

) 1

k0

(1

k 00 )

(1

b) Let g(k) = k; then the necessary condition for a stationary point is given
by,
1
=
)k =
:
1 k
k
1
+
Note that k is the unique candidate for a stationary point. It is easy to verify
that it is also su cient.
c) Lets linearize the Euler equation around k ,
( 1)
(k )
(
( 1)
divide by ( 1)
have

1
(k )
(1
)(1 (k ) (1 ) 1 )[kt k ] +
(1
)( (1
) 1)(1 (k ) (1 ) 2 )[kt+1 k ] +
2
1)(k )
(1 (k ) (1 ) )[kt+1 k ] +
( (1
))(k ) (1 (k ) (1 ) 1 )[kt+2 k ];

( (1
(kt+2

))(k ) (1

(k )

k ) + B(kt+1

where
B=

(1

) 1
1

k )+

(1
) 1
+
(1
)

) and substitute k . Then we

(kt

k ) = 0;
:

Hence,
(kt+2
(kt+1

k )
k )

=
|

0
{z
A

(kt+1 k )
(kt k )

and the behavior of the system depends on the roots of A. Note that 1 +
1
B; and 1 2 = det(A) =
: You can easily check what
2 =trace(A) =
conditions we need to have one of the roots being greater than one and other
being less than one in absolute value.
162

11

Stochastic Case Finite Number of Shocks

Consider the stochastic case of the standard one sector growth model. Output is
given by f (k)z where z is a stochastic technology shock. The Bellman equation
for this problem will be
V (k; z) =

max

fU [f (k)z

0 k0 f (k)z

k 0 ] + E [V (k 0 ; z 0 )]g ;

(83)

where z 0 is the value of next period shock which is unknown at the current
period when k 0 is chosen, and E( ) is the expected value operator. Hence, we
have two new features in the stochastic case: First, the state of the problem
now consists of both the current capital stock k and current shock z: Therefore,
in order to be able to analyze how the solution of this problem behaves over
time, we need to keep track of both k and z. Second, we need to characterize
what we mean by the expression E( ).

11.1

Preliminaries

Suppose z can only take values from a nite set Z = fz1 ; z2 ; : : : ; zn g : In this
case a probability distribution over Z is simply an assignment of probabilities
( 1 ; 2 ; : : : ; n ) to each element of Z. Since i s are probabilities,
i

= Pr(z = zi ),

0 8i; and

n
X

= 1:

i=1

We can then rewrite (83) as

V (k; z) =

max

0 k0 f (k)z

U [f (k)z

k0 ] +

n
X
i=1

V (k 0 ; zi )

(84)

In this setup, assignment of probabilities to the elements of Z will also dene

an assignment of probabilities for any subset of Z: For any set A
Z, let
IA = fi : zi 2 Ag be the set of indices of elements of Z that belong to subset A:
We can dene Pr(z 2 A); call it (A); as follows
X
(A) =
i where IA = fi : zi 2 Ag :
i2IA

Denition 208 Function ( ) denes a probability measure over any given family Z of subsets of Z that includes ; and Z if it satises
i) (A)

0 8A 2 Z , ii) (;) = 0 and (Z) = 1; and

iii) for any A1 ; A2 ; : : : of pairwise disjoint sets in Z, ([i Ai ) =

X
i

163

(Ai ):

How large can we pick the family of subsets of Z and dene a probability
measure over it? Obviously, when Z is nite, we can take Z to be any family
of subsets of Z; including the set of all subsets of Z; and dene a probability
measure. If Z is not nite, then we can not dene a probability measure on all
subsets of Z that has the obvious adding-up property for disjoint unions. What
properties a family of subsets must have so that we can dene a meaningful
probability measure over it?
Denition 209 Let S be a set and S be a family of subsets of S. S is called a
-algebra if
a); 2 S, S 2 S
b)A 2 S )Ac = S n A 2 S

c)An 2 S, n = 1; 2; : : : ) [n=1 An 2 S.
Hence the main properties of a -algebra are closures under complementation
and countable union.
Denition 210 (S; S) where S is a -algebra is called a measurable space, and
any A S is called a measurable set.
Denition 211 (S; S, ) where S is a -algebra and
over S is called a probability space.

is a probability measure

If S is a nite set the family of all subsets of S obviously constitutes a algebra. The family of all subsets of a nite set S is called the complete -algebra
for S; and it is the -algebra routinely used for nite sets.
Denition 212 Given a measurable space (S; S), a real-valued function f :
S ! R is a measurable function w.r.t. S if
fs 2 S : f (s)

ag 2 S for all a 2 R;

If the space is a probability space f is called a random variable.

Note that when S is a nite set, this denition is trivial. For any a 2 R
there must be certain number of values of S for which f (s) a; and these sets
necessarily belongs to the complete -algebra for S: Therefore, if S is a nite
set, and S is the complete -algebra for S; then any function f : S ! R is
measurable w.r.t. S.
Denition 213 Let Z be a nite set, and
P Z be the complete -algebra of Z:
Then given a probability space (Z; Z; ), i f (zi ) (zi ) for all zi 2 Z is called
the expected value of the random variable f w.r.t the distribution :
Hence, the problem (83) with nite number of shocks is a well-dened problem. We can dene the probability measure ( ) over any subset of shocks and
dene the expected value of any function.
164

11.2

Transition Functions

We would also like to analyze the problem where the expected value of the
next period shock depends on the current value of the shock. An autoregressive
process over the shocks will be such a case. Then a general form of (83) will be
V (x; z) = max fF (x; y; z) + E [V (y; z 0 ) j z]g ;
y2 (x;z)

(85)

where E [V (y; z 0 ) j z] denotes to fact that the expected value of V (y; z 0 ) depends
on z: Suppose Z was a nite set. Then instead of assigning unconditional
probabilities for each element of Z; we need to dene conditional probabilities
or transition probabilities. Let ij denote the probability that the next periods
shock will be zj given that todays shock is zi . Then (85) becomes
8
9
n
<
=
X
V (x; zi ) = max
F (x; y; zi ) +
V (y; zj ) ij :
(86)
;
y2 (x;zi ) :
j=1

The matrix formed by

for all i; j is called a transition matrix.

2
3
:::
11
12
1n
6 21
7
:::
22
2n 7
=6
4
5
:::
n1
n2
n3
ij

(87)

The dening properties of a transition matrix are:

X
i) ij 0 for all i; j and ii)
ij = 1 for all i:
j

Each row of a transition matrix must sum up to one. Note that each row of
a transition matrix denes a probability measure over the all subsets of Z: A
transition matrix naturally denes a transition function.
Denition 214 Let Z be a nite set and Z be the complete -algebra of Z: A
transition function : Z Z ! [0; 1] satises:
i) for all z 2 Z; (z; ) is a probability measure,
ii) for all A 2 Z, ( ; A) is a measurable function.
Note that for every combination of current shock and a subset of next period
shocks
assigns a probability. Hence, a transition function does two things:
First, for any given value of current shock z; it denes a probability measure
over next period shocks (each row of matrix
above). Second, for any given
subset of next period shocks it nds the probability distribution over current
shocks of moving into that subset .
Using a transition matrix we can dene two important operators:

165

Denition 215 For any Z-measurable function f; the Markov operator T over
f is given by
X
(T f )(zi ) =
f (zj ) (zi ; zj ); all zi 2 Z:
j

Denition 216 For any probability measure over (Z; Z); adjoint of Markov
operator T over is given by
X
(T )(A) =
(zi ; A) (zi ); all A 2 Z:
i

Note that T f (zi ) is the expected value of f next period if the current shock
is zi , and T (A) is the probability that next period shock lies in the set A; if
the current state is drawn according to the probability measure : Hence, T
denes a probability measure over next period, if is the probability measure
over the current period.

11.3

Policy Functions and Transitions Functions

Consider the following dynamic programming problem

8
n
<
X
V (x; zi ) = max
F (x; y; zi ) +
V (y; zj )
y2 (x;zi ) :

ij

j=1

9
=
;

(88)

where z 2 Z = fz1 ; z2 ; : : : ; zn g ; x 2 X; and the transition probability are given

by elements of transition matrix = [ ij ] :
The state of this problem is s = (x; z) 2 X Z; and as for the deterministic
case, we would like to know how s moves over time. Behavior of z is governed
by which is exogenously given. On the other hand behavior of x is governed
by the optimal policy function g : X Z ! X; Q
and it is part of the solution.
Hence, the behavior of s is governed by g and
together. Indeed, g and
dene a transition function P over (S; S) where S = X Z as follows
P [(x; zk ) , A

B] =

(zk ; B) if g(x; zk ) 2 A
;
0 if g(x; zk ) 2
=A

where x 2 X; zk 2 Z; A 2 X, and B 2 Z.
Suppose X is also a nite set. Then X is the complete -algebra for X:
Given the current state (x; zk ), P [(x; zk ) , A B] gives the probability that
next period state will be in the set A B: This will happen with probability
(zk ; B) if the optimal choice x0 is in the set A:
Example 217 Consider the following dynamic programming problem from Greenwood, Hercowitz, and Haufman (AER, 1988)
Z
V (kt ; "t ) =
max
U (ct ; lt ) +
V (kt+1 ; "t+1 )d ("t+1 j "t )
(89)
ct ; kt+1 ; lt ; ht

166

s.t.

kt+1
kt
+
(1
1 + "t
1 + "t

ct = F (kt ht ; lt )

(ht )):

Let k 2 K = fk1 ; k2 ; : : : ; kn g ; and " 2 E = f"r ; "s g where "r = e

"s = e s 1: The transition matrix for " is given by
=
where
sr +

rs
ss

= Pr "0 = e s 1 j " = e
= 1: Then (2.6) becomes
"
V (ki ;

r)

rr

rs

sr

ss

U (c; l) +

max

c; k0 ; l; h

1 ; 0

1; and

ij

2
X

rs V

rr

rs

= 1; and

(k 0 ;

s)

s=1

s.t.
ct = F (ki h; l)

k0 e

+ ki e

Suppose there exists a unique value of k 0 = k 0 (ki ;

Hence,
Pr [k 0 = kj j k = ki ; =

r]

(1

r)

(h)):

2 K for all (ki ;

r)

2 K E:

1 for some j 2 f1; 2; : : : ; ng

:
0 otherwise

Using this probabilities we can dene a transition matrix P over the state space
of this problem. Note that the state is given by
S = f(k1 ;
which has 2

r ); (k2 ; r ); : : : ; (kn ; r ); (k1 ; s ); (k2 ; r ); : : : ; (kn ; r )g

n elements, and P will be given by

P = [pir; js ] = Pr [k 0 = kj j k = ki ; =

r]

rs

(90)

for all i; j = 1; 2; : : : ; n and for all r; s = 1; 2:

Then, analyzing the behavior of the state of this problem over time, as we
did for non-stochastic case in Recitations #6 and #7, amount to analyzing how
this matrix P behaves over time.

12

Markov Chains

Let Z = fz1 ; z2 ; : : : ; zl g be a nite set and l be the l-dimensional unit simplex.

Then a probability distribution p over Z is given by a row vector in l
(
)
l
X
l
l
p2
= p 2 R : p 0;
pi = 1 :
i=1

167

Let

=[

ij ]

be a transition matrix (Markov chain) over Z

l
X

0;

ij

ij

= 1:

j=1

Suppose p is the probability distribution over the current state, then the
distribution over next period state is given by
2
3
:::
11
12
1l
6 21
7
:::
22
2l 7
pl 6
pb = p = p1 p2
4
5
:::
:
:
:
l1
l1
ll
h P
i
Pl
Pl
l
=
:
p
p
:
:
:
p
i=1 i i1
i=1 i i2
i=1 i il

P
Pl
P
Note that pb 2 l since
bj =
jp
i=1 pi
j ij = 1: Similarly, if p is the
probability distribution over current state then the probability distribution over
two periods ahead is given by (p ) = p(
) = p 2:
As we did with the deterministic case, we would like to know what happens
to the state of the problem as we follow the optimal policy from any given inial
value. In the deterministic case the optimal policy rule g(x) provided us with
all the information we need about the behavior of the model.
Suppose we are given some initial probability distribution p0 over Z. The
long run behavior of the state space is given by p0 n . On the other hand, if the
initial state is zi , the probability distribution over states n period ahead will be
simply the ith row of n . The steady state for this problem will obviously be a
stationary probability distribution over Z.
Denition 218 A set E
Z is called an ergodic set, if (zi ; E) = 1 for all
zi 2 E; and if no other proper subset of E has this property.
Hence, if the current state is in an ergodic set then with probability one next
period will be also in this ergodic set. It is obvious that there will be a close
link between ergodic sets and the stationary distribution.
Denition 219 An invariant distribution p over Z is a probability distribution
such that p = p :
Our main concern is with the conditions under which p0

12.1

Examples

Example 220 Let l = 2, and suppose

=

3=4 1=4
1=4 3=4

168

! p as n ! 1:

In this example Z itself is the only ergodic set, and

1=2 1=2
1=2 1=2

lim

n!1

=Q=

converges to

Then for any initial distribution p0 the invariant distribution is given by p =

(1=2; 1=2) ; and each row of Q is an invariant distribution:
1=2

1=2

3=4 1=4
1=4 3=4

1=2

1=2

Example 221 Let l = 3; and suppose

2
3
1
=2
=2
1=2 1=2 5 :
=4 0
0
1=2 1=2

In this example if the current state is z1 ; then there is a positive probability

that next period state will be z2 or z3 : But once we reach z2 or z3 there is no
possibility of going back to z1 : A state such as z1 is called a transient state. Here
E = fz2 ; z3 g is the only ergodic set. Moreover,
3
3
2
2
n
0 1=2 1=2
(1
)
n =2
n =2
n
1=2
1=2 5 ! Q = 4 0 1=2 1=2 5 ;
=4 0
0
1=2
1=2
0 1=2 1=2
where 2 (0; 1) and n = 1 (1
)n : Whatever the initial distribution p0 ,
the invariant distribution is given by p = (0; 1=2; 1=2) : Hence, economy will
eventually leave the transient state and will end up in the ergodic set:
3
2
1
=2
=2
1=2 1=2 5 = 0 1=2 1=2
0 1=2 1=2 4 0
0
1=2 1=2
Note that in these two examples there is a unique ergodic set and a unique
invariant distribution. Economy will eventually enter into the ergodic set regardless of where it starts.
Example 222 Suppose
=

0
2

where
2n

and
(

are k

(l

n
2)

0
(

k) and (l
n

1)

and

0
k)

2n+1

;
k Markov matrices. Then
(

0
n
1)

(
2

n
2)

In this example there is only one ergodic set Z itself. But ergodic set has
cyclically moving subsets. If the system begins in C1 = fz1 ; z2 ; : : : ; zk g Z then
after any even number of periods it will be back in the set C1 ; and after every
169

odd number of periods it will be in the set C2 = Z n C1 : Reverse will happen if

the system begins in the set C2 : In this example contrary to the rst two n do
not converge. Let
3=4 1=4
;
1 = 2 =
1=4 3=4
then
2n

Q 0
0 Q

Note that although

2n+1

and

0 Q
Q 0

where Q =

does not converge, the following

2
1=4 1=4 1=4
N
1
1 X n 6
1=4 1=4 1=4
=6
lim
4 1=4 1=4 1=4
N !1 N
n=0
1=4 1=4 1=4

and each row of this matrix is an invariant

2
0
0
3=4
6 0
0
1=4
1=4 1=4 1=4 1=4 6
4 3=4 1=4 0
1=4 3=4 0

average does
3
1=4
1=4 7
7
1=4 5
1=4

distribution:
3
1=4
4=4 7
7 = 1=4
0 5
0

Example 223 Suppose

1=2 1=2
1=2 1=2

1=4

1=4

1=4

where 1 and 2 are k k and (l k) (l k) Markov matrices. In this

example there are two ergodic sets, E1 = fz1 ; z2 ; : : : ; zk g and E2 = Z n E1 : If
the economy starts in the set E1 it will stay there forever and if the economy
starts in the set E2 it will also stay there forever. In this example
n

and

n
1

converges if and only if

and

Then
lim

n!1

0
n
2

0
2

converge. Let

3=4 1=4
1=4 3=4

1=2 1=2
6 1=2 1=2
=6
4 0
0
0
0

3
0
0
0
0 7
7:
1=2 1=2 5
1=2 1=2

In this case there are two invariant distributions p1 = (1=2; 1=2; 0; 0) and p2 =
(0; 0; 1=2; 1=2) : Note also that any convex combinations of p1 and p2 are also
invariant distributions. Contrary to the previous examples, where the economy
170

will end up depends on the initial distribution p0 : If p0 = ( ; 1

; 0; 0) then p1
will be invariant distribution. On the other hand if p0 = (0; 0; ; 1
) then p2
will be the invariant distribution. Suppose, for example, p0 = (1=3; 1=3; 1=3; 0)
then p = (1=3; 1=3; 1=6; 1=6) which is a convex combination of p1 and p2 .
Example 224 Let l = 3; and suppose
2
1
=4 0
0

1
0

0 5;
1

where ; ; 2 (0; 1) and + = 1: In this example z1 is a transient state.

Note that if the system ever reaches z2 or z3 it will never leave. Such states are
called absorbent. Here, we have two ergodic sets, E1 = fz2 g and E2 = fz3 g :
Furthermore
3
2
3
2
n
(1
)
0
n
n
n
0
1
0 5 ! Q = 4 0 1 0 5;
=4
0 0 1
0
0
1
where

=1

(1
0

)n : Each row of Q is an invariant distribution:

2
3
1
4 0
1
0 5= 0
0
0
1

Our concern is to nd the conditions under which there exists a unique

ergodic set, and a unique invariant distribution p : Since p was dened to be
the probability distribution for which p = n p holds, it is obvious that we
need a xed point argument to establish the uniqueness of p

12.2

Invariant Distributions

Let l be the l-dimensional unit simplex. We already show that the transition
matrix maps l into itself. Hence, if we can show that l is a complete metric
space with some appropriate metric and denes a contraction mapping on l ;
then we can use the contraction mapping theorem to show that has a unique
xed point in l :
Let k k denote the norm on Rl dened by
kxk =

l
X
i=1

jxi j:

You can easily verify that Rl ; k k is a complete metric space. Since l

l
is a closed subset of Rl ,
; k k is also a complete metric space.
l
l
Let T :
!
be denes as T p = p : All we need is to nd the
conditions on so that T is a contraction mapping on l :
171

Lemma 225 Let l l be a Markov matrix, and for j = 1; 2; : : : ; l let "j =

Pl
mini ij (the minimum element in column j). If j=1 "j = " > 0; then T :
l
! l dened by T p = p is a contraction mapping with modulus 1 ":
Proof: Let q; p 2 l ; then
kT p
=

T qk = kp
l
l
X
X
j
(pi
j=1

l X
l
X
j=1 i=1

q k =
qi )(

l
l
X
X
j
(pi
j=1

"j ) +

ij

i=1

j pi
=

l
X

ij

j pi

qi j

i=1

l
X

(pi

i=1

qi j (

= (1

"j ) +

l
X
j=1

l
X

ij

qi )

ij

i=1

"j j

l
X
i=1

qi )"j j
(pi

qi ) j

"j ) + 0

j=1

")kp

qk

Note that for " to be positive we need at least one column in

with all
positive entries. What happens if this condition fails? If " is not positive then
in each column we have at least one zero entry. In other words, for every
given state there exist another state from which it is impossible to transit into
the given state. It is obvious this will lead into having several ergodic sets or
cyclically moving subsets within an ergodic set.
Theorem 226 Let Z = fz1 ; z2 ; : : : ; zl g be a nite set, and let the Markov ma(n)
(n)
trix dene transition probabilities on Z: For n = 1; 2; : : : let "j = mini ij
(n)

for j = 1; 2; : : : ; l (where ij is the elements of the matrix n ) and let "(n) =

Pl
(n)
j=1 "j : Then Z has a unique ergodic set with no cyclically moving subsets
if and only if for some N
1; "(N ) > 0: In this case fp0 n g converges to a
l
unique limit p 2
for all p0 2 l ; and convergence is at a geometric rate
independent of p0 .
Proof: Suppose "(N ) > 0: Then by previous Lemma T N : l ! l dened
as T p = p N is a contraction mapping with modulus 1 "(N ) . Since l is a
closed subset of a complete metric space, by the contraction mapping theorem
T N has a unique xed point. Let this xed point be p : Moreover
kp0

kN

p k

Now suppose fp0

converge to p ;

(1
n

"(N ) )k kp0

p k

g ! p for all p0 2
2

p1
6
p1
f ng ! 6
4
p1

p2
p2
p2
172

for k = 1; 2; : : : and for all p0 2

l

: Then every row of the

3
: : : pl
: : : pl 7
7;
5
:::
: : : pl

must

where pi is the unique probability of being in state i in the long run. Then for
(N )
some N su ciently large there is at least one column j for which ij > 0
(otherwise p will not dene a probability distribution over Z), then "(N )
(N )
"j > 0

173

13

Recursive Competitive Equilibrium

V (x0 ) =

sup

1
X

fxt+1 g1
t=0 t=0

F (xt ; xt+1 );

then this function should satisfy the following functional equation (FE);
V (x) = sup [F (x; y) + V (y)] ;

(FE)

subject to
y 2 (x):
The supremum function V (x0 ) tells us the innite discounted value of fol1
lowing the best sequence fxt+1 gt=0 : Our strategy was that rather than nding
1
the best sequence xt+1 t=0 ; we can try to nd the function V (x0 ) as a solution to the FE and use the associated policy rule y = g(x) to analyze the
optimal sequence. We then showed that: 1) The supremum function V has to
satisfy FE, and if there is a solution to FE, then it is the supremum function.
2) There is indeed a solution V to FE. We also analyzed: 1) Properties of V: 2)
Dynamic behavior implied by the policy function, y = g(x):
Our analysis so far was based on a planners (or a representative agents)
problem. Now we will talk about market economies. We will imagine a large
number of representative agents interacting in a market economy and try to
understand what conditions have to be satised for this market economy to be
in equilibrium. Once we move to a market economy we have to be clear about:
Ownership structure (who own capital, hence who decides on capital accumulation).
Which markets are open (goods, capital, labor).
Economic agents (households and rms).
In general a market equilibrium will be a situation such that for some given
prices, individuals and rms decisions are such that markets clear. There is more
than one way to think about the markets. We have already seen Arrow-Debrue
economies with time zero trade (i.e. all agents participating in a big market at
time 0); and economies with sequential markets where assets are traded every
period. We will now introduce a third equilibrium concept called the recursive
competitive equilibrium that is most suitable for analysis of dynamic economies.
Recursive competitive equilibrium is based on the idea that dynamic programming problems can be split into decisions about today and the entire future.
As you remember the key in our dynamic programming problem was the idea
of the state, i.e. the variables that provide all the information we need to make
decisions. In a recursive competitive equilibrium the prices are denes as functions of the state. Hence, in a recursive competitive equilibrium both individual
decisions (characterized by a value function and a decision rule) and the prices
will be functions of the state.
174

We will now dene the recursive competitive equilibrium for our one-sector
growth model. We will imagine an economy that is populated by a large number
of identical agents (households). The households own both capital and labor.
They rent their capital and labor to a single rm that produces output by a
constant returns to scale technology and pays the rental rate and the wage rate
to the households. Household decide how much of their total resources (which
consist of undepreciated capital, rental income and wage income) to consume
and how much of it to save as future capital.
We will denote an individual capital holdings by k; and the aggregate stock
of capital by K: Suppose each household has one unit of time and start with k0
units of capital. Let rt and wt be period t rental rate and the wage rate (we
do not know yet how they are determined). The households want to maximize
their lifetime utility by choosing the optimal consumption/saving path given a
set of prices. Hence, the agents problem is
X
t
max
u(ct );
((HHP))
ct ;kt+1

s.t.

ct + kt+1 = wt + (1 + rt

)kt = wt + (1

)kt + rt kt ;

and
k0 > 0 given.
Note that since agents do not value leisure they will supply all of their time to
the rm.
The rm faces a simple prot maximization problem each period given by
max (F (Kt ; Nt )

Kt ;Nt

rt K t

wt Nt ) for all t;

(FP)

The rst order conditions associated with the rms maximization problem are
rt = FK (Kt ; Nt );
and
wt = FN (Kt ; Nt );
where Kt and Nt are aggregate capital stock and labor demanded by the rm.
Note that in equilibrium it must be the case that Nt = 1: We also know
that in equilibrium, the households will supply all of their capital stock to the
rm, i.e. Kt = Kt , hence
rt = FK (Kt ; 1);
and
wt = FN (Kt ; 1):
These FOCs dene for every value of aggregate capital stock a rental rate and
a wage rate. We will therefore dene a rental rate function r : K ! R+ ; and a
wage function w : K ! R+ as
r = r(K) and w = w(K):
175

Therefore, in each period if each household knows the aggregate capital stock
K; then they know exactly what the current rental rate and the wage rate are.
Hence, an household needs to know both k and K to be able to solve its dynamic
programming problem. Furthermore, each household also needs to know how
K evolves over time. The aggregate capital stock, however, evolves over time as
a result of the decisions of all of the households. This implies that we will need
a consistency condition.
Lets rst look at the household problem. Let V (k; K) be the value function
for a household with k units of capital if the aggregate capital stock is K: This
value function is dened as
V (k; K) = max
[u(c) + V (k 0 ; K 0 )] ;
0
c;k

(HHP)

subject to
c + k 0 = r(K)k + (1

)k + w(K);

and
K 0 = G(K):
Note that the solution to this problem will imply a law of motion for individual
capital given by
k 0 = g(k; K) = arg max(HHP )
Here, G(K) is the law of motion for the aggregate capital stock. The household
need to know G in order to be able to predict K 0 : Of course in equilibrium G is
not an arbitrary object.
Now we can dene a recursive competitive equilibrium (RCE): A RCE is a
set of functions for quantities G(K) and g(k; K); for utility level V (k; K); and
for prices r(K) and w(K) such that:
V (k; K) solves (HHP) and g(k; K) is the associated policy function.
Prices are competitive, i.e.
r(K) = FK (K; 1);
and
w(K) = FN (K; 1):
Individual and aggregate decisions are consistent, i.e.
G(K) = g(K; K) for all K:
Note the following:
1. The third condition is the key feature of a RCE. It requires that whenever the individual consumer is endowed with aggregate capital stock, his
individual behavior is exactly same as the aggregate behavior.

176

2. We did not mention the price of the aggregate output. Other prices are
in terms of output.
3. We did not mention a market clearing condition such as
C + K 0 = F (K; 1) + (1

)K:

This will hold by CRS assumption, since

F (K; 1) + (1

)K = r(K)K + (1

)K + w(K):

We will now look at a slightly dierent example. Consider now an economy

populated by innitely many identical households that live forever. Each household is endowed with 1 unit of time to allocate between leisure l; and work h:
Hence in each period
lt + ht = 1:
In addition households own an initial stock of capital k0 ; which they rent to
rms and augment through investment. Households utility is given by
(1
)
X
t
U (c( ); h( )) = E
U (ct ; 1 ht ) ; with 2 (0; 1);
t=0

where c( ) and h( ) are innite sequences of consumption and leisure, U is continuously dierentiable in both arguments, U1 > 0; U2 > 0, and U is strictly
concave.
Let K and H denote the aggregate capital stock and labor supply. Firms
2
! R
has access to the following CRS production technology F (Kt ; Ht ) : R+
with F1 > 0; F2 > 0; F (0; 0) = 0; and F is concave in K and H separately.
Moreover,
Yt = ezt F (Kt ; Ht ) with zt+1 = zt + "t+1 ;

2 (0; 1); and " s N (0;

" ):

and the aggregate capital stock evolves according to

Kt+1 = (1

)Kt + Xt ;

where 2 (0; 1) is the depreciation rate and Xt is the aggregate investment.

The aggregate resource constraint is then
Yt = Ct + Xt :
We could also write the aggregate resource constraint as
Kt+1 + Ct = (1

)Kt + Yt :

The rms problem is given by following static maximization problem (note

that due to CRS there is only one rm)
max (ezt F (Kt ; Ht )

Kt ; Ht

rt K t
177

wt Ht ) for all t;

where rt is the rental cost of capital and wt is the wage rate. The rst order
conditions for this problem is given by
rt = ezt FK (Kt ; Ht ); and wt = ezt FH (Kt ; Ht )
The representative households problem in this economy is given by the following
Bellman equation
V (z; k; K)

max [U (c; 1

c; x; h

h) + E(V (z 0 ; k 0 ; K 0 ) j z)]

s:t: c + x
r(z; K)k + w(z; K)h
0
k = (1
)k + x
0
K = (1
)K + X(z; K)
0
z =
z + "; c 0; 0 h

1;

where c; k; x are individual consumption, capital stock and investment and

K and X are aggregate capital stock and investment. Note that r(z; K) and
w(z; K) indicate the fact that these prices depend on the aggregate capital
stock. In this problem the state variable for a representative household is given
by st = (zt ; kt; Kt ) while the aggregate state is given by St = (zt ; Kt ):
Note that we could also write the representative households problem as
max [U (c; 1

h) + E(V (z 0 ; k 0 ; K 0 ) j z)]

V (z; k; K)

s:t: c + k 0
K0
z0

r(z; K)k + w(z; K)h + (1

= G(z; K)
=
z + "; c 0; 0 h 1;

c; k0 ; h

)k

where c and k are individual consumption and capital stock, K is the aggregate
capital stock and investment. Note that r(z; K) and w(z; K) indicate the fact
that these prices depend on the aggregate capital stock.
REMARK: In the rst formulation the households is given pricing functions w and r as well as an aggregate investment function X: This allows consumer to gure out current income as well as future aggregate capital stock
K 0 : In the second formulation, the households is given a function that maps
(z; K) to K 0 directly.
Then a RCE for this economy is a value function V (z; k; K); a set of decisions rules c(z; k; K); h(z; k; K); and x(z; k; K); corresponding aggregate decision rules C(z; K); H(z; K); and X(z; K); and factor prices w(z; K) and r(z; K)
such that:
V (z; k; K) solves the household problem with associate solutions given by
c(z; k; K); h(z; k; K); and x(z; k; K):
Firms FOCs are satises, i.e.
r(z; K) = ez FK (K; H(z; K)); and w(z; K) = ez FH (K; H(z; K)):
178

Individual and aggregate decisions are consistent, i.e.

c(z; K; K) = C(z; K);
h(z; K; K) = H(z; K);
x(z; K; K) = X(z; K):
Aggregate resource constraint has to be satised
C(z; K) + X(z; K) = Y (z; K) for all z and K:
Alternatively, a RCE for this economy is a value function V (z; k; K); a set of
decisions rules g(z; k; K) and h(z; k; K); corresponding aggregate decision rules
G(z; K); and H(z; K); and factor prices w(z; K) and r(z; K) such that:
V (z; k; K) solves the household problem with associate solutions given by
g(z; k; K) and h(z; k; K):
Firms FOCs are satises, i.e.
r(z; K) = ez FK (K; H(z; K)); and w(z; K) = ez FH (K; H(z; K)):
Individual and aggregate decisions are consistent, i.e.
g(z; K; K) = G(z; K);
h(z; K; K) = H(z; K);
Aggregate resource constraint has to be satised
C(z; K) + G(z; K) = Y (z; K) + (1

179

)K for all z and K:

14
14.1

Introduction to Numerical Methods

Deterministic Case

Consider the following version of the neoclassical growth model:

max

1
X

U (ct )

(P1)

t=0

subject to
ct + kt+1 = f (kt ) and k0 > 0 given.
The DP problem associated with P1 is given by
V (kt ) =

max

kt+1 2[0;f (kt )]

fU (f (kt )

kt+1 ) + V (kt+1 )g :

(P2)

First order and envelope conditions for this problem are given by
U 0 (f (kt )

kt+1 ) = V 0 (kt+1 );

(FOC)

and
V 0 (kt ) = U 0 (f (kt )

kt+1 )f 0 (kt ):

(Envelope)

Suppose the utility function and the production function take the following
parametric forms:
c1
;
U (c) =
1
and
f (k) = k a :
Then, the steady state level of capital stock, k = kt = kt+1 ; is given by the
solution to the following Euler equation
U 0 (k a

k) = U 0 (k a

k)ak a

which gives us
k = (a )1=1

Hence, given a set of parameters we can nd the steady state value of capital
stock.
Remark 227 We know that given any value of k0 > 0, this economy converges
monotonically to k : Hence, when we choose a grid for k below, we should make
sure that k is within that grid.
In order to be able to solve P2 numerically, rst assume that the capital
stock can only take values in a discrete set given by
kt 2 K = fk1 ; k2 ; : : : ; kN g ; 8t:
180

This will make maximization over next periods capital stock k 0 much more
easier since we only need to search over a nite number of possibilities.
Given this discrete set, we can dene the following iteration on K
V n (ki ) = max
0

k 2K
k0 f (kt )

U (f (ki )

k0 ) + V n

(k 0 ) ; i = 1; : : : ; N:

(P3)

Let the optimal value of k 0 that solves P3 be denoted by

g n (ki ) = arg max
0

U (f (ki )

k0 ) + V n

(k 0 ) ; i = 1; : : : ; N:

(P4)

k 2K
k0 f (ki )

Start with some initial guess for the value function V (k); denoted by V 0 (k);
on this set K: Note that for a discrete set a function is simply a list of numbers
corresponding to each element in that set. Hence, we can take
3
2 3
2 0
0
V (k1 )
0
6
7
6
0 7
V
(k
)
2
7
7
=6
V 0 (ki ) = 6
5
4 5:
4
0
V 0 (kN ) N 1
Then, for i = 1; : : : ; N

V 1 (ki )

max fU (f (ki )

k 0 ) + 0g ;

k 2K
k0 f (ki )

max fU (f (ki )

k1 ) ; U (f (ki )

This way we have a maximum value for each

as our new guess for V
2 1
V (k1 )
6 V 1 (k2 )
1
V (ki ) = 6
4
V 1 (kN )

k2 ) ; : : : ; U (f (ki )

kN )g :

ki and we can store these values

3
7
7
5

N 1

we can also store the maximizing values for k 0 (assuming that it is unique)
as our policy function
2
g 1 (k1 ) = arg max fU (f (k1 ) k1 ) ; U (f (k1 ) k2 ) ; : : : ; U (f (k1 ) kN )g
6 g 1 (k2 ) = arg max fU (f (k2 ) k1 ) ; U (f (k2 ) k2 ) ; : : : ; U (f (k2 ) kN )g
g 1 (ki ) = 6
4
g 1 (kN ) = arg max fU (f (kN ) k1 ) ; U (f (kN ) k2 ) ; : : : ; U (f (kN ) kN )g
Now, we can proceed to the next iteration and get for i = 1; : : : ; N;
8
9
>
>
=
<
2
0
1 0
V (ki ) =
max
U (f (ki ) k ) + V (k ) ;
| {z }>
k0 2K >
:
;
0
k

6= 0

f (ki )

maxfU (f (ki ) k1 ) + V (k1 ); : : :

; U (f (ki ) kN ) + V 1 (kN )g;
181

3
7
7
5

N 1

where the values for V 1 (ki ) comes from the matrix that we stored in the previous
iteration.
Let
m=

norm(V n+1 V n )
;
norm(V n )

and continue iterating until m < "; where " is a small number.
If our problem satises some nice properties, then we know that these iterations will converge to the unique value function of this problem. In the nal
iteration, we can save the value function and policy function in order to analyze
how this economy behaves:
2
3
2
3
V (k1 )
g(k1 )
6 V (k2 ) 7
6 g(k2 ) 7
7
7
; g(ki ) = 6
V (ki ) = 6
4
5
4
5
V (kN ) N 1
g(kN ) N 1
Remark 228 Matlab program growmodel.m implements a discrete state space
solution for nonstochastic one sector growth model.
Remark 229 Note that we can easily add endogenous labor supply decision to
one sector growth model. If we do that, we will have a static maximization
problem for labor supply. Indeed, before going into value function iteration we
can nd the optimal labor supply decisions for all combinations of k and k 0 .
Then, we can use these labor supply values whenever we need them. Matlab
program growmodel2.m solves nonstochastic version of one sector growth model
with endogenous labor supply decision. The utility function is assumed to have
the following form
(1
) log(c) + log(1 n):
Note that the program starts with nding the optimal labor supply decisions and
utility values for all feasible combinations of (k; k 0 ); and then enters the value
function iteration stage. It uses fsolve.m, a built-in Matlab function tha1t nd
the zero of a function of one variable. The function solvelab.m calculates, for
any given value of n; the value of the FOC for n.
Remark 230 Note that we use the set K = fk1 ; k2 ; : : : ; kN g in above algorithm
in two places: First, we dened value functions on K: Second, we nd the
maximizer in the following problem from the set K
g n (ki ) = arg max

k0 2K
k0 f (ki )

U (f (ki )

k0 ) + V n

(k 0 ) ; i = 1; : : : ; N:

Indeed, we can solve this maximization problem better. Suppose we want to

nd the maximizer to this problem not on the grid (i.e. we want to ignore the
constraint k 0 2 K), but among any feasible values (i.e. we want to care only
about the constraint k 0
f (ki )): How can we do this? Note that there are

182

two parts to this maximization problem: First, U (f (ki ) k 0 ) part, which is

continuous and can be evaluated for any value of k 0 : Second, V n 1 (k 0 ) part,
which is only dened on K: However, we can use linear (or other forms of )
interpolation to nd the value of V n 1 for any k; given the value of V n 1 on
K: The basic idea of linear interpolation is shown in Figure 1. Suppose we want
to evaluate
U (f (ki ) k 0 ) + V n 1 (k 0 )
for a value of k between ki and ki+1 : Then, given V n
V n 1 (k) is given by
Vn

(k) = V n

(ki ) +

Vn

Vn
ki

(ki+1 )
ki+1

(ki ) and V n

(ki )

(k

(ki+1 );

ki ):

Matlab code growmodel3.m implements this procedure. The function value.m

nds U (f (ki ) k 0 ) + V n 1 (k 0 ) for any value of k 0 : We then use a built-in
maximizer fminbnd.m to nd the maximum of this function over k 0 : What
is the advantage of doing this? When we use a small number of grid points,
maximization over K will not provide an accurate solution of these maximization
problems. The interpolation methods works ne even with small number of grid
points. Figures at the end show results from growmodel.m and growmodel3.m
with 21 grid points. As you can see, decision rule growmodel.m is not very
good, whereas growmodel3.m has no problem generating a nice function for k 0 :

14.2

Stochastic Case

Consider now the stochastic version of the neoclassical growth model:

max

1
X

U (ct )

t=0

subject to
ct + kt+1 = exp(zt )f (kt );
and
ln zt = ln zt

ut
|{z}

iidN (

u; u)

Then the dynamic programming problem is given by

V (kt ; zt ) =

max

kt+1 2[0;zt f (kt )]

fU (exp(zt )f (kt )

kt+1 ) + E [V (kt+1 ; zt+1 )]g :

(P5)
The state for this economy is now given by st = (kt ; zt ):
Hence, in order to be able to apply discreet state space methods, we also
need to dene a grid for zt : Suppose zt can only take values in a nite set given
by
z 2 Z = fz1 ; z2 ; : : : ; zM g :
183

V(ki+1)
V(k)

V(ki+2)

V(ki+1)- V(ki)

V(ki)

ki

ki+1

ki+2

k - ki

ki+1 - ki

Figure 21: Linear Interpolation

184

Then, one can represent the autocorrelation for zt using a transition matrix
2
3
:::
11
12
1M
6 21
7
6
7
;
=6 .
7
.
4 .
5
M1

MM

where

ij

= Pr [zt+1 = zj j zt = zi ] ;
0 8i; j;

ij

and

M
X

ij

= 1;

8i; 9j s.t.

> 0:

ij

Now we can rewrite our iterations on the set K

j = 1; : : : ; M;
V n (ki ; zj )

max

k0 2K
exp(zj )f (ki )

M
X

jr

Z as, for i = 1; : : : ; N;

fU (exp(zj )f (ki )

Vn

r=1

k0 )

(P6)

(k 0 ; zr ) g;

(91)

and
g n (ki ; zj )

arg

max

k0 2K
exp(zj )f (ki )

M
X

jr

Vn

r=1

[U (exp(zj )f (ki )

k0 )

(k 0 ; zr ) g:

(92)

Since now V is a function of both k and z; in a discrete state

be given by a N M matrix. Again let
2 0
V (k1 ; z1 ) V 0 (k1 ; z2 ) : : : V 0 (k1 ; zM )
6 V 0 (k2 ; z1 )
6
V 0 (ki ; zj ) = 6 .
4 ..
2

Then,

6
6
= 6
4

V 0 (kN ; z1 )
0
0
..
.

0 ::: 0

V 1 (ki ; zj ) =

max

k0

k0 2K
exp(zj )f (ki )

(P7)

V 0 (kN ; zM )

space it will
3
7
7
7
5

7
7
7:
5
fU (exp(zj )f (ki )

185

k 0 ) + 0g =

N M

2
6
6
6
6
6
6
6
4

max

k0 2K
exp(z1 )f (k1 )

fU (exp(z1 )f (k1 )

k0 2K
k0 exp(z1 )f (k2 )

fU (exp(z1 )f (k2 )

k0

max

k 0 )g max

k0 2K
exp(z2 )f (k1 )

fU (exp(z2 )f (k1 )

k0 2K
k0 exp(z2 )f (k2 )

fU (exp(z2 )f (k2 )

k0

k 0 )g max

..
.
V 1 (kN ; z1 )

V 1 (k1 ; zM )

k 0 )g

..
.

V 1 (kN ; zM )

Once we have stored V 1 (ki ; zj ) and g 1 (ki ; zj ); we can move to the next
iteration and compute:
(
)
M
X
1 0
V 2 (ki ; zj ) =
max
U (exp(zj )f (ki ) k 0 ) +
;
jr V (k ; zr )
0
k 2K
k0 exp(zj )f (ki )

r=1

where jr are given exogenously and V 1 (ki ; zr ) are the values that we stored
in the previous iteration. We can keep repeating this procedure until we have
convergence.
14.2.1

k 0 )g : : :

Tauchens Method

In the previous section we represented the autocorrelated stochastic process

zt = z t

ut
|{z}

iidN (

u; u)

with a transition matrix : If the only information we had was the information
on ; u ; and u ; could we come up with the entries on ? One way to achieve
this is to use Tauchens method
1. Determine a grid for Z: Let z1 = z q z and zq = z + q z ; where q is
some integer value, and z and z are the unconditional mean and standard deviations for z: Then simply pick M equally spaced points between
z1 and zq to form
X = fz1 ; : : : ; zM g :
2. Let the distance between each points be represented by w = zk zk 1 :
Then for each i; if j 2 f2; : : : ; M 1g ; the transition probabilities are
given by
pij

Pr[zj

= F

zj

w
zi + u
2
zi + w2

zj +
zj

w
2

where
Pr[ut

w
]
2
zi

a] = F (

is the cumulative density function for a standard normal distribution.

186

7
7
7
7
7
7
7
5

z2
z2 w/2

z2 + w/2
w

Prob(z = z2) = F(z2 + w/2 ) - F(z2 - w/2 )

Figure 22: Tauchens Method

3. Finally
z1

pi1 = F

zi +

w
2

and
piM = 1

zM

zi +

w
2

14.2.2

Simulations

So far we have looked at the numerical solutions of the nonstochastic and stochastic versions of the one sector growth model. We showed how we can use
discrete state space methods to nd V and g: Once we nd value and policy
functions, we would like to know how these articial economies behave.
In the nonstochastic case, since the capital stock is the only state variable,
g(k) gives us all the information we need. Given an initial capital stock k0 ;
we can analyze, using g(k), how this economy evolves, i.e. how the optimal
sequence fkt g1
t=0 behaves. We know that under standard assumptions on utility

187

and production functions, g(k) is strictly increasing, and has a unique positive
stationary point dened by
k = (f 0 )

(1= ):

(93)

If we have the following parametric forms:

U (c) =

c1
1

and
f (k) = k a + (1

)k;

then,
k =

1
a

(1

1=

Indeed, growmodel.m solves this particular model with = 1; = 0:75; and

= 0:3: Then, k = 11.0766. This program generates the policy function, g(k);
shown in Figure 23, and the transition path (for 100 periods) shown in Figure
24.
12.5

12

next period capital

11.5

11

10.5

10

9.5
9.5

10

10.5

11
current capital

11.5

12

12.5

Figure 23: Policy Function from growmodel.m

Consider now the stochastic version of the one-sector growth model:

"1
#
X
t
max E0
u(ct )
t=0

188

11.4

11.2

11

capital stock

10.8

10.6

10.4

10.2

10

9.8

10

20

30

40

50
time period

60

70

80

90

100

Figure 24: Transition path from growmodel.m

subject to
ct + kt+1 = exp(zt )F (kt ):
Matlab code bc.m solves a particular version of this problem in which
u(c) =

c1
1

; and F (k) = k ;

and zt follows
z t = zt

+ u;

where u is iid normal with zero mean and standard deviation

parameter values are

u:

In bc.m, the

0.95

0.3

0.81

0.02

Figure 25 shows the decision rule for kt+1 that bc.m generates. Note that each
line is a decision rule for a given value of z:
Given these decision rules, we would like to know how this economy behave.
In this case, we have to analyze how the capital stock and the exogenous shocks
behave jointly.
In order do this, let pij;rs be the probability that the current capital stock
is ki and the current shock is zj and the economy moves to a state where the
capital stock is kr and the shock is zs : That is,
pij;rs = Pr [k 0 = kr ; z 0 = zs j k = ki ; z = zj ] :
189

optimal investment decision

0.2

0.19

0.18

0.17

0.16

0.15

0.14

0.13
0.13

0.14

0.15

0.16

0.17

0.18

0.19

0.2

Figure 25: Policy Function from bc.m

We know that the probability of moving from zj to zs is given by

pij;rs = Pr [k 0 = kr j k = ki ; z = zj ]

js :

Hence,

js :

What is the probability that we end up in kr next period, given that the current
state is s = (ki ; zj )? We know the policy function g(k; z): Hence, given s =
(ki ; zj ); we know exactly what is the choice for the optimal capital stock. Then,
pij;rs =

js ; if g(ki ; zj ) = kr
:
0; otherwise

Consider the following ordering of the all possible states for this economy:
S = f(k1 ; z1 ); (k2 ; z1 ); : : : ; (k1 ; zM ); (k1 ; z2 ); : : : ; (kN ; zM )g ;
and construct the following transition matrix on S
2
p11;11
p11;21 : : : p11;N M
6 p21;11
6
P =6 .
4 ..
pN M;11

Suppose now p0 is an initial N

pN M;N M

3
7
7
7
5

NM

;
NM

M vector of probability distribution on S:

190

That is

6
6
p0 = 6
4

with
p0ij

p011
p021
..
.
p0N M

0; 8i; j; and

3
7
7
7
5

NM

N X
M
X

pij = 1:

i=1 j=1

Then, we can iterate on p0 using the transition matrix P to get the next
periods distribution over S
p1 = P p0 :
If we keep iterating on
pn = P pn

we can hope that we nd the stationary distribution over S:

p = Pp :
For the stochastic model, p characterizes the stationary distribution over
k: Note that p is N M dimensional. If we integrate it over productivity
shocks, we will get a distribution over k: This distribution is equivalent of k
in the nonstochastic version, i.e. there isnt a single level of k that economy
converges, but there is probably distribution over k. Figure 26 shows the long
run distribution of capital stock from bc.m.
Grid for k In nonstochastic version of this problem we rst nd k ; and since
we know that economy will converge to k ; we constructed a grid around k :
In stochastic version of this problem, we can use p to nd the appropriate
grid for capital. Suppose you start from some grid for k; and plot the long run
distribution of k (like in Figure 26). If your grid is not wide enough, then you
observe accumulation of probabilities at the end points of your grid. Figure 27
shows results from bc.m with a narrow grid for k. If this occurs, you can expand
your grid until you capture the ergodic set for k:
In this particular example, there is another way to construct the grid for k,
which is used in bc.m Note that, if = 0, the value function for this problem
is
V (kt ; zt ) = maxf(exp(zt )kt
kt+1 )
Et [V (kt+1 ; zt+1 )]g:
The solution can be characterized by the following Euler equation,
1

Et [(exp(zt+1 )kt+11 ]

Et [(exp( zt + u)kt+11 ]:

191

long run distribution of capital

0.04

0.035

0.03

0.025

0.02

0.015

0.01

0.005

0
0.13

0.14

0.15

0.16

0.17

0.18

0.19

0.2

Figure 26: Long Run Distribution of k from bc.m

long run distribution of capital

0.035

0.03

0.025

0.02

0.015

0.01

0.005

0
0.15

0.155

0.16

0.165

0.17

0.175

0.18

0.185

Figure 27: Long run disrtibution of capital with narrow grid for k

192

Since only random variable here is u; the above equation can be rearranged as
1

Et [exp( zt )(exp u)kt+11 ]

exp( zt )kt+11 Et [exp(u)]:

Taking logarithm of both sides, we get

0 = ln

+ zt + (

1) ln kt+1 + ln Et [exp(u)]:

Now notice that moment generating function for normal distribution is

E(etx ) = M (t) = exp(xt +

2 2

t =2):

Then,
Et [exp(u)] = M (1) = exp(

2
u =2);

and we can obtain

0 = ln

+ zt + (

1) ln kt+1 + ln[exp(

2
u =2)]:

Therefore,
ln kt+1 =

1
(1

[ln

+ zt +

2
u

]:

Now it immediately follows that,

E(ln kt+1 ) =

1
(1

[ln

2
u

];

and
V ar(ln kt+1 )

=
=
=

1
)2

(1
1

V ar( zt )

)2

(1
1
(1

2
u

)2 (1

2 2
u
:
2)

So, we can construct a grid for k around E(ln kt+1 ): Although

this approach still provides us with a nice grid for k:

> 0 in bc.m;

Generating Time Series from the Model Once we have a well-dened

grid for k; we can use the optimal decision rule, g(k; z); to generate time series
from the model that we can compare with the data. Suppose we want to create
time series for T periods. We can do this as follows:
1. First, we need to generate a time series for z: In order to do that:
Pick a starting value for z; suppose z1 = zj :
193

Determine z2 using [ j1 ; j2 , j3 ; :::; jm ] : Note that this is a probability distribution over Z: To determine the value for z2 ; we can use
a random number generator. For example Matlab command rand
gives a uniform random number between 0 and 1. Suppose we draw
such a random number. Call it u: Now if u is less than j1 ; then we
will set z2 = z1 : If it is larger than j1 ; but less than j1 + j2 ; we
will set z2 = z2 (note that subscript on the left hand side refers to
time and the one on the right hand side refers to the index from the
set Z); and so on. This way we can determine z2 : Suppose, given
u;we have z2 = z5 ; then we will use [ 51 ; 52 , 53 ; :::; 5m ] ; and a new
random uniform number and determine z3 ; etc.
Often you generate a series longer than T; and discard the rst set
of simulations to avoid the eect of initial z.
2. Once you have T periods z; you can start from some value of k (for example, you can set k1 to the expected value of k using p ); and, given k1 and
z1 ; get k2 = g(k1 ; z1 ): Then, you can get k3 = g(k2 ; z2 ); etc. This way we
can generate time series for yt ; ct ; kt that we can compare with the data.
3. This way we can generate all the variables that we care about. Usually
this procedure is repeated several times to get multiple simulations for
each variable we care about.
After nding V; g; and p ; bc.m generates simulated data. It rst generates
100 period long productivity shocks (in total 20 such series). Figure 28 shows
examples of such time series for z:
Once we have a time series for z; we can use it to generate time series for
100
other variables we care about using g(k; z): In order to do this, let fzt gt=0 be one
of the time series bc.m generates. Furthermore let k0 be the mean of capital
stock in Figure 26. Then, y0 = exp(z0 )k0 ; k1 = g(k0 ; z0 ); y1 = exp(z1 )k1 ;
k2 = g(k1 ; z1 ); ::::: This way we can construct a series for yt : Figure 29 shows
one such series for yt that bc.m generates.
The next question is how we can compare time series that our articial
economy generates with the U.S. data.
Hodrick-Prescott Filter In order to be able to compare our simulated data
with the U.S. data, we will rst look at the U.S. data. The following gure
shows the U.S. real quarterly GDP between 1947 and 2003. At a rst glance it
does not look anything like the picture that comes out of our model. But our
model did not have growth, so we have to remove the growth component from
the data. Furthermore, we are mainly interested in business cycle uctuations,
i.e. uctuations that occur with a frequency of 3 to 5 years.
The common procedure in the real business cycle literature is to use HodrickT
Prescott (HP) lter that works as follows. Consider a series fYt gt=1 : Suppose
Yt consists of two components: a trend component and a cyclical component:
Yt =

+ Ytd :

194

first seri

second seri

0.1

0.1

0.05

0.05

-0.05

-0.05

-0.1

50

100

-0.1

150

50

tenth seri

100

150

100

150

twientieth seri

0.1

0.15
0.1

0.05

0.05
0
0
-0.05

-0.1

-0.05

50

100

-0.1

150

50

Figure 28: Simulated z series from bc.m

output -- one simulation

0.68

0.66

0.64

0.62

0.6

0.58

0.56

0.54

0.52

0.5

20

40

60

80

Figure 29: Simulated yt from bc.m

195

100

120

DEVIATIONS FROM TREND for Y

0.08

0.06

0.04

0.02

-0.02

-0.04

-0.06

20

40

60

80

100

120

Figure 30: Deviations from Trend, Simulated Data

HP lter pick

min
f

from the following minimization problem.

T
t gt=1

T
X
t=1

(Yt

2
t)

T
X1

[(

t+1

t)

2
t 1 )]

t=2

In this problem there is a trade o between the extent to which the trend
component tracks the actual series against the smoothness of the trend. If
= 0; then Yt = t ; Ytd = 0; and if ! 1; then t approaches a linear trends.
For quarterly data, it is customary to set = 1600:This way HP lter eliminates
uctuations at frequencies lower than 32 quarters (8 years).
The following pictures show the actual data together with the trend component as well as deviations from the trend.
Suppose we apply the same procedure to our simulated data. Figure 30
shows the cyclical component in the model. The basic comparison is between
the cyclical component in the data and the cyclical component in the model.
14.2.3

Calibration Prescott (1986)

In order to simulate the model, we need to choose functional forms for u and f
and specify parameter values. We also have to specify the stochastic structure
for z: How can we do this? Prescott (1986) uses long run (secular) growth
observations to choose functional forms and parameters:
Observation 1: In the U.S. economy, the capital and labor shares of output
has been relatively constant, while their relative prices change over time. This

196

suggests a Cobb-Douglas production function

zf (k; n) = zk 1

n ;

with labor share parameter : Furthermore, the average value of labor share
in total output in the U.S. has been about 64%. Hence, we set
= 0:64.
Observation 2: In the U.S. economy, real wages have increased over time,
yet per capita market hours have been relatively constant. This suggests a unit
elasticity of substitution between consumption an leisure. Hence, the following
functional form is appropriate
u(c; 1

(c1

n) =

n) )1

(1

where 1 is the intertemporal elasticity of substitution.

There exists a range of estimates for in the micro studies. Prescott (1986)
picks = 1; which imply.
u(c; 1

n) = (1

) log(c) + log(1

n):

Observation 3: Given the Cobb-Douglas production function, we have

log(zt+1 )

log(zt )

(log(Yt+1 ) log(Yt )) (1
(log(Nt+1 ) log(Nt ));

)(log(Kt+1 )

log(Kt ))

where Yt ; Kt and Nt are aggregate output, capital and labor. Setting = 0:64;
one can construct a time series for zt and analyze its statistical properties.
Remaining Parameters: we still need to determine ; ; and : In order to
choose these variables, consider again the DP given by
V (kt ; zt ) = max f(1
kt+1 ;nt

) log(c) + log(1

n) + Et V (kt+1 ; zt )g ;

subject to
zkt1

ct + kt+1

nt + (1

)kt

kt+1 ;

zt+1 = zt + "t :
Then, the FOC for kt+1 is given by
(1

1
= Et V1 (kt+1 ; zt+1 );
ct

the Envelope condition is given by

V1 (kt ; zt+1 ) = (1

1
ct+1

(zt (1

197

)kt nt + 1

);

(P1)

while the FOC for nt is

(1

1
zt kt1
ct

nt

nt

Finally, we can write the following Euler equation that determines the accumulation of capital
(1

1
= Et (1
ct

1
(zt+1 (1
ct+1

)kt+1 nt+1 + 1

) :

Consider now the nonstochastic version of the Euler equation:

ct+1
= ((1
ct

)kt+1 nt+1 + 1

ct+1
= ((1
ct

) =)

1
kt+1
nt+1
+1
kt+1

):

Since in the steady state, ct+1 = ct = c and kt+1 = kt = k; this equation

becomes
y
1 = ((1
) +1
)
k
or
y
1
+
1 = (1
) :
(C1)
k
Which can also be written as
1
= 1 + MPK
(C1)
| {z }
return on capital

Next consider the FOC for labor, which is given by

(1

1 1
k
ct t

nt

1
1

nt

and can be written as

(1

1 kt1 nt
1
=
;
ct
nt
1 nt

or in steady state
y1 n
=
c n
(1

(C2)

In the U.S. economy, rate of return on capital is about 4%. This suggests
1

= 1 + 0:04 =)

Given = 0:64; = 0:96; and

economy); and using
1
+
we have
= 0:36

k
y

= 0:96:

= 2:6 (which is the ratio in the U.S.

1 = (1

1
+1
2:6
198

y
) ;
k

1
= 0:097
0:96

Finally, given = 0:64; n = 1=3; and yc = 1:3 (again the ratio for the U.S.
economy), and using
y1 n
=
;
c n
(1
)
we have
(1

= 0:64(1:3)

2=3
= 0:936 = 1:664;
1=3

which implies
=

1:664
= 0:625:
2:664

Remark 231 See Cooley and Prescott (1995) for a much more detailed discussion of the basic business cycle regularities and the calibration procedure.
The standard RBC model relies on a single productivity shock to generate
business cycles. The success of the model is judged by its performance in
generating unconditional second moments of de-trended data. Table 1 and 2
(taken from Hansen and Wright (1992)) shows the basic set of observations that
a standard RBC model tries to replicate. These observations are: a) relative
volatility of dierent variables (output, consumption, investment, productivity,
and labor input); b) contemporaneous correlations between variables (correlations of consumption, investment and labor input with output as well as the
correlation between productivity and labor input).
Table 3 (again taken from Hansen and Wright (1992)) shows the performance
of the standard RBC model. A comparison between Tables 1 and 2 and Table 3
reveals that: a) The standard model is not able to generate the level of volatility
we observed in the data. b) It does relatively well with respect to relative
volatilities; c) It does a poor job of generating the low correlation between
hours and productivity (which is 0.93 in the standard model).
Remark 232 Note that the set-up that Hansen and Wright (1992) consider is
exactly the one we outlined above.
Chart 1 and 2 (from Hansen and Wright (1992)) shows the relation between
productivity and hours in the US data and in the standard model. The standard
model generates a positive relation between hours and productivity. In the
standard model, productivity shocks operate as labor demand shocks along a
stable labor supply curve. Furthermore, with the current calibration, the labor
supply is not very elastic, so the relation in Chart 2 is quite steep. The low
elasticity is responsible for the low volatility of output in Table 3. How can we
improve upon this? There are two obvious candidates:
increase the elasticity of labor supply.
introduce labor demand shocks.
Hansen and Wright (1992) discusses dierent ways to achieve these goals.
199

14.2.4

Linear Decision Rules

In the previous sections we analyzed how we can simulate articial data from a
stochastic dynamic general equilibrium model and looked at one possible way to
confront the model with the data (by looking at unconditional second moments).
The key step in this analysis was an approximation of the decision function
g(kt ; zt ). We approximated the value function V and the associated decision
rule g on a nite number of grid points. Obviously, this is not the only way to
nd an approximation for g:
An alternative way is to linearize the model dynamics around non-stochastic
steady state and try to nd a decision rule of the following sort
g(kt ; zt ) = akt + bzt :
In some (indeed one) particular cases such linear rules emerge without any
approximation.
Remark 233 Unfortunately, the notation changes as we go along in these
notes, e.g. here we use K to denote per capita capital stock while we used k
for this variable above.
Consider again the standard stochastic one-sector growth model. The output
is produced according to
Yt = Kt (At Lt )1 ;
where Y; K; A; and L are output, capital stock, labor-augmenting productivity
shock, and labor input, respectively. The law of motion for the capital stock is
given by
Kt = Kt (1

) + It

and the aggregate resource constrain is

Yt = Ct + It :
Hence, the wage rate and gross rate of return on capital are given by
wt = (1

)Kt (At Lt ) At = (1

Kt
At Lt

At ;

and
Rt = 1 +

At Lt
Kt

The economy is populated by identical agents (population is normalized to 1)

who want to maximize
U = E0

1
X

u(Ct ; 1

t=0

200

Lt );

where the momentary utility function is

U (C; 1

L) = ln(C) + b ln(1

L); b > 0:

The productivity shock At follows

ln At = A + gt + at ;
where
at = at
Let

+ "t ; "t is white noise.

= 1: Then,
Kt+1 = It = Yt

Ct ;

and
Rt =

At Lt
Kt

The Euler equation for the representative agent is given by

1
Rt+1
= Et
:
Ct
Ct+1
We will try a guess and verify approach to the solution. Let st be agents savings
at time t: Then,
Ct = (1 st )Yt :
Taking the logarithm of the Euler equation
Rt+1
;
Ct+1

ln(Ct ) = ln( ) + ln Et
and substituting our guess we have:
ln [(1

st )Yt ] = ln( ) + ln Et

(1

Rt+1
:
st+1 )Yt+1

Substituting Rt , we have
ln(1

st )

ln(Yt ) = ln( ) + ln Et

st Yt (1

st+1 )

which becomes
ln(st )

ln(1

st ) = ln( ) + ln( ) + ln Et

There is nothing stochastic in this equation, so Et

we have
ln(s)

ln(1

s) = ln( ) + ln( )
201

1
h

ln(1

1
:
st+1
i
1
st+1 =
s);

1
1 st+1 ;

and

which gives us a constant savings rule.

s=

Hence, in this economy

Kt+1 =

(Kt (At Lt )1

Yt =

);

which can be written simply in terms of Kt+1 and At ; once we solve for Lt .
To nd Lt , note that the static FOC for Lt is
wt
Ct
=
;
1 Lt
b
Substituting Ct = (1

st )Yt and wt ; it is easy to show that

Lt = L =

1
) + b(1

(1

s)

Lets put the pieces together now. First note that ln(Yt ) is
ln(Yt )

=
=
=

ln(Kt ) + (1
)(ln At + ln Lt )
ln(s) + ln(Yt 1 ) + (1
)(ln At + ln L)
ln(s) + ln(Yt 1 ) + (1
)(A + gt) + (1

(94)
)at + (1

) ln(L);

which can be written as

ln(Yt )

gt =
=
=

ln(s) + ln(Yt 1 )
gt + (1
)[A + ln(L)] + (1
ln(s) + [ln(Yt 1 ) g(t 1)] + (1
)[A + ln(L)]
ln(s) + (1
)[A + ln(L)]
g + [ln(Yt 1 ) g(t
|
{z
}

)at
g + (1
1)] + (1

Then, on non-stochastic (i.e. with at = 0) balanced growth path, we have

ln(Yt )

gt = X + [ln(Yt )

g(t

1)] = X + [ln(Yt )

gt];

where we used the fact that ln(Yt ) ln(Yt 1 ) = g: Let ln(Yt ) be the value of
ln(Yt ) along a non-stochastic growth path, which is given by
ln(Yt ) =

+ gt:
1
Finally, let ybt be the deviations of ln(Yt ) from ln(Yt ); i.e.
Then,

ybt = ln(Yt )
ybt

ln(Yt ) = ln(Yt )

= ln(Yt

1)

X
1
202

gt:

g(t

1);

(95)

)at
)at :

which gives us
ln(Yt

1)

= ybt

X + g(t

1):

(96)

Substituting (95) and (96) into (94); we have

ybt

=
=

ln(s) + ybt

X + g(t

1
X gt
1
gt + (1
)gt gt + ybt
ybt 1 + (1
)at ;

1) + (1

+ (1

)[A + gt + at + ln(L)]

)at

Then, it can be shown that the deviations of the logarithm of the output from
its steady state follows an AR(2) process.
ybt

=
=
=

ybt 1 + (1
)( at 1 + "t )
ybt 1 + (b
yt 1
ybt 2 ) + (1
( + ) ybt 1
ybt 2 + (1

)"t
)"t :

Note that in this simple model, the linear decision rules allowed us to show that
ybt follows a particular stochastic process. Hence, linearized models provides us
with additional tools to confront the model with the data. Based on this simple
model, a natural question we can ask, for example, is: If the deviations of the
logarithm of GDP in the U.S. data follow an AR(2) process.
Method of Undetermined Coe cients Campbell (1994) analyzes a linearized version of the stochastic one-sector growth models that do not allow for
analytic solutions. His strategy is rst to linearize the model around a nonstochastic growth model. His basic set-up is very similar to the one we analyzed
above without the restriction of = 1.
The output is produced according to
Yt = (At Nt ) Kt1

where Y; K; A; and L are output, capital stock, labor-augmenting productivity

shock, and labor input, respectively. The law of motion for the capital stock is
given by
Kt = Kt (1

) + Yt

Ct

(97)

and
ln(At ) =

ln(At ) + "t :

(98)

He start with an economy in which the labor input is xed, i.e. Nt = 1; and
identical agents who want to maximize
U = E0

1
X
i=0

203

1
i Ct+i

The Euler equation for this economy is then given by

Ct

= Et Ct+1 Rt+1 :

(99)

His strategy is rst log-linearize equations (97) and (99) to arrive at

kt+1

1 kt

2 at

+ (1

1 )ct ;

(100)

kt+1 );

(101)

and
Et ct+1 =

3 Et (at+1

where x = ln(X) for any variables of interest, 1 ; 2 and 3 are constants that
depend on models parameters and = 1= :
Then equations (100), (101), and (98) dene a log-linear system. His strategy
is to arrive from these three equations to linear decision rules for kt+1 and ct+1
that have the following forms
kt+1 =
ct =

kk kt

ck kt

ka at ;

ca at :

The idea is nd these four coe cients (where the term the method of undetermined coe cients come from) that are consistent with (100), (101), and (98).
Once we have these linear rules then we can: 1) use them to simulate data
from our model economies (exactly as we did when we solve the model directly on
a grid), 2) use these linear decision rules to determines the statistical properties
that model variables has to follow (e.g. in this case yt is an ARMA(2,1) process).
Can we generalize this procedure? This is done by Christiano (1992). His
strategy is rst to show that a linearized stochastic model can be written as
#
" r
r 1
X
X
(102)
Et
i st+r 1 i = 0;
i zt+r 1 i +
i=0

i=0

where zt is an endogenous state variable (like capital stock) and st is an exogenous shock, and 0 s and 0 s are constants that depend on model parameters.
Suppose the shock follows an AR(1) process
st = st

+ "t :

(103)

Then, Christiano (1992) shows that (102) and (103) can be written as
zt = Azt

+ Bst ;

where A and B are some matrices (if zt contains more than 1 variable).
Remark 234 Obviously, this note constitutes a very preliminary introduction
to numerical methods that are used to study dynamic macroeconomic models.
You can check Judd (1998) and Marimon and Scott (1999) for more trough
analysis.
204

References
[1] Campbell, John. 1994. Inspecting the Mechanism. Journal of Monetary
Economics, 33, 463-506.
[2] Christiano, Lawrance J. 2001. Solving Dynamic Equilibrium Models by a
Method of Undetermined Coe cients. mimeo. Nortwestern University.
[3] Cooley, Thomas F. and Prescott, Edward C. (1995). Economic Growth
and Business Cycles. in Frontiers of Business Cycle Research, edited by
Thomas F. Cooley, Princeton University Press.
[4] Hansen, Gary. (1985). "Indivisible Labor and the Business Cycle. Journal
of Monetary Economics, 16, 309-327.
[5] Hansen, Gary and Wright, Randall. 1992. The Labor Market in Business
Cycle Theory Federal Reserve Bank of Minneapolis Quarterly Review,
Spring.
[6] Judd, Kenneth L. 1998. Numerical Methods in Economics. MIT University
Press.
[7] Marimon, Ramon and Scott, Andrew (eds.). 1999. Computational Methods
for the Study of Dynamic Economies. Oxford University Press.
[8] Prescott, Edward C. 1986. Theory Ahead of Business Cycle Measurement
Federal Reserve Bank of Minneapolis Quarterly Review, Fall.
[9] Tauchen, George. 1985. Finite State Markov Chain Approximations to Univariate and Vector Autoregressions,Economics Letters, vol. 20, pp. 177181.

205