0 and Q is the following quadratic form
Ax.) =
(l= p)
Routine integrations (exercise) show that:
(a) X is N(1, 07) and ¥ is N(u2, 03),
(b) the correlation between X and Y is p,
(c) X and ¥ are independent if and only if p = 0.
Finally, here is a hint about calculating integrals associated with normal density functions,
Itis an analytical exercise (Problem (4.11.1)) to show that
[oP axa vim
and hence that
FQ)
is indeed a density function. Similarly, a change of variables in the integral shows that the
more general function
an (: = a
oVin?| 2\@
is itself a density function. This knowledge can often be used to shorten calculations. For
example, let X and Y have joint density function given by (10). By completing the square in
the exponent of the integrand, we see that
f@=
cov(X, ¥) = Tl xyf (c, y) dx dy
=foyeer (f x00s..948) ay 102 4.5 Continuous random variables
02.9) = at ( a
1 Vax =p?) a=?)
is the density function of the N(py, | —p”) distribution. Therefore f xg(x, y) dx is the mean,
py, Of this distribution, giving
where
1
cov(X, Y) = f eB" dy.
ply ie y
However, the integral here is, in turn, the variance of the N(0, 1) distribution, and we deduce
that cov(X, Y) = p, as was asserted previously. e
(11) Example. Here is another example of how to manipulate density functions. Let X and
Y have joint density function
f(x,y) eo(-»-*), O oo if
Yeix}| = 0((artSpy72)
i
The following steps may be useful. Let o? = var(Xj), o(n)
and Yn be the characteristic functions of X; and S, /a (n) respectively.
var(Sn), pj = E|X}|, and $j 5.12 Problems 21
(i) Use Taylor's theorem to show that |@j() — 1| <= wo? and |$j(1) — 1+ pop] << [1 p; for
jek
(ii) Show that | log(1 + z) — z| < |z|? if |z| < }, where the logarithm has its principal value.
ii) Show that 03 < pj, and deduce from the hypothesis that max;
(xo =i) {; ifi =2,
irrespective of the initial distribution of X (0). This shows that x = (1 — p, p) is the limiting
distribution, Check that xG = 0. The method of Example (6.9.15) provides an alternative
and easier route to these results. e
(13) Example. Birth process. Recall the birth process of Definition (6.8.11), and suppose
that A; > 0 forall i. The process is uniform if and only if sup;(—gis) = sup;{A;) < 00, and
this is a sufficient condition for the forward and backward equations to have unique solutions.
We saw in Section 6.8 that the weaker condition >; 4; | = oo is necessary and sufficient for
this to hold. e
6.11 Birth-death processes and imbedding
A birth process is a non-decreasing Markov chain for which the probability of moving from
state n to state n+ 1 in the time interval (¢, +h) is Anft-+0(h). More realistic continuous-time
models for population growth incorporate death also, Suppose then that the number X(t) of
individuals alive in some population at time f evolves in the following way:
(a) X is a Markov chain taking values in (0, 1,2,..-).
(b) the infinitesimal transition probabilities are given by
Anh +o(h) ifm =1,
a P(X(t+h) =n+m| X(t) =n) =} pnh+o(h) ifm=—
o(h) if |m| > 1, 6.11 Birth-death processes and imbedding 269
(c) the ‘birth rates’ A, A1,... and the ‘death rates’ wo, 1,... satisfy 4; = 0, wi > 0,
M9 = 0.
‘Then X is called a birth-death process. It has generator G = (gij : i, j = 0) given by
ho do 0 0 0
wi —Oa + 1) A 0 0
0 M2 —Qa + H2) da 0
0 0 u3 —Q3 +43) 43
‘The chain is uniform if and only if sup; (A; + s4i} < 00. In many particular cases we have that
Ao = 0, and then 0 is an absorbing state and the chain is not irreducible.
The transition probabilities pij(t) = P(X(t) = j | X(O) = i) may in principle be
calculated from a knowledge of the birth and death rates, although in practice these functions
rarely have nice forms. It is an easier matter to determine the asymptotic behaviour of the
process —> oo. Suppose that 4; > Oand 1; > Oforall relevant. A stationary distribution
x would satisfy 7G = 0 which is to say that
—homo + wim = 0,
An-1Tn=1 ~ On + Mn) + Mots! =O if nl
A simple induction yields that
dod
@ Ty =
mina
Such a vector z is a stationary distribution if and only if 0,» = 1; this may happen if and
only if
So od Ant
(3) ———_ <0,
where the term n = 0 is interpreted as 1; if this holds, then
co 1
a nu= (53 4x1) ,
a Ha Hen
We have from Theorem (6.9.21) that the process settles into equilibrium (with stationary
distribution given by (2) and (4)) if and only if the summation in (3) is finite, a condition
requiring that the birth rates are not too large relative to the death rates.
Here are some examples of birth-death processes.
(5) Example. Pure birth. The death rates satisfy ji = 0 for all n e
+ Alternatively, note thatthe matrix is tridiagonal, whence the chain is reversible in equilibrium (see Problem
(6.15.16¢)). Now seek a solution to the detailed balance equations. 270 6.11 Markov chains
(6) Example. Simple death with immigration. Let us model a population which evolves in
the following way. At time zero the size X (0) of the population equals /. Individuals do not
reproduce, but new individuals immigrate into the population at the arrival times of a Poisson
process with intensity 4 > 0. Each individual may die in the time interval (¢, 1 + h) with
probability 2h + o(k), where. > 0. The transition probabilities of X (¢) satisfy
pilh) = P(X +h) = j |X =i)
_ [ PC ~é arrivals, no deaths) + 0(h) if j > i,
~~ | B@ —j deaths, no arrivals) + 0(h) if j 1 then < 1 and extinction is
not certain, Indeed E(Z,,) grows geometrically as n —> 00, and it can be shown that
P(Zn — 00 | non-extinction) = I
when this conditional probabi
define Wy = Zn/E(Zn) where E(Zp
show that
bly interpreted. To see just how fast Z, grows, we
11", and we suppose that p: > 1. Easy calculations
ol
>
E(Wp) = 1, var(W,) = 2 # O _,
wap wae
asin — 00,
and it seems that W, may have some non-trivial limit}, called W say. In order to study W,
define gn(s) = E(s”*), Then
Bn(s) = B97") = Gy(s*")
and (1) shows that gq satisfies the functional recurrence relation
8n(5) = G(gn1(s'/)).
Now, as 1 + 00, we have that W, > W and gn(s) > g(s) = E(s™), and we obtain
© (3) = G(g(s'/*))
We are asserting that the sequence {Wn} of variables converges to a limit variable W. The convergence of
random variables is a complicated topic described in Chapter 7. We overlook the details for the moment. 5.5 Age-dependent branching processes 175
by abandoning some of our current notions of mathematical rigour. This functional equation
can be established rigorously (see Example (7.8.5)) and has various uses. For example,
although we cannot solve it for g, we can reach such conclusions as ‘if E(Z?) < 00 then W
is continuous, apart from a point mass of size 1 at zero’
We have made considerable progress with the theory of branching processes. They are
reasonably tractable because they satisfy the Markov condition (see Example (3.9.5)). Can
you formulate and prove this property?
Exercises for Section 5.4
1. Let Z, betthe size of the nth generation in an ordinary branching process with Zo = 1,E(Z1) = 1,
and var(Z}) > 0. Show that E(ZpZm) = h"-"E(Z3) for m 0, there exists 5 > 0, such
that E(\X|74) <¢ for all A such that P(A) < 6.
5.7 Characteristic functions
Probability generating functions proved to be very useful in handling non-negative integral
random variables. For more general variables X it is natural to make the substitution s =e"
in the quantity G(s) = E(s*).
(1) Definition. The moment generating function of a variable X is the function M : R >
[0, 00) given by M(t) = E(e"*).
‘Moment generating functions are related to Laplace transforms} since
MO) = [evar = ferroas
if X is continuous with density function f. They have properties similar to those of probability
generating functions. For example, if M(¢) < 00 on some open interval containing the origin
then:
(@) EX = MO), E(X*) = M® (0);
(b) the function M may be expanded via Taylor’s theorem within its circle of convergence,
which is to say that M is the ‘exponential generating function’ of the sequence of
moments of X:
Note the change of sign from the usual Laplace transform of f, namely f(t) = fe" f(x) dx. 182 5.7 Generating functions and their applications
(©) if X and ¥ are independent then} My+v(t) = Mx(e)My (1).
Moment generating functions provide a very useful technique but suffer the disadvantage
that the integrals which define them may not always be finite. Rather than explore their
properties in detail we move on immediately to another class of functions that are equally
useful and whose finiteness is guaranteed
(2) Definition. The characteristic function of X is the function 6 : R -> C defined by
v=1.
(t) =E(e*) where
We often write ¢x for the characteristic function of the random variable X. Characteristic
functions are related to Fourier transforms, since $(t) = fe!" dF(x). In the notation of
Section 5.6, ¢ is the abstract integral of a complex-valued random variable. It is well defined
in the terms of Section 5.6 by #(t) = E(costX) + iE(sin¢X). Furthermore, ¢ is better
behaved than the moment generating function M
(3) Theorem. The characteristic function @ satisfies:
(a) 6) = 1, 16@| <1 forall t,
(b) @ is uniformly continuous on R,
(©) @ is non-negative definite, which is to say that D4 6(t; — %)zjzk > 0 for all real
1h, tay ++ sty and complex 21,22, +++ 5%n
Proof. (a) Clearly ¢(0) = E(1) = 1. Furthermore
wor s fiemar = far =
(b) We have that
IG +h) — G()| = [E(eO*O* — ef%)| < BlelX (eX — 1)| < EH)
where Y(h) = |e!" —1|. However, |Y (h)| < 2and Y(h) > Oash > 0,andsoE(¥(h)) > 0
by bounded convergence (5.6.12).
(©) We have that
Yo - w= Df exp(itjx)][Ze exp(—inx) | dF
ie
ie
=#(
Theorem (3) characterizes characteristic functions in the sense that @ is a characteristic
function if and only if it satisfies (3a), (3b), and (3c). This result is called Bochner’s theorem,
for which we offer no proof. Many of the properties of characteristic functions rely for their
proofs on a knowledge of complex analysis. This is a textbook on probability theory, and will
2.
yz exo4iy%] ) > 0. .
i
++This is essentially the assertion that the Laplace transform of a convolution (see equation (4.8.2)) is the
product of the Laplace transforms, 5.7 Characteristic functions 183
not include such proofs unless they indicate some essential technique. We have asserted that
the method of characteristic functions is very useful; however, we warn the reader that we
shall not make use of them until Section 5.10. In the meantime we shall establish some of
their properties.
First and foremost, from a knowledge of $y we can recapture the distribution of X. The
full power of this statement is deferred until the next section; here we concern ourselves only
with the moments of X, Several of the interesting characteristic functions are not very well
behaved, and we must move carefully.
(4) Theorem.
(a) f6(O) exists then {
(b) IfE|X*| < 00 then?
E|X*| <0o ifkis even,
E|X'!| < 00 ifk is odd.
S e(xs
60) = Mwy 00,
jo J
and so 6 (0)
KE(X*),
Proof. This is essentially Taylor's theorem for a function of a complex variable. For the
proof, see Moran (1968) or Kingman and Taylor (1966). .
One of the useful properties of characteristic functions is that they enable us to handle sums
of independent variables with the minimum of fuss.
(5) Theorem, If X and ¥ are independent then $y+y(t) = 6x(Oor(0).
Proof. We have that
bxsy(t) = E(e# XH) = Bei eit,
Expand each exponential term into cosines and sines, multiply out, use independence, and put
back together to obtain the result. a
(6) Theorem. Ifa, b € Rand Y = aX +b then gy (t) ox(at).
Proof. We have that
by () = Be OX* — BeitdgiladXy
= elt Ee) = el" x(at). .
‘We shall make repeated use of these last two theorems. We sometimes need to study
collections of variables which may be dependent.
(7) Definition. The joint characteristic function of X and ¥ is the function x,y : R? > RB
given by @y,y(s, 1) = E(el** e'*”),
Notice that px, y(s, 1) = $sx42v(1). As usual we shall be interested mostly in independent
variables.
+{See Subsection (10) of Appendix I for a reminder about Landau’s O/o notation, 184 5.7 Generating functions and their applications
(8) Theorem. Random variables X and Y are independent if and only if
xy(s,t) = Ox(s)py(t) for alls and t.
Proof. If X and Y are independent then the conclusion follows by the argument of (5). The
converse is proved by extending the inversion theorem of the next section to deal with joint
distributions and showing that the joint distribution function factorizes. .
Note particularly that for X and ¥ to be independent it is not sufficient that
0} dxv(t.t) =ox(Obv() forall
Exercise. Can you find an example of dependent variables which satisfy (9)?
We have seen in Theorem (4) that it is an easy calculation to find the moments of X by
differentiating its characteristic function x(t) at ¢ = 0. A similar calculation gives the ‘joint
moments’ E(X/Y*) of two variables from a knowledge of their joint characteristic function
x,y (s, #) (See Problem (5.12.30) for details).
‘The properties of moment generating functions are closely related to those of characteristic
functions. In the rest of the text we shall use the latter whenever possible, but it will be
appropriate to use the former for any topic whose analysis employs Laplace transforms; for
example, this is the case for the queueing theory of Chapter 11
(10) Remark. Moment problem. If Iam given a distribution function F, then I can calculate
the corresponding moments m,(F) = [°, xd F(x), k = 1, 2, ..., whenever these integrals
exist. Is the converse true: does the collection of moments (m,(F) : k = 1,2,...) specify F
uniquely? The answer is no: there exist distribution functions F and G, all of whose moments
exist, such that F 4 G but mg(F) = my(G) for all k. The usual example is obtained by using
the log-normal distribution (see Problem (5.12.43)).
Under what conditions on F is it the case that no such G exists? Various sets of conditions
are known which guarantee that F is specified by its moments, but no necessary and sufficient
condition is known which is easy to apply to a general distribution. Perhaps the simplest
sufficient condition is that the moment generating function of F, M(t) = [°%_ e"* d F(x), be
finite in some neighbourhood of the point ¢ = 0. Those familiar with the theory of Laplace
transforms will understand why this is sufficient.
(11) Remark. Moment generating function. The characteristic function of a distribution is
closely related to its moment generating function, in a manner made rigorous in the following
theorem, the proof of which is omitted. [See Lukacs 1970, pp. 197-198.]
(12) Theorem. Let M(t) = E(e'*), t € R, and $(t) = E(e''*), t € C, be the moment
generating function and characteristic function, respectively, of a random variable X. For
any a > 0, the following three statements are equivalent:
(a) |M(@)| < 0 for |t| tim Gq).
st
n— G(s)
or
Let n + 00 in (5) to obtain
(6) H(G(s))=G(mH(s) if O 1, there is probability x—' that the m-cells die out, and probability
1 —«7! that their number grows beyond all bounds.
Thus there is strictly positive probability of the malignant cells becoming significant if and
only if the carcinogenic advantage exceeds one.
Proof. Let X(t) be the number of m-cells at time 1, and let Ty (= 0), Ti, Ta,... be the
sequence of times at which X changes its value. Consider the imbedded discrete-time process
X = {X,), where X, = X(Ty+) is the number of m-cells just after the nth transition; X is
a Markov chain taking values in (0, 1, 2, ...). Remember the imbedded random walk of the
birth-death process, Example (6.11.9); in the case under consideration a little thought shows
that X has transition probabilities
K
weBT ear if 140, poo=1
Pit = Pii-1 =
K+1
‘Therefore X,, is simply a random walk with parameter p = «/(k + 1) and with an absorbing
barrier at 0. The probability of ultimate extinction from the starting point X(0) = 1 is «~!
‘The walk is symmetric and null persistent if « = 1 and all non-zero states are transient if
k>l. .
If « = 1, the same argument shows that the m-cells certainly die out whenever there is a
finite number of them to start with. However, suppose that they are distributed initially at the
points of some (possibly infinite) set. It is possible to decide what happens after a long length
of time; roughly speaking this depends on the relative densities of benign and malignant cells
over large distances. One striking result is the following.
(18) Theorem. If « = 1, the probability that a specified finite collection of points contains
only one type of cell approaches one as t > oo.
Sketch proof. If two cells have a common ancestor then they are of the same type. Since off-
spring displace any neighbour with equal probability, the line of ancestors of any cell performs
a symmetric random walk in two dimensions stretching backwards in time. Therefore, given
any two cells at time #, the probability that they have a common ancestor is the probability
that two symmetric and independent random walks $1 and 2 which originate at these points
have met by time ¢. The difference S} — Sp is also a type of symmetric random walk, and, as
in Theorem (5.10.17), $1 — S2 almost certainly visits the origin sooner or later, implying that
P(Si(t) = S2(¢) for some t) = 1 .
(19) Example. Simple queue. Here is a simple model for a queueing system. Customers
enter a shop in the manner of a Poisson process, parameter 2. They are served in the order of 6.13 Spatial Poisson processes 281
their arrival by a single assistant; each service period is a random variable which we assume
to be exponential with parameter j1 and which is independent of all other considerations. Let
X (0) be the length of the waiting line at time ¢ (including any person being served). It is easy
to see that X is a birth—death process with parameters 2, =A forn > 0, fy = forn > 1
‘The server would be very unhappy indeed if the queue length X(t) were to tend to infinity as
1 — 00, since then he or she would have very few tea breaks. It is not difficult to see that the
distribution of X (1) settles down to a limit distribution, as ¢ + 00, if'and only if A < 42, which
is to say that arrivals occur more slowly than departures on average (see condition (6.11.3)).
We shall consider this process in detail in Chapter 11, together with other more complicated
queueing models, e
Exercises for Section 6.12
1. Customers entering a shop are served in the order of their arrival by the single server. They
arrive in the manner of a Poisson process with intensity 4, and their service times are independent
exponentially distributed random variables with parameter j.. By considering the jump chain, show
that the expected duration of a busy period B of the server is ( — 4)~! when 4 < py. (The busy
period runs from the moment a customer arrives to find the server free until the earliest subsequent
time when the server is again free.)
2. Disasters. Immigrants arrive at the instants of a Poisson process of rate v, and each independently
founds a simple birth process of rate 4. At the instants of an independent Poisson process of rate 5,
the population is annihilated. Find the probability generating function of the population X(t), given
that X (0) = 0.
3. More disasters, In the framework of Exercise (2), suppose that each immigrant gives rise to a
simple birth-death process of rates 2 and j1. Show that the mean population size stays bounded if and
only if 8 > A —
4, ‘The queue M/G/oo. (See Section 11.1.) An fip server receives clients at the times of a Poisson
process with parameter A, beginning at time 0. The ith client remains connected for a length 5;
of time, where the S; are independent identically distributed random variables, independent of the
process of arrivals. Assuming that the server has an infinite capacity, show that the number of clients
being serviced at time t has the Poisson distribution with parameter A {1 — G(x)] dx, where G is
the common distribution function of the $}
6.13 Spatial Poisson processes
The Poisson process of Section 6.8 is a cornerstone of the theory of continuous-time Markov
chains. It is also a beautiful process in its own right, with rich theory and many applications.
While the process of Section 6.8 was restricted to the time axis R. = [0, 00), there is a useful
generalization to the Euclidean space R? where d > 1.
We begin with a technicality. Recall that the essence of the Poisson process of Section 6.8,
was the set of arrival times, a random countable subset of R. Similarly, a realization of a
Poisson process on R¢ will be a countable subset IT of R¢. We shall study the distribution of
TI through the number |I7 9 A| of its points lying in a typical subset A of R¢. Some regularity
will be assumed about such sets A, namely that there is a well defined notion of the ‘volume’
of A. Specifically, we shall assume that A ¢ 84, where B4 denotes the Borel o-field of R4,
being the smallest o-field containing all boxes of the form []/_; (a;, b:]. Members of B4 are
called Borel sets, and we write |A| for the volume (or Lebesgue measure) of the Borel set A. 282 6.13 Markov chains
(J) Definition, The random countable subset I] of R@ is called a Poisson process with
(constant) intensity 4 if, for all A € 4, the random variables N(A) = [IT 7 A| satisfy:
(a) N(A) has the Poisson distribution with parameter \|A], and
(b) if Aj, Aa,..., An are disjoint sets in B4, then N(Ay), N(Az).<.., (An) are indepen-
dent random variables.
‘We often refer to the counting process N as being itself a Poisson process if it satisfies (a)
and (b) above. In the case when 4 > 0 and |A| = 00, the number |IT 7 A| has the Poisson
distribution with parameter 00, a statement to be interpreted as P(|ITN Al = 00) =
tis not difficult to see the equivalence of (1) and Definition (6.8.1) when d = 1. That is,
if d = 1 and N satisfies (1), then
(2) M(t)=N((0,tl), =0,
satisfies (6.8.1). Conversely, if M satisfies (6.8.1), one may find a process N satisfying (1)
such that (2) holds, Attractive features of the above definition include the facts that the origin
plays no special role, and that the definition may be extended to sub-regions of IR? as well as
to 0-fields of subsets of general measure spaces.
‘There are many stochastic models based on the Poisson process. One-dimensional pro-
cesses were used by Bateman, Erlang, Geiger, and Rutherford around 1909 in their investiga-
tions of practical situations involving radioactive particles and telephone calls, Examples in
two and higher dimensions include the positions of animals in their habitat, the distribution
of stars in a galaxy or of galaxies in the universe, the locations of active sites in a chemical
reaction or of the weeds in your lawn, and the incidence of thunderstorms and tomadoes. Even
when a Poisson process is not a perfect description of such a system, it can provide a relatively
simple yardstick against which to measure the improvements which may be offered by more
sophisticated but often less tractable models.
Definition (1) utilizes as reference measure the Lebesgue measure on IR4, in the sense that
the volume of a set A is its Euclidean volume. It is useful to have a definition of a Poisson
process with other measures than Lebesgue measure, and such processes are termed ‘non-
homogeneous’. Replacing the Euclidean element 2 dx with the element 4(x) dx, we obtain
the following, in which A(A) is given by
@) Nea) = f a00dx, Ac 8.
A
(4) Definition. Let d > 1 and let 4: R¢ — R be a non-negative measurable function such
that A(A) < 00 for all bounded A, The random countable subset II of R¢ is called a non-
homogeneous Poisson process with intensity function 2 if, for all A € B4, the random
variables N(A) = |IT9 A| satisfy:
(a) N(A) has the Poisson distribution with parameter A(A), and
(b) if Aj, Aa, ..., An are disjoint sets in B4, then N(A1), N(A2),
dent random variables.
N(Aq) are indepen-
We call the function A(A), A € 84, the mean measure of the process [. We have
constructed A as the integral (3) of the intensity function 4; one may in fact dispense altogether
with the function 2, working instead with measures A which ‘have no atoms’ in the sense that
A({x}) = 0 for all x € R4 6.13 Spatial Poisson processes 283
Our first theorem states that the union of two independent Poisson processes is also a
Poisson process. A similar result is valid for the union of countably many independent
Poisson processes.
(5) Superposition theorem. Ler I’ and 1” be independent Poisson processes on R4 with
respective intensity functions 2! and 2". The set TI = TI’ U 11" is a Poisson process with
intensity function } = +2".
Proof. Let N'(A) = |TV’ Al and N’”"(A) = |TV" Al. Then N’(A) and N”(A) are indepen-
dent Poisson-distributed random variables with respective parameters A’(A) and A/(A), the
integrals (3) of 4 and 2”. It follows that the sum S$(A) = N’(A) + N”(A) has the Poisson
distribution with parameter A'(A) + A’(A). Furthermore, if Ay, A2,... are disjoint, the
random variables S(A1), S(Aa), ... are independent. It remains to show that, almost surely,
= |I1N A| for all A, which is to say that no point of 11’ coincides with a point of 11”
This is a rather technical step, and the proof may be omitted on a first read.
Since R¢ is a countable union of bounded sets, it is enough to show that, for every bounded
A © R@, A contains almost surely no point common to [’ and T1”, Let n > 1 and, for
k = (ki, kay. yka) € 24, let Buln) = [1d (ki2-", (ki + 127%]; cubes of this form are
termed n-cubes or n-boxes. Let A be a bounded subset of R@, and A be the (bounded) union
of all Bx (0) which intersect A. The probability that A contains a point common to IT’ and I”
is bounded for all n by the probability that some By (m) lying in A contains a common point.
This is no greater than the mean number of such boxes, whence
P(N'AM’NA ge) =< > P(N'(Bq() = 1, N"(Bu(m)) = 1)
K:By(m)SA
= ¥a-
kK: Bun) SA
< > A'(By(n))A(By(n)) since 1 — e* < x forx > 0
k:B(McA
<_ max_{A'(Bin))} YA Buln)
eS kROSA
= NBM) (| =e ANAL)
= M,(A)A"(A)
where
M,(A) = max_ A’(Bx(n)).
K:Bq(n) CA
Itis the case that M,(A) > 0. asm —> 00. This is easy to prove when 2! is a constant function,
since then My (A) |Bg(n)| = 27". It is not quite so easy to prove for general 4'. Since we
shall need a slightly more general argument later, we state next the required result.
(6) Lemma. Let 1 be a measure on the pair (R4, B4) which has no atoms, which is to say
that w({y}) = Ofor ally € R4, Letn > 1, and By(n) = [If (i2™, (ki + 127"), k € 24
For any bounded set A, we have that
max j1(By(n)) +0 asn > 0.
kBROSA 284 6.13 Markov chains
Returning to the proof of the theorem, it follows by Lemma (6) applied to the set A that
M,(A) + 0.as.n > 00, and the proof is complete. :
Proof of Lemma (6). We may assume without loss of generality that A is a finite union of
O-cubes. Let
M,(A) = max_ u(Bx(n)),
()= |. naen H(Bx(n))
and note that M, > Mni1. Suppose that M,(A) 7 0. There exists 5 > 0 such that
M,(A) > 6 for all n, and therefore, for every n > 0, there exists an n-cube By(n) CA with
(Bx (n)) > 8. We colour an m-cube C black if, for all n > m, there exists an n-cube C’ CC
such that j.(C’) > 5. Now A is the union of finitely many translates of (0, 1]¢, and for at
least one of these, By say, there exist infinitely many n such that Bo contains some n-cube B’
with 1.(B') > 6. Since 1(-) is monotonic, the 0-cube Bo is black. By a similar argument,
Bo contains some black I-cube By. Continuing similarly, we obtain an infinite decreasing
sequence Bo, B),... such that each B, is a black r-cube. In particular, (B,) > 6 for all r,
whereas}
jim, wea) = n(B.) = md =o
by assumption, where y is the unique point in the intersection of the B,. The conclusion of
the theorem follows from this contradiction. .
It is possible to avoid the complication of Lemma (6) at this stage, but we introduce the
Jemma here since it will be useful in the forthcoming proof of Rényi’s theorem (17). In
an alternative and more general approach to Poisson processes, instead of the ‘random set”
TT one studies the ‘random measure’ N. This leads to substantially easier proofs of results
corresponding to (5) and the forthcoming (8), but at the expense of of extra abstraction.
The following ‘mapping theorem’ enables us to study the image of a Poisson process I
under a (measurable) mapping f : R4 > R°, Suppose that IT is a non-homogeneous Poisson
process on R¢ with intensity function 4, and consider the set f(T) of images of I under f
We shall need that f(I1) contains (with probability 1) no multiple points, and this imposes a
constraint on the pair 4, f. The subset B C RY contains the images of points of TT lying in
f~'B, whose cardinality is a random variable having the Poisson distribution with parameter
‘A(f7'B). The key assumption on the pair A, f will therefore be that
(a) ACf7l{y}) =0 forally € RB,
where A is the integral (3) of &
(8) Mapping theorem. Let I be a non-homogeneous Poisson process on R@ with intensity
function d, and let f : R4 > RS satisfy (7). Assume further that
OO) uB)=Ae'B)= [| neds, Bea,
lye
satisfies 4.(B) < 00 for all bounded sets B. Then f(T) is a non-homogeneous Poisson
process on RS with mean measure L.
+ We use here a property of continuity of general measures, proved in the manner of Lemma (1.3.5). 6.13 Spatial Poisson processes 285
Proof. Assume for the moment that the points in f(I1) are distinct. The number of points
of f(T) lying in the set B (C R°) is [TN f~!B], which has the Poisson distribution with
parameter 1(B), as required. If By, Bo, ... are disjoint, their pre-images f—'By, f~'Bp, ...
are disjoint also, whence the numbers of points in the B; are independent. It follows that
F(T) is a Poisson process, and it remains only to show the assumed distinctness of f (I1).
The proof of this is similar to that of (5), and may be omitted on first read.
We shall work with the set I U of points of TT lying within the unit cube U = (0, 1)? of
R¢. This set is a Poisson process with intensity function
A(x) ifxe U,
0 otherwise,
du) = {
and with finite total mass A(U) = fy, 4(x) dx. (This is easy to prove, and is in any case a
very special consequence of the forthcoming colouring theorem (14).) We shall prove that
F(A) isa Poisson process on IR with mean measure
A similar conclusion will hold for the set (IT Uy) where Uy = k + U fork € Z4, and
the result will follow by the superposition theorem (5) (in a version for the sum of countably
many Poisson processes) on noting that the sets f (11M Ux) are independent, and that, in the
obvious notation,
B)= anooldx=[ agadr.
Leal ) fA nc} IK I, (x) dx
Write IY = 117 U, and assume for the moment that the points f(I1’) are almost surely
distinct. The number of points lying in the subset B of R° is |T1’ 9 f—!B|, which has the
Poisson distribution with parameter zy (B) as required. If By, Bo, ... are disjoint, their pre-
images f~'B,, f~'B,... are disjoint also, whence the numbers of points in the Bj are
independent. It follows that /(I1’) is a Poisson process with mean measure jy
It remains to show that the points in f(I1’) are almost surely distinct under hypothesis (7).
The probability that the small box Bx = []}_y(k:2~", (ki + 1)2-"] of R¥ contains wo or
more points of f(I’) is
1 eM = pape < 1 (1+ wx) (1 = x) = WR
where 1x = 1y(Bx), and we have used the fact that e* > 1 —x for x > 0. The mean
number of such boxes within the unit cube Us = (0, 1)" is no greater than
Doak = Mn Yo an = Money Us)
x x
where
Mn = max Hx > 0 asin > 00
by hypothesis (7) and Lemma (6). Now yy(Us) = A(U) < 00, and we deduce as in the
proof of Theorem (5) that U, contains almost surely no repeated points, Since R° is the union 286 6.13 Markov chains
of countably many translates of U, to each of which the above argument may be applied, we
deduce that R® contains almost surely no repeated points of { (I1’). The proof is complete. Ml
(10) Example. Polar coordinates. Let I be a Poisson process on R? with constant rate 2,
and let f : R? + R? be the polar coordinate function f (x, y) = (r, @) where
It is straightforward to check that (7) holds, and we deduce that f (11)
? with mean measure
isson process on
uo) f nadrdy = f ardr dd,
yop Bos
where S = f(R?) is the strip {(r,@) : r > 0, 0 < @ < 2m}. We may think of f(T) asa
Poisson process on the strip S having intensity function Ar. e
We turn now to one of the most important attributes of the Poisson process, which unlocks
the door to many other useful results. This is the so-called ‘conditional property’, of which
we saw a simple version in Theorem (6.12.7).
(11) Theorem, Conditional property. Let II be a non-homogeneous Poisson process on R4
with intensity function i, and let A be a subset of R4 such that) < A(A) < 00. Conditionalon
the event that |[17 A| = n, the n points of the process lying in A have the same distribution as
1 points chosen independently at random in A according to the common probability measure
— AB)
QB) = Tay’ cA.
Since
_ f 20)
(12) ce = f 2 as,
the relevant density function is 4(x)/A(A) for x ¢ A. When T1 has constant intensity 4, the
theorem implies that, given |I1 9 A| = n, the n points in question are distributed uniformly
and independently at random in A.
Proof. Let Aj, A2,..., Ag be a partition of A. It is an elementary calculation that, if my +
ny tet =n,
(13) P(N (AL) = m1, N(A2) = m2, NA) = 4 |N(A) =)
sees by independence
Th AAD" e MA) nit
A(A)"eAA Jat
n!
Ailnpl nn VAD" Ar)? QWAwy"™.
ke 6.13 Spatial Poisson processes 287
‘The conditional distribution of the positions of the n points is specified by this function of
Al, Ady ey Ane
We recognize the multinomial distribution of (13) as the joint distribution of n points
selected independently from A according to the probability measure Q. It follows that the
joint distribution of the points in I19 A, conditional on there being exactly n of them, is the
same as that of the independent sample. .
‘The conditional property enables a proof of the existence of Poisson processes and aids
the simulation thereof. Let 4 > 0 and let Ay, 2, ... be a partition of R7 into Borel sets of
finite Lebesgue measure. For each i, we simulate a random variable N; having the Poisson
distribution with parameter A|A;|. Then we sample n independently chosen points in A;, each
being uniformly distributed on A;. The union over i of all such sets of points is a Poisson
process with constant intensity A. A similar construction is valid for a non-homogeneous
process. The method may be facilitated by a careful choice of the A;, perhaps as unit cubes
of R’.
The following colouring theorem may be viewed as complementary to the superposition
theorem (5). As in the latter case, there is a version of the theorem in which points are marked
with one of countably many colours rather than just two.
(14) Colouring theorem. Let 1 be a non-homogeneous Poisson process on R¢ with intensity
function 2. We colour the points of Tl in the following way. A point of U1 at position x
is coloured green with probability y(x); otherwise it is coloured scarlet (with probability
a(x) = 1 = y(%)). Points are coloured independently of one another. Let I" and ¥: be the sets
of points coloured green and scarlet, respectively. Then and © are independent Poisson
processes with respective intensity functions y (x)A(X) and 0 (X)A(X).
Proof. Let A C R@ with A(A) < 00, By the conditional property (11), if [1M A] = n,
these points have the same distribution as n points chosen independently at random from A
according to the probability measure Q(B) = A(B)/A(A). We may therefore consider n
points chosen in this way. By the independence of the points, their colours are independent
of one another. The chance that a given point is coloured green is 7 = ¥, (x) dQ, the
corresponding probability for the colour scarlet being 7 = 1 — 7 = J, o(x) dQ. It follows
that, conditional on |I19 Al = n, the numbers Nz and N, of green and scarlet points in A
have, jointly, the binomial distribution
P(N, = g, Ns =5|N(A)= , where g +5
The unconditional probability is therefore
(g +5)! g, A(A EAD
ye —
gis! (g +s)!
_ FAA) Se-FAA ACA eA“
~ gl - s!
which is to say that the numbers of green and scarlet points in A are independent. Furthermore
they have, by (12), Poisson distributions with parameters
7aca)= f yooaarde= f voor as,
P(g = 8, Ne=5)=
7aa)= f ow@ade= f o@neoax
A A 288 6.13 Markov chains
Independence of the counts of points in disjoint regions follows trivially from the fact that
TT has this property. .
(15) Example. The Alternative Millennium Dome contains zones, and visitors are required
to view them all in sequence. Visitors arrive at the instants of a Poisson process on Ry. with
constant intensity 2, and the rth visitor spends time X,,. in the sth zone, where the random
variables X,,s,7 > 1, 1 0, and let V,() be the number of visitors in zone s at time t. Show that, for fixed
1, the random variables V,(t), | 1.
(17) Rényi’s theorem. Let TI be a random countable subset of R4, and let i: R4 > R be a
non-negative integrable function satisfying A(A) = f, W(x) dx < 00 for all bounded A. If
(1s) PIINA =) =e 44
{for any finite union A of boxes, then T1 is a Poisson process with intensity function }.
Proof. Let n > 1, and denote by /x(n) the indicator function of the event that By(n) is
non-empty. It follows by (18) that the events /k(n), k € Z¢, are independent.
Let A be a bounded open set in R¢, and let (A) be the set of all k such that By(n) CA.
Since A is open, we have that
(ag) N(A) =|TINAl= lim T%(A) where T;(A)= > h(n):
ue ke uta)
note that, by the nesting of the boxes By(v), this is a monotone increasing limit. We have also
that
(20) A(A) = lim, YY ABx(n)).
keX(A) 6.13 Spatial Poisson processes 289
‘The quantity T,(A) is the sum of independent variables, and has probability generating
function
ey E(sTa(A)
T] {5+ = se aur},
ke Kn(A)
We have by Lemma (6) that A(Bq(n)) —> 0 uniformly in k € K,(A), as n > 00. Also, for
fixed s € [0, 1), there exists (5) satisfying (5) t 1 as 6 | 0 such that
(22) e985 4 (1— sje se CO" ifO 0.
We take the limit as 5 | 0 to obtain the Poisson distribution of N(A)
It remains to prove the independence of the variables N(Aj), N(A2),... for disjoint
Ai, Ao,.... This is an immediate consequence of the facts that T,(A1), Tn(A2),---
independent, and Ty (Aj) —» N(Aj) as n > 00. :
‘There are many applications of the theory of Poisson processes in which the points of a
process have an effect elsewhere in the space. A well-known practical example concerns the
fortune of someone who plays. lottery. The player wins prizes at the times of a Poisson process
TI on Ry, and the amounts won are independent identically distributed random variables.
Gains are discounted at rate a. The total gain G(¢) by time t may be expressed in the form
Go= YO aT wr,
Tent
where Wr is the amount won at time T (€ I). We may write
Gw)= > r(t—T)Wr
Ten
where
0 ifu <0,
ru) = :
a ifu>0,
Such sums may be studied by way of the next theorem. We state this in the special case
of a homogeneous Poisson process on the half-line R,., but it is easily generalized. The
one-dimensional problem is sometimes termed shot noise, since one may think of the sum
as the cumulative effect of pulses which arrive in a system, and whose amplitudes decay
exponentially. 290 6.13 Markov chains
(23) Theorem. Let [1 be a Poisson process on R with constant intensity d, letr: R > R
be a smooth function, and let {Wx : x € TI} be independent identically distributed random
variables, independent of TI. The sum
GO= YI ore-Ws
xell, 220
has characteristic function
a
E(eO) = ovofp. [ (Ee) — nas],
0
where W has the common distribution of the Ws. In particular,
E(G(t)) = az f r(s)ds.
lo
Proof. This runs just like that of Theorem (6.12.11), which is in fact a special case. It is
left as an exercise to check the details. The mean of G(f) is calculated from its characteristic
function by differentiating the latter at 9 = 0. a
A similar idea works in higher dimensions, as the following demonstrates.
(24) Example. Olbers’s paradox. Suppose that stars occur in ° at the points (Rj: i > 1)
of a Poisson process with constant intensity 4. The star at R; has brightness B;, where the B;
are independent and identically distributed with mean . The intensity of the light striking
an observer at the origin O from a star of brightness B, distance r away, is (in the absence
of intervening clouds of dust) equal to cB/r?, for some absolute constant c, Hence the total
illumination at O from stars within a large ball S with radius a is
a=» RP
EIR [sa
Conditional on the event that the number Na of such stars satisfies Ny = n, we have from the
conditional property (11) that these n stars are uniformly and independently distributed over
S. Hence fl
iri?
1
E(/a | Na) = NacB—
{aNd = Neots [
Now E(N,) = |S], whence
1
Ela = eB [ peal = Bra)
The fact that this is unbounded as a — 00 is called ‘Olbers’s paradox’, and suggests that
the celestial sphere should be uniformly bright at night. The fact that it is not is a problem
whose resolution is still a matter for debate. One plausible explanation relies on a sufficiently
fast rate of expansion of the Universe. e
This theorem is sometimes called the Campbell-Hardy theorem. See also Exercise (6.13.2). 6.14 Markov chain Monte Carlo 291
Exercises for Section 6.13
1. Ina certain town at time ¢ = 0 there are no bears. Brown bears and grizzly bears arrive as
independent Poisson processes B and G with respective intensities 6 and y.
(a) Show that the first bear is brown with probability 6/(8 + y).
(b) Find the probability that between two consecutive brown bears, there arrive exactly r grizzly
bears.
(©) Given that B(1) = 1, find the expected value of the time at which the first bear arrived,
2. Campbell-Hardy theorem. Let IT be the points of a non-homogeneous Poisson process on R¢
with intensity function A. Let S = Ser g(X) where g is a smooth function which we assume for
convenience to be non-negative. Show that E(S) = faa g(u)A(u) du and var(S) = fea g(u)?A(u) du,
provided these integrals converge.
3. Let I be a Poisson process with constant intensity 2. on the surface of the sphere of with radius
1. Let P be the process given by the (X, Y) coordinates of the points projected on a plane passing
through the centre of the sphere. Show that P is a Poisson process, and find its intensity function.
4, Repeat Exercise (3), when [1 is a homogeneous Poisson process on the ball {(x1, x2,.3) : x? +
a3 +23 <1).
5. You stick pins in a Mercator projection of the Earth in the manner of a Poisson process with
constant intensity 4. What is the intensity function of the corresponding process on the globe? What
would be the intensity function on the map if you formed a Poisson process of constant intensity 2. of
meteorite strikes on the surface of the Earth?
6 Shocks. The rth point 7 of a Poisson process V of constant intensity 4 on Ry gives rise to an
effect Xpe~*¢—™) at time ¢ > T;, where the X, are independent and identically distributed with
finite variance, Find the mean and variance of the total effect $(¢) = SN Xpe-*(—™ in terms of
the first two moments of the X,, and calculate cov(S(s), S(t)).
‘What is the behaviour of the correlation p(S(s), S(t)) as s > 00 with t — s fixed?
7. Let N be a non-homogeneous Poisson process on IR. with intensity function 2, Find the joint
density of the first two inter-event times, and deduce that they are not in general independent.
8. Competition lemma, Let {-(t) : 7 = 1} be acollection of independent Poisson processes on +
with respective constant intensities {2p : r= 1}, such that SD, Ar = < 00. Set N(t) =D, Nr(t),
and let / denote the index of the process supplying the first point in V, occurring at time 7. Show that
PU =i, Ten=Pu appr n= 4 Mo GS
6.14 Markov chain Monte Carlo
In applications of probability and statistics, we are frequently required to compute quantities
of the form fe g(0)7(8) d0 or yee (0) (0), where g : © > Rand x isa density or mass
function, as appropriate. When the domain is large and x is complicated, it can be beyond
the ability of modern computers to perform such a computation, and we may resort to ‘Monte
Carlo’ methods (recall Section 2.6). Such situations arise surprisingly frequently in areas as
disparate as statistical inference and physics. Monte Carlo techniques do not normally yield
exact answers, but instead give a sequence of approximations to the required quantity. 292 6.14 Markov chains
(1) Example. Bayesian inference. A prior mass function 1r(9) is postulated on the discrete
set © of possible values of 6, and data x is collected. The posterior mass function 2(0 | x) is
given by
Itis required to compute some characteristic of the posterior, of the form
E(g() | x) = 8@)x@ |x).
@
Depending on the circumstances, such a quantity can be hard to compute. This problem arises
commonly in statistical applications including the theory of image analysis, spatial statistics,
and more generally in the analysis of large structured data sets. e
(2) Example. Ising model}. We are given a finite graph G = (V, E) with vertex set V and
edge set E. Each vertex may be in either of two states, —1 or 1, and a configuration is a vector
6 = {6, : v € V} lying in the state space © = {—1, 1}. The configuration 6 is assigned the
probability
1
10) = Zen > 6.0
vw, vw
where the sum is over all pairs v, w of distinct neighbours in the graph G (the relation ~
denoting adjacency), and Z is the appropriate normalizing constant, or ‘partition function’,
Ze Leo . Ate.
For t,u € V, the chance that r and wv have the same state is,
DY 7) = Yo 30, + Dx).
@
0:
The calculation of such probabilities can be strangely difficult. e
It can be difficult to calculate the sums in such examples, even with the assistance of
ordinary Monte Carlo methods. For example, the elementary Monte Carlo method of Section
2.6 relied upon having a supply of independent random variables with mass function . In
practice, © is often large and highly structured, and 7x may have complicated form, with the
result that it may be hard to simulate directly from xr. The ‘Markov chain Monte Carlo”
(McMC) approach is to construct a Markov chain having the following properties
(a) the chain has 7 as unique stationary distribution,
(b) the transition probabilities of the chain have a simple form.
Property (b) ensures the easy simulation of the chain, and property (a) ensures that the distri-
bution thereof approaches the required distribution as time passes. Let X = {Xp :n > 0} be
such a chain, Subject to weak conditions, the Cesiro averages of g(X;) satisfy
al
5 Let > Ls).
+This famous model of ferromagnetism was proposed by Lenz, and was studied by Ising around 1924. 6.14 Markov chain Monte Carlo 293
‘The convergence is usually in mean square and almost surely (see Problem (6.15.44) and
Chapter 7), and thus the Cesaro averages provide the required approximations.
Although the methods of this chapter may be adapted to continuous spaces ©, we consider
here only the case when @ is finite. Suppose then that we are given a finite set © and a mass
function x = (xr; : i € @), termed the ‘target distribution’. Our task is to discuss how to
construct an ergodic discrete-time Markov chain X on © with transition matrix P = (pi),
having given stationary distribution x, and with the property that realizations of the X may
be readily simulated,
There is a wide choice of such Markov chains. Computation and simulation is easier
for reversible chains, and we shall therefore restrict out attention to chains whose transition
probabilities pj; satisfy the detailed balance equations
QB) Tk Pkj = Tj Pjkr JAKE O;
recall Definition (6.5.2). Producing a suitable chain X turns out to be remarkably straightfor-
ward, There are two steps in the following simple algorithm. Suppose that X, = i, and it is
required to construct X41.
(i) LetH = (hy : i, | € ©) be an arbitrary stochastic matrix, called the ‘proposal matrix’
We pick ¥ € © according to the probabilities P(Y = j | Xn =i) = hi
(i) Let A = (aij : i, j € ©) be a matrix with entries satisfying 0 < aj; <1; the ajj are
called ‘acceptance probabilities’. Given that Y = j, we set
i with probability aij,
Xntl = . 1
X, with probability 1 — aj.
How do we determine the matrices H, A? The proposal matrix H is chosen in such a way
that it is easy and cheap to simulate according to it. The acceptance matrix A is chosen in
such a way that the detailed balance equations (3) hold. Since piy is given by
hijaij itt #4,
(4) Pij=) 1- > hixaix if i = j,
kekfi
the detailed balance equations (3) will be satisfied if we choose
6) ay =1A (4)
mihiy
where x A y = min{x, y} as usual. This choice of A leads to an algorithm called the Hastings
algorithm}. It may be considered desirable to accept as many proposals as possible, and
this may be achieved as follows. Let (fj) be a symmetric matrix with non-negative entries
satisfying aj;1;; <1 for alli, j € ©, and let ai; be given by (5). Itis easy to see that one may
choose any acceptance probabilities a; given by af, = ajt;j. Such a generalization is termed
Hastings’s general algorithm.
While the above provides a general approach to McMC, further ramifications are relevant
in practice. Itis often the case in applications that the space @ is a product space. For example,
Or the Metropolis-Hastings algorithm; see Example (8). 294 6.14 Markov chains
it was the case in (2) that © = {—1, 1)” where V is the vertex set of a certain graph; in the
statistical analysis of images, one may take @ = SY where S is the set of possible states of
a given pixel and V is the set of all pixels. It is natural to exploit this product structure in
devising the required Markov chain, and this may be done as follows.
Suppose that S is a finite set of ‘local states’, that V is a finite index set, and set © = SY.
For a given target distribution x on ©, we seek to construct an approximating Markov chain
X. One way to proceed is to restrict ourselves to transitions which flip the value of the
current state at only one coordinate v € V; this is called ‘updating at v’. That is, given that
Xn =i = (iw: w € V), we decide that X,.41 takes a value in the set of all j = (jw : w € V)
such that jy, = iy whenever w # v. This may be achieved by following the above recipe in
a way specific to the choice of the index v.
How do we decide on the choice of v?- Several ways present themselves, of which the
following two are obvious examples. One way is to select v uniformly at random from V at
each step of the chain X. Another is to cycle through the elements of V is some deterministic
manner.
(6) Example. Gibbs sampler, or heat bath algorithm. As in Example (2), take © = SY
where the ‘local state space’ $ and the index set V are finite. For i = (iy : w € V) € © and
v EV, let Oxy = {i € ©: jw = iy for w 4 v}. Suppose that X, = i and that we have
decided to update at v. We take
Lil
Leee, 7"
which is to say that the proposal Y is chosen from @;,» according to the conditional distribution
given the other components iy, w # v.
We have from (5) that a; = 1 for all j € ©;y, on noting that @;,y = @j.» if j € ©».
Therefore Py(Xn41 = j | Xn =i) = hij for j € ©, where P, denotes the probability
measure associated with updating at v.
We may choose the value of v either by flipping coins or by cycling through V in some
pre-determined manner, e
M hag = JE Oi,
(8) Example. Metropolis algorithm. If the matrix H is symmetric, equation (5) gives aij =
1A (aj /x;), whence pij = hij {1 A (rj/m)} fori A j
A simple choice for the proposal probabilities h;; would be to sample the proposal ‘uni-
formly at random’ from the set of available changes. In the notation of Example (6), we might
take
{Fi JE Oi,
if j
The accuracy of McMC hinges on the rate at which the Markov chain X approaches its
stationary distribution x. In practical cases, it is notoriously difficult to decide whether or not
X,, is close to its equilibrium, although certain theoretical results are available. The choice of
distribution e of Xo is relevant, and it is worthwhile to choose o in such a way that Xo has
strictly positive probability of lying in any part of the set © where w has positive weight. One
might choose to estimate >, ¢()2(@) by n~! SM 4-! g(x,) for some large ‘mixing time’
M. We do not pursue here the determination of suitable M. 6.14 Markov chain Monte Carlo 295
This section closes with a precise mathematical statement concerning the rate of conver-
gence of the distribution aP" to the stationary distribution 7. We assume for simplicity that
X is aperiodic and irreducible. Recall from the Perron—Frobenius theorem (6.6.1) that P has
T = |@| eigenvalues 41, 22,..., Ar such that 4; = 1 and |a;| < 1 for j # 1. We write 2
for the eigenvalue with second largest modulus. It may be shown in some generality that
P" =I’ +00" [a2!"),
where I is the identity matrix, x is the column vector (x; : i € @), and m is the multiplicity
of Az. Here is a concrete result in the reversible case.
(9) Theorem. Let X be an aperiodic irreducible reversible Markov chain on the finite state
space @, with transition matrix P and stationary distribution x. Then
(10) Slew) = xe S101 aol" supller@Ol sr €O}, FEO, n>1,
ko
where v-(i) is the ith term of the rth right-eigenvector v, of P.
We note that the left side of (10) is the total variation distance (see equation (4.12.7))
between the mass functions p;.(n) and 7.
Proof. Let 7 = |@| and number the states in © as 1,2,...,7. Using the notation and
result of Exercise (6.14.1), we have that P is self-adjoint. Therefore the right eigenvec-
tors Vv), V2, ..., Vr corresponding to the eigenvalues 41, 42,..., Ar, are real. We may take
Vi, V2,... , ¥7 to be an orthonormal basis of R? with respect to the given scalar product. The
unit vector eg, having | in its kth place and 0 elsewhere, may be written
r r
an 4 = So lek, vr)¥r = Dvr Wtevr.
ml
rl
Now Pre, = (pix(n), pre(n), ---, pre(n))', and P"v, = X"'v,. We pre-multiply (11) by P”
and deduce that
r
Pit n) = > vr(k)teX vr (i),
ral
Now v; = Land 4; = 1, so that the term of the sum corresponding to r = 1 is simply zr. It
follows that
r
Do pie) = el sO ee OLY zelerW1
r = K
By the Cauchy-Schwarz. inequality,
Yoel = [Yo zelv-Gor = 1,
and (10) follows. s 296 6.15 Markov chains
Despite the theoretical appeal of such results, they are not always useful when P is large,
because of the effort required to compute the right side of (10). Itis thus important to establish
readily computed bounds for |A2|, and bounds on | pix (1) — z,|, which do not depend on the
vj. We give a representative bound without proof.
(12) Theorem. Conductance bound. We have under the assumptions of Theorem (9) that
1-20 <2) <1 $W? where
w= inf my [Tom 8 co, 0También podría gustarte
![American Gods [TV Tie-In]: A Novel](https://imgv2-2-f.scribdassets.com/img/audiobook_square_badge/626321117/198x198/22ab6b48b6/1718315036?v=1){kind=icon}
