Documentos de Académico
Documentos de Profesional
Documentos de Cultura
com
Probability:
Basic Ideas and Selected Topics
Eric B. Hall
Gary L. Wise
www.MathGeek.com
www.MathGeek.com
Preface
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
Contents
Preface 1
Acknowledgments 7
Introduction 9
Notation 11
1 Set Theory 13
1.1 Introduction 13
1.2 Unions and Intersections 14
1.3 Relations.. 19
1.4 Functions . . . . . . . 19
1.5 cr-Algebras . . . . . . . 24
1. 6 Dynkin 's 1l"- A Theorem 28
1.7 Topological Spa<:es . . 30
1.8 Caveats and Curiosities. 31
2 Measure Theory 33
2.1 Definitions.. 33
2.2 Snpremums and Infimnms . . . . . . . . . 35
2.3 Convergen<:e of Sets: Lim Inf and Lim Sup 36
2.4 Measurable Functions. . . . . . . . . . . . 38
2.5 Real Borel Sets . . . . . . . . . . . . . . . 39
2.6 Lebesgue Measure and Lebesgue Measurable Sets 43
2.7 Caveats and Curiosities. . . . . . . . . . . . . .. 46
3 Integration 47
3.1 The Riemann Integral . . . . . 47
3.2 The Riemann-Stieltjes Integral 49
3.3 The Lebesgue Integral . . . . . 51
www.MathGeek.com
www.MathGeek.com
4 Functional Analysis 59
4.1 Vector Spaces 59
4.2 Normed Linear Spaces 60
4.3 Inner Product Spaces . 62
4.4 The Radon-Nikodym Theorem 68
4.5 Caveats and Curiosities . 68
5 Probability Theory 69
5.1 Introduction. 69
5.2 Random Variables and Distributions 70
5.3 Independence 75
5.4 The Binomial Distribution 80
5.4.1 The Poisson Approximation to the Bino-
mial Distribution 82
5.5 Multivariate Distributions 83
5.6 Caratheodory Extension Theorem 86
5.7 Expectation 94
5.8 Useful Inequalities 98
5.9 Transformations of Random Variables. 102
5.10 Moment Generating and Characteristic Functions 105
5.11 The Gaussian Distribution 108
5.12 The Bivariate Gaussian Distribution 112
5.13 Multivariate Gaussian Distributions 113
5.14 Convergence of Random Variables 116
5.14.1 Pointwise Convergence 116
5.14.2 Almost Sure Convergence 116
5.14.3 Convergence in Probability. 117
5.14.4 Convergence in Lp 118
5.14.5 Convergence in Distribution 119
5.15 The Central Limit Theorem 120
5.16 Laws of Large Numbers 122
5.17 Conditioning 123
www.MathGeek.com
www.MathGeek.com
7 Problems 169
7.1 Set Theory. 169
7.2 Measnre Theory . 171
7.3 Integration Theory 172
7.4 Functional Analysis 174
7.5 Distributions & Probabilities. 174
7.6 Independence 176
7.7 Random Variables. 177
7.8 Moments. 179
7.9 Transformations of Random Variables. 181
7.10 The Gaussian Distribution 182
7.11 Convergen<:e . 183
7.12 Conditioning 185
7.13 True/False Questions 187
8 Solutions 193
8.1 Solutions to Exercises. 193
8.2 Solutions to Problems 202
8.3 Solutions to True/False Questions 238
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
Acknow ledgments
The authors would like to thank David Drumm for many help-
ful suggestions. In addition, they would like to thank Dr. Herb
\Voodson, Dr. Stephen Szygenda, Dr. Tom Edgar, Dr. Edward
Powers, Dr. Francis Bostick, and Dr. James Cogdell. Also, they
would like to acknowledge the wonderful help that GL\V received
in his recovery from a stroke, and in this regard, they mention
the supportive friendship of the preceding friends as well as that
of Dr. Michael Edmond and many dedicated therapists, includ-
ing Michelle Sanderson, Jerilyn Iliff, Janice Johnson, Audrey
Schooling, Liz Larue, and Mischa Smith. Finally, they are grate-
ful to Carey Taylor of the Texas Rehabilitation Commission for
his help in providing services for GL\V's recovery.
This book was typeset using the u\TEX typesetting system de-
veloped by Donald Knuth and Leslie Lamport.
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
Introduction
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
Notation
www.MathGeek.com
www.MathGeek.com
12
www.MathGeek.com
www.MathGeek.com
1 Set Theory
1.1 Introduction
\Ve will take a naive approach to set theory. That is, we will
assnme that any describable collection of objects is a set. Con-
sider a set A By writing x E A we will mean that x is an
element of the set A By writing x t/:. A we will mean that x is
not an element of the set A. Note that x E A and x t/:. A cannot
both be true simultaneously. To see why our approach is naive,
let R denote the set of all sets A such that A t/:. Al If R E R
then by definition it follows that R t/:. R. Similarly, if R t/:. R
then by definition it follows that R E R. Thus, although R is a
describable collection of objects, R is not a set!
This paradox was discovered by Bertrand Russell and had a
rather devastating effect on the work of a German logician
named Gottlob Frege who later wrote: "To a scientific author
hardly something worse can happen than the destruction of the
foundation of his edifice after the completion of his work. I
was placed in this position by a letter of Mr. Bertrand Russell
when the printing came to a dose." To avoid such paradoxes,
set theory is based upon systems of axioms such as the Zermelo-
F):·aenkel system. Mathematics is based upon such systems of
axioms and "mathematical truths" mnst be nnderstood in that
light. One such axiom that we will use without hesitation is
the Axiom of Choice which simply states that for any collec-
tion {Xa : a E A} of nonempty sets, there exists a function c
mapping A to UaEA Xa such that c( a) E Xa for each a E A
Although seemingly innocuous, there are many deep and dark
consequences of the Axiom of Choice.
The set with no elements is called the empty set and is denoted
by 0. Vve say that a set B is a subset of a set A and we write
B c A if x E A whenever x E B. Vve sometimes denote this by
1 It is possible for a set to be an element of itself. For example, the set
of all sets that contain more than one element is itself a set that contains
more than one element, and hence is an elelnent of itself.
www.MathGeek.com
www.MathGeek.com
14 Set Theory
www.MathGeek.com
www.MathGeek.com
nA =
iEI
{x En: x E Ai Vi E I}.
as n
as
=
2. A n0 = 0 and A U 0 = A.
2This "or" is not an exclusive or. That is, a point that is in both A and
B is also in A u B.
www.MathGeek.com
www.MathGeek.com
16 Set Theory
5. (A n B) c A and A c (A U B).
6. A c B if and only if A U B = B.
7. If A c C and B c C then (A U B) c C.
8. (A U B) n C = (A n C) U (B n C) and (A n B) U C =
(A U C) n (B U C). That is, nnions and intersedions are
distributive.
2. Au AC = n.
3. A and AC are disjoint.
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
18 Set Theory
II A)...
)..EA
In the context of this product, the set A).. is called the A-th
factor. Also, if {h En: A E A} is a point in the product
then h is called the A-th coordinate of the point. For A E A,
we will let 'iT)..: ITaEA A" ---7 A).. be the mapping that assigns a
point in IT)..EA A).. to its A-th coordinate. The map 'iT).. is called
the canonical projedion into the A-th fador or the evaluation
at A.
nA =
iEliEI
0 = n c
= (u A~) c
(uA~)C c nA.
lEliEI
www.MathGeek.com
www.MathGeek.com
Relations 19
Hence, we have
nAi = (U A~)C
iEI iEI
1.3 Relations
1.4 Functions
www.MathGeek.com
www.MathGeek.com
20 Set Theory
www.MathGeek.com
www.MathGeek.com
Functions 21
Exercise 1.6 Let S be any set with exactly two elements and
let R be any set with exactly three elements. Does there exist
a bijection from R into S? Why or why not? Does there exist a
bijection from S into R? Why or why not?
<) Example 1. 2 Let A = JR. and let B denote the set of all
functions that map A into {O, I}. Vve will show that A and B
www.MathGeek.com
www.MathGeek.com
22 Set Theory
1.2 Theorem (Dedekind) A set ,is an infinite set if and only 'if it
is eq1L'ipotent to a proper subset of 'itself.
1.2 Lemma Let T be a set having at least two distinct elements and
let I be an infinite set. The set of all functions mapping I to T
is uncountable.
1.3 Lemma ( Cantor) For any set OJ the sets 0 and lfD(O) are not
equipotent.
www.MathGeek.com
www.MathGeek.com
Functions 23
the desired result follows. Note that this lemma implies that
"the set of all sets" is not a set! D
Q- U[U {::}]
nE.:z kEN k
www.MathGeek.com
www.MathGeek.com
24 Set Theory
1.5 o--Algebras
1. DE A.
2. If A E A then Ac E A.
3. If A E A and B E A then A U B E A.
1. DE A.
2. If A E A then AC E A.
www.MathGeek.com
www.MathGeek.com
cr -Algebras 25
Exercise 1.14 Let 0 be the set of all real nnmbers and let A
be the collection of all snbsets of D that are either finite or have
finite complements. (A set with a finite complement is said to
be cofinite.) Is A an algebra on O? Is A a a-algebra on O?
Exercise 1.15 Let 0 be the set of all real numbers and let
A be the collection of all subsets of 0 that are either countable
or have conntable complements. (A set with a countable com-
plement is said to be coconntable.) Is A an algebra on D? Is A
a cr-algebra on O?
www.MathGeek.com
www.MathGeek.com
26 Set Theory
iEI
(by definition of the inverse)
{a E A : 3i E I st .I (a) E Bi }
(by definition of the union)
{a E A : 3i E I st a E .1- 1 (Bin
(by definition of .1- 1 )
{a E A: a E U f-1(B i n
iEI
(by definition of the union)
U f-1(B i ).
iEI
www.MathGeek.com
www.MathGeek.com
cr-Algebras 27
j-l (n
1EI
Bi) ; via DeMorgan's Law.
1.6 Lemma n
1EI
j-l(Bi ) = j-l (n 1EI
Bi) .
www.MathGeek.com
www.MathGeek.com
28 Set Theory
Proof. It follows from Lemma 1.4 and Lemma 1.5 that the
collection Q of all subsets A of O2 such that f-1(A) E .r1 is a cr-
algebra on O2 . Note that O2 E Q since f-1(0) = 0. (Note that
in the last equation the first empty set is o~ and the second
empty set is n1-) Further, note that A c Q. This implies
that rr(A) C Q. Since rr(A) = .r2 the desired result follows
immediately. D
1. 0 E L.
2. If A E L then AC E L.
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
30 Set Theory
Example 1.3 Consider the set JR:k for a positive integer k. For
x E JR:k and r E (0, (0), let B(x, r) denote the open Euclidean
ball in JR:k centered at x with radius r; that is,
For the usual topology on JR: k , a subset U of JR:k is open if and only
if for any x E U there exists a positive real number r snch that
B(x, r) C U. Unless noted otherwise, we will always assnme
that JR:k is equipped with its usual topology. D
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
32 Set Theory
www.MathGeek.com
www.MathGeek.com
2 Measure Theory
2.1 Definitions
2. J-L(0) = 0
www.MathGeek.com
www.MathGeek.com
34 Measure Theory
n-l
A~ An \ U Ak for n E N \ {I}.
k=l
Note that
U An = U A~
nEN nEN
and that the A~ 's are disjoint. (The collection {A;' : n E N}
is called a disjointification of the collection {An : n E N}.)
Further, since A;1 c An for each n, Theorem 2.1 implies that
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
36 Measure Theory
www.MathGeek.com
www.MathGeek.com
liminf An =
00
U n Am
k=l m=k
00
and
limsllpAn= n U Am.
00
k=l m=k
00
A. - {(-lin, 1] if n is odd
n - (-1, lin] if n is even
for positive integers n. Show that lim inf An = {O} and that
limsllpAn = (-1,1].
then JL(limsupAn) = o.
n U Ak
00 00
lim sup An =
m=l k=m
it follows that lim sup An C U~m Ak for each mEN. Thus, the
monotonicity of JL implies that
www.MathGeek.com
www.MathGeek.com
38 Measure Theory
k=m,
for each mEN. Note that since ~~=1 P,(An) < 00 it follows that
~~m p,(Ak) ---7 0 as m ---7 00; that is, the tail of the convergent
sequence vanishes. Therefore, since p,(lim sup An) is nonnegative
and must be smaller than any positive value we conclnde that
p,(lim sup An) = O. D
2.1 Lemma Consider a measure space (n, F, p,) such that p,(n) <
00. If {An}nEN is a sequence of measurable sets that converges
to some (measurable) set A then the sequence p,( An) converges
to p,(A) as 17 ---7 00.
www.MathGeek.com
www.MathGeek.com
<:; Exercise 2.5 Try to find a subset of JR that is not a real Borel
set.
www.MathGeek.com
www.MathGeek.com
40 Measure Theory
Consider a measurable space (0, :F). If f: (0, :F) ---+ (JR, B(JR))
then f is said to be a real-valued :F-measurable function defined
on 0. If.f: (JR, B(JR)) ---+ (JR, B(JR)) then f is said to be a
real-valued Borel measurable function defined on JR.
Proof. Suppose that f- 1 (U) is open for each open set U of real
numbers, and let x be an arbitrary real number. Then, given
any real number c > 0, the interval 1= (.t(x) -c, f(x) +c) is an
open set, and so f-l(1) must be open. Now, since x E f- 1 (1),
there must exist some real number 8 > 0 such that (x - 8,
x + 8) C f-l(1). But this implies that if Ix - yl < 8, then
f(y) E (f(x) - c, f(x) + c). Hence, f is continuous at x and,
since x was arbitrary, f is continuous.
Now, suppose that f : JR ---+ JR is continuous, and let U be
a nonempty open subset of R If f-l(U) is empty then the
desired result follows since the empty set is open. Assume then
that f-l(U) is not empty and let x E f-l(U). Then, since
f(x) E U there exists some c > 0 such that (f(x) - s, f(x) + c)
is a subset of U. Since f is continuous at x there exists a b > 0
such that If(x) - f(y)1 < c when Ix - YI < b. Thus, for every
y E (x - 8, x + 8), it follows that f(y) E (f(x) - c, f(x) + c)
which is an open subset of f-l(U). Thus, f-l(U) is open. D
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
42 Measure Theory
The desired result then follows since (rn' (0) and (-00, rn) is
in B(JR) for each n E N. D
www.MathGeek.com
www.MathGeek.com
For an open interval (a, b) of JR., let £( (a, b)) denote the length
of the interval (a, b). That is, if I = (a, b) with a < b then
£(1) = b - a.
Let A be a snbset of R \lVe will say that a conntable colledion
{In: n E 11{ C N} of open intervals covers A if
A C (U
nEM
In).
Let M (JR.) denote the collection of all subsets of JR. that satisfy
the Caratheodory Criterion; that is, .1\// (JR.) denotes the collec-
tion of all Lebesgue measurable subsets of R
www.MathGeek.com
www.MathGeek.com
44 Measure Theory
Proof. For each positive integer TI, let In denote the subset of
~ given by
I
n
= (x -~.
211'
x+ ~)
2n
.
www.MathGeek.com
www.MathGeek.com
°
Consider a measure space (0, :F, JL). A subset of is said to be
a nnll set (or a JL-null set) if it is measurable and has measnre
zero. That is, A is a null set if A E :F and if JL(A) = O. Let
A C B where where B is a null set. If A E :F then A must
also be a nnll set since JL(A) ::::; JL(B). In general, however, A
need not be a null set since A need not be an element of :F. A
measure space is said to be complete if every snbset of a null set
is a measurable set. Note that while the empty set is always a
null set, a null set need not be empty.
2.9 Theorem Corresponding to any measure space (0, :F, JL) there
exists a complete measure space (0, :Fa, JLo) such that
1. :F c :Fa.
2. JL(A) = JLo(A) for each set A E :F.
www.MathGeek.com
www.MathGeek.com
46 Measure Theory
For a positive integer n, let JRn denote the n-fold Cartesian prod-
uct of JR with itself. That is, an element of JRn is an ordered
n-tuple of the form (aI, ... ,an) where ai E JR for each i. A set
I of the form I = II X ... x In where h is an open interval of
the form (ak' bk) for each k is called an open rectangle in JRn.
The smallest IT-algebra on JRn that contains every open rectan-
gle in JRn is denoted by B(JRn) and is called the set of Borel
measurable subsets of JRn. Note that, for any positive integer
n, (JRn, B(JRn)) is a measurable space. If f: (JR k , B(JR k )) ---+ (JR,
B (JR)) for some kEN then f is said to be a real-valued Borel
measurable function defined on JR k .
for any sets AI, ... , Ak from B(JR) where..\ is Lebesgue measure
on (JR, B(JR)). The measure A on (JRk, B(JRk)) is called Lebesgue
measure on (JRk, B(JRk)).
www.MathGeek.com
www.MathGeek.com
3 Integration
and.
n
S:z(r) = 2]a; - O:i-l) inf{f(x) : ai-l < x ::::; ai}
i=1
f(x) = {~ if x is irrational
if x is rational.
www.MathGeek.com
www.MathGeek.com
48 Integration
www.MathGeek.com
www.MathGeek.com
v = sup{S(r) : rEg}.
If V < 00 then we say that f is of bounded variation on [a, bj.
If V = 00 then we say that f is of unbounded variation on [a,
bj.
Example 3.3 Let f(x) = I<QI(x) for x E [a, bj. Then, given
any positive number B there exists a subdivision r of [a, bj such
that S(r) > B. (Simply choose r = {(Yo, ... , an} such that 11
is large and such that ai is rational when i is even and irrational
when i is odd.) Thus, V = 00 and we conclude that f is of
unbounded variation on [a, bj. D
www.MathGeek.com
www.MathGeek.com
50 Integration
www.MathGeek.com
www.MathGeek.com
1 [a, b]
f(x) dg(x).
(N ate that this limit does not depend on 1>.) If 9 (x) = x then
I[a, b] f (x) dg (x) is simply the Riemann integral of the function f
over [a, b].
1 [a. b]
f(x) dg(x)
1 [a, b]
f(x) dg(x) = (f(b)g(b) - f(a)g(a)) -1
[a, b]
g(x) df(x).
{
J[a,b]
f dg = Ib f g' dx.
a
www.MathGeek.com
www.MathGeek.com
52 Integration
Example 3.4 Let 0 = {Head, Tail} and let .:F = lP(O). Let
J-L be a measure defined on (0, .:F) via J-L( {Head}) = 1/2 and
J-L( {Tail}) = 1/2. Let f map 0 to JR via
www.MathGeek.com
www.MathGeek.com
!
. A
j d~ = r
Jo
j 1/1 dp .
www.MathGeek.com
www.MathGeek.com
54 Integration
or -00 + 00.
as n ---7 00.
www.MathGeek.com
www.MathGeek.com
Proof. For a proof of this theorem, see page 172 of Real and
Abstract Analysis by E. Hewitt and K. Stromberg (Springer-
Verlag, New York, 196.5). D
1. f is integrable,
2. lim
n->= Jor Ifn - fl dp = 0, and,
www.MathGeek.com
www.MathGeek.com
56 Integration
k
f(x) dx.
limF(y) = F(x)
ylx
www.MathGeek.com
www.MathGeek.com
la
b
f(x)dF(x) = 1
(a,~
fdJ-L.
www.MathGeek.com
www.MathGeek.com
58 Integration
www.MathGeek.com
www.MathGeek.com
4 Functional Analysis
8. Ix = x for each x in X.
www.MathGeek.com
www.MathGeek.com
60 Functional Analysis
www.MathGeek.com
www.MathGeek.com
4. P(XI' X3) :::; P(XI' X2) + P(X2' X3) for each Xl, x2, and X3
from X.l
www.MathGeek.com
www.MathGeek.com
62 Functional Analysis
3. (x, x) ~ O.
www.MathGeek.com
www.MathGeek.com
(Xl
X - Lajxj
j=l
www.MathGeek.com
www.MathGeek.com
64 Functional Analysis
is minimized when aj = (x, Xj) faT j = 1, ... ,n. (That is, the
aj's provide the coefficients for a best linear estimator of x in
terms of the Xj's.)
slllce
(aj - (x, Xj)) 2 2 2
= aj - 2aj(x, Xj) + (x, Xj) .
Thus, we have
2
n n n
0::::; X - Lajxj = IIxI1 2 - L(x, Xj)2 + L(aj - (x, Xj))2,
j=1 j=1 j=1
www.MathGeek.com
www.MathGeek.com
Proof. Let d = inf{llx - yll : Y E !vI} and choose points Yl, Y2,
... E !vI such that Ilx - Ynll ----Jo d as n ----Jo 00. \lVe will show that
{Yn}nEN is a Cauchy sequence.
The parallelogram law states that Ilu+vl1 2+ Ilu -1)11 2 = 211ul12 +
211vl12 for all u and v in H. Let u = Yn - X and v = Ym - X to
obtain:
2 2 2
llYn + Yrn - 2xl12 + llYn - Yrnl1 = 211Yn - xl1 + 211Yrn - x11 ,
or,
2
2 2
llYn - Yrnl1 = 211Yn - xl1 + 211Ym - xl1
2
- 411~(Yn + Ym) _ xl1
Since ~(Yn + Yrn) E M (by convexity), it follows that
Thus
Since the right hand side of this expression goes to 0 as '17" m ----Jo
But,
2
Ilyo + Zo - 2xl12 = 411~(YO + zo) - xii ;:::: 4d
2
.
www.MathGeek.com
www.MathGeek.com
66 Functional Analysis
Ilx - Yo - (y - Yo)11 2
Ilx - yol12 + Ily - yol12 - 2(x - Yo, Y - Yo)
Ilx - Yol12 + Ily - Yol12 ~ Ilx - Yol12
Thus,
c211YI12 - 2(x - Yo, cy) ~ o.
Let c = b(x - Yo, y) for some b E R Then
Thus,
www.MathGeek.com
www.MathGeek.com
Note that N(f) is a vector space. That is, if 'U and 1) are in N(f)
then, since f is linear, f(u) + f(v) = f(u + v) = O. Further, if
a is a scalar and if u E fl(f) then au E N(f) since af(u) =
f (au) = O. Note also that N (f) is closed since f is a bounded,
linear, and hence continuous, map. Thus, since f i- 0 it follows
that N(f) i- H and hence, via Theorem 4.7, that J-/(f)..1 i- {O}.
Let Zo be any nonzero element of N (f)..1, and let 1) = f (x) Zo -
f(zo)x for some fixed x E H. Applying f to ea<:h side implies
that f(v) = f(x)f(zo) - f(zo)f(x) = O. That is, 'U E N(f).
Fnrther, sin<:e Zo E N(f)..1 it follows that ~/}, zo) = 0 = f(x)~zo,
zo) - f(zo)~x, zo) whkh implies that f(x)llzoI12 - f(zo)~x, zo) =
O. Solving for f(x) implies that
f(zo)
f (x) = I Zo 112 ~ x, zo)
www.MathGeek.com
www.MathGeek.com
68 Functional Analysis
v(F) = k h dJ-L
www.MathGeek.com
www.MathGeek.com
5 Probability Theory
5.1 Introduction
www.MathGeek.com
www.MathGeek.com
70 Probability Theory
1. lim F(x) =
x-----t-CXJ
o.
2. lim F(x) = 1.
X-----t(X)
4. F is right continuous.
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
72 Probability Theory
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
74 Probability Theory
1. F(x) = 1 (-00, x]
l(s) ds.
2. k1 (x) dx = 1.
www.MathGeek.com
www.MathGeek.com
Independence 75
5.3 Independence
www.MathGeek.com
www.MathGeek.com
76 Probability Theory
P(A I n A 2) =
P(A l )P(A2)
pmnVlse P(A 2 n A 3 ) =
mutual independence P(A 2)P(A 3 )
independence P(A I n A 3 ) =
P(A l )P(A 3 )
P(A I n A2 n A 3 ) =
P(A l )P(A 2)P(A 3 )
www.MathGeek.com
www.MathGeek.com
Independence 77
via independence ofthe An's (and hence of the A~'s). Note that
1 - x ::; e- X for all x E JR. and, in particular, for all x E [0, 1].
www.MathGeek.com
www.MathGeek.com
78 Probability Theory
r Cd A~;) k+j
II (1 -
n=k
P(An))
k+j
< II exp( - P(An))
n=k
---+0
n A~ ---+ n A~
n=k n=k
www.MathGeek.com
www.MathGeek.com
Independence 79
That is, lim sup En is the set of w such that wEEn for infinitely
many different values of n. Thus, our problem is to determine
a condition that is both necessary and sufficient to ensure that
P(lim sup En) = O. If this probability is zero then with proba-
bility one there will be only a finite number of errors.
By the first Borel-Cantelli lemma we know that if L:~=1 Pn < 00
then P(lim sup En) = o. Fnrther, the second Borel-Cantelli
lemma implies that if L:~=1 P(En) = 00 then P(lim sup En) =
1. That is, there will almost surely be infinitely many errors.
Thus, it is necessary that L:~=1 Pn < 00 for there to be almost
surely only a finite number of errors. Finally, we conclude that
L:~=1 Pn < 00 occurs if and only if there are almost surely a
finite number of errors. D
( 1 - _1 ) (1 - ~) (1 - ~) x ... x (1 - ~) .
365 365 365 365
Checking this numerically, we find that foro, = 23, this probabil-
ity is less than 1/2. Thus, for 23 or more people, the probability
that at least two have a common birthday exceeds 1/2. D
www.MathGeek.com
www.MathGeek.com
80 Probability Theory
(n-1)! (n-1)!
N(n, k) =
k!(n - 1 - k)! + (k - l)!(n - k)!
n! (n - k k)
k!(n - k)! -17- +;:;-
n!
k!(n - k)!·
n!
k!(17 - k)!
which is denoted by
(~)
and read as "17 choose k."
www.MathGeek.com
www.MathGeek.com
~ (~)pkqn-k.
A random variable X taking values in the set {O, 1, ... , n} for
some positive integer n such that
www.MathGeek.com
www.MathGeek.com
82 Probability Theory
..\2 - ....
Inb(O; 17, p) = 17ln 1 - -..\) = -..\ - -.
( 17 217
Thus, for large 17, it follows that b(O; 17, p) ~ e-).. Alternatively,
we could have obtained this result by recalling that for fixed A,
-..\)71 =
lim
71--'= (1 - 17
e-).
Also, for any fixed positive integer k, it follows that for suffi-
ciently large n,
b(k;n,p) "\-(k-1)p..\
b(k - 1; n, p) = k(l - p) ~ k'
From this we successively conclude that
and,
1 1 2 -).
b(2; n, p) ~ "2..\b(l; n, p) ~ "2..\ e .
www.MathGeek.com
www.MathGeek.com
Multivariate Distributions 83
Jx; (Xi) = r
Jrrt. n - 1
JX1' ... , Xn (Xl, ... , Xn) dXI ... dXi-1 dXi+1 ... dx n.
www.MathGeek.com
www.MathGeek.com
84 Probability Theory
0 if X <a
x-a
Fx(x) = if a < x ::::; b
b-a
1 if X> b.
Note that a density function for X is given by
1
fx(x) = b _ a I[a,b](x),
www.MathGeek.com
www.MathGeek.com
Multivariate Distributions 85
For what values of z and 8 will the needle intersect the line
immediately above its center? If z < 1/2 then the needle cannot
intersect the line above its center. Assume then that 1/2 :::::; z :::::;
1. In this case the needle intersects the line directly above its
<:enter if and only if 80 :::::; 8 :::::; 1T - 80 where 80 = sin-1(2(1 - z)).
Thus, the probability that the needle intersects the line above
it is given by
111 l1f-sin-l(2(1-Z))
- d8dz
1T 1/2 sin- 1 (2(1-z))
1 2lo1/2
- - - sin- 1(2y) dy
2 1T 0
"21 -:;2 [
y sin -1 ( 2y) + "21 viI - 4y2 ] 11/2
0
www.MathGeek.com
www.MathGeek.com
86 Probability Theory
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
88 Probability Theory
www.MathGeek.com
www.MathGeek.com
and
www.MathGeek.com
www.MathGeek.com
90 Probability Theory
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
92 Probability Theory
tion and let JLo((a, b]) = F(b) - F(a) for -00 ::::; a < b. Then
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
94 Probability Theory
5.7 Expectation
E[X] = kXdP
provided the integral exists. If 9 : (JR., B(JR.)) -7 (JR., B(JR.)) then
E[g(X)] = k g(X) dP.
A random variable X for which E[X] exists and is finite is said
to be integrable or to have a finite mean or to be a first order
random variable.
www.MathGeek.com
www.MathGeek.com
Expectation 95
0 if x < -2
F (x) = ~ if - 2 ::; x < 3
{
1 if x ;:::: 3.
k(X 2
+ 1) dP
L + 1) + k
(X2 dP (X2 + 1) dP
L( + + k +
4 1) dP (9 1) dP
15
5P(A) + 10P(B) = 2.
www.MathGeek.com
www.MathGeek.com
96 Probability Theory
5.3 Lemma If
G(x) = {a(3 if x ~ y
'if x < y
where /3 > a and if h : lR ----Jo lR is continuous at y then
k (x
2
+ 1) dF(x)
2
j-2-E, -2+e) (x + 1) dF(x)
2
+ j3-E,3+c) (x + 1) dF(x)
1 1 15
-(4+1)+-(9+1) =-.
2 2 2
www.MathGeek.com
www.MathGeek.com
Expectation 97
E[X] = L 2k 2- k = 00.
k=l
The paradox arises since most people would "expect" their win-
nings to be much less. The problem arises from our inability to
put in perspective the very small probabilities of winning very
large amounts. The problem returns a much more realistic value
if we assign a maximum amount that can be won; that is, if we
are allowed to "break the bank" when we reach a preassigned
level. D
www.MathGeek.com
www.MathGeek.com
98 Probability Theory
some positive time t if and only if Ti > t for each i. Thus, for
t> 0,
= 1
E[T] =
1 o
tfT(t) dt = -
nA
where we have used the fact that fo= ye-Ydy = 1. Note that
the expeded time of the first failure decreases as either 17 or A
mcreases. D
www.MathGeek.com
www.MathGeek.com
Useful Inequalities 99
j
E[IXl ] in IXl j dP
j j
r
J{IXlj<l}
IXl dP + r
l{IXlj:;"l}
IXl dP
< r
J{IXlj<l}
IdP+ r
J{IXlj:;"l}
IXlkdP
j
< P({IXl < I}) +E[IXlk] < 00.
Thus, if the kth moment is finite then all lower moments are
also finite. D
. 1
f(x) = ---:-::-
7r(1 + x 2 )
www.MathGeek.com
www.MathGeek.com
5.1 Inequality (Holder) Ifl < P < 00, 1 < q < 00, and 1+1
p q
= I,
then
P(H = h) = (N)
h
(2)h
-;
(1 - -;2)N-h
for h = 0, 1, ... , N, where we have nsed the binomial dis-
tribntion from Section 5.4 on page 80. Thns, E[Y] = 2/Tr
and VAR[Y] = ~~ (1 - ~). What value of N ensures that
IY - (2/it) I < 0.01 with probability 0.999? Chebyshev's in-
equality implies that such will be true if
www.MathGeek.com
www.MathGeek.com
5.15 Theorem If Xl, ... ,Xn are second order random variables
then
n n
VAR[XI + ... +Xn] = LVAR[Xi ] + 2 L COV[Xi' Xj].
i, j=l
i<j
www.MathGeek.com
www.MathGeek.com
P(Z:::::;z) = jjfX,y(x,y)dydx
Az
www.MathGeek.com
www.MathGeek.com
Ix (Y:b) I~I.
D
www.MathGeek.com
www.MathGeek.com
aa aa
and (3(g(x , y), h(X, y)) = y and such that -a (x, y), -a (x, y),
'x y
l6
aa (x, y), and aa(3 (x, y) each e:rist. The random vaTiables B =
x y
g(X, Y) and T = h(X, Y) possess a joint pmbability density
function given by
www.MathGeek.com
www.MathGeek.com
=
1. 1\1x(s) = L skE[Xk]jkL
k=O
www.MathGeek.com
www.MathGeek.com
NIx(s)
www.MathGeek.com
www.MathGeek.com
3. <I>x(t) = <I>x(-t).
-4. <I> x (t) is real-valued if and only if F x is symmetric; that
is, if and only if IE dFx(x) = LE dFx(x) for any real
Borel set B where - B = {-x: x E B}. (Note that a
random variable with a symmetric, absolutely continuous
probability distr"ibution function possesses an even proba-
bility density function.)
www.MathGeek.com
www.MathGeek.com
1
fx(x) = ~exp
(-(X - m)2) 2
21l"(J"2 2(}
www.MathGeek.com
www.MathGeek.com
I Distribution I Ix <Px
e't - 1
Uniform I(O,l)(X)
~t
1
Exponential e-xI(o, =) (x) --
I - ~t
1
Laplace le- 1xl --
2 1 + {2
1 1
Cauchy - e- 1tl
1[" 1 + X2
1 e -x2/2
-- e- t2 /"2
Gaussian
yI27r
www.MathGeek.com
www.MathGeek.com
Mx(t)
www.MathGeek.com
www.MathGeek.com
Note 5.4 If
x 1
cD(x) = j
. -00
;;cexp(-t 2 /2)dt
V 21f
then, for x 2 0,
www.MathGeek.com
www.MathGeek.com
x=t-
Co + C1t + C2 t "2 +;:-(p)
1+q1t+q2t2+q3t3 ~
and where
Co 2.515517
C1 0.802853
C2 0.010328
ql 1.432788
q"2 0.189269
q3 0.001308.
fx,Y(x, y) = 2
1
VI - P2 exp
(-q(X,2 y))
KO"lO""2
with q(x, y) =
www.MathGeek.com
www.MathGeek.com
where IT1 > 0, IT2 > 0, m1 E lR, m2 E lR, and Ipi < 1. Our nota-
tion for this distribution is N(m1' rn2, o"i, 0"3, p). Such random
variables X and Yare said to be jointly Gaussian or mutually
Gaussian.
5.31 Theorem Let X and Y have a N(m1' m2, O"i, 0"3, p) distribu-
tion. The random variables X and Yare independent if and
only 'if p = O. That is, mutually Gaussian random var'iables X
and Yare 'independent if and only 'if they are uncorrelated.
Proof. \Ve have already seen on page 101 that if the two random
variables are independent then they are uncorrelated. To see
that, in this case, if the random variables are uncorrelated then
they are independent, simply let p = 0 and note that fx,Y(x,
y) = fx(x)fy(y). D
1
---===----exp [ --(x
1 - m) T ~- l (x
' - m) ]
V(21l")nVdet~ 2
www.MathGeek.com
www.MathGeek.com
2.:=
Ltnl an 2 (Y nn
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
which is finite. D
www.MathGeek.com
www.MathGeek.com
P(Xn=(Y)={~1-- 1
if (Y = 1
if 0: = O.
n
01 ~ c) ~ s) = { ~ if c > 1
P(IXn - = P(Xn if 0 < c :S 1.
5.14.4 Convergence in Lp
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
Relation Reference
Xn ---7 X in Lp #- Xn ---7 X a.s. Exercise 5.10
Xn ---7 X a.s. #- Xn ---7 X in Lp Exercise 5.11
p
Xn ---7 X a.s. ::::} Xn ----? X Theorem 5.38
p
Xn ----? X #- Xn ---7 X a.s. Example 5.16
p
Xn ---7 X in Lp ::::} Xn ----? X Theorem 5.39
p
Xn ----? X #- Xn ---7 X in Lp Problem 11.3
p L
Xn ----? X ::::} X n ----? X Theorem 5.40
L P
Xn ----? X #- Xn ----? X Example 5.17
The Central Limit Theorem states that the sum of many in-
dependent random variables will be approximately Gaussian if
each term in the sum has a high probability of being small.
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
be aTbitraTily slow. For details about this result, see the article
"A Lower Bound for the Convergence Rate in the Central Limit
Theorem" by V. K Matskyavichyus in the Theory of Probability
and its Applications, 1983, Vol. 28, No.3, pp. 596-601. This
results calls into question the standard engineering claim that
the sum of a few dozen random variables is always approximately
Gaussian.
- - - - - - ----+ In.
n
Proof. We will prove only the special case when the Xn's each
have a finite positive variance (J"2. Let
1 n
X= - LXk
n k=l
www.MathGeek.com
www.MathGeek.com
Conditioning 123
---+ Tn a.s.
n
5.17 Conditioning
1. E[XIQ] is Q-measurable.
1. P(AIQ) is Q-measurable.
www.MathGeek.com
www.MathGeek.com
E[E[XIQ1lIQ2l
E[XIQ1l a.s.
www.MathGeek.com
www.MathGeek.com
Conditioning 125
Recall that if
1. Y is Qrmeasnrable, and
www.MathGeek.com
www.MathGeek.com
\Ve will now continue with our proof of Theorem 5.46. By defi-
nition, E[XIY1] is Yl-measurable. Lemma 5.5 thus implies that
E[XIY1] is also Y2-measurable. Corollary 5.6 thus implies that
E[E[XIY1]IY2] = E[XIY1] a.s. D
www.MathGeek.com
www.MathGeek.com
Conditioning 127
if and only if
E[E[XIQl]2] E[(E[E[XIQ2]IQl])2]
< E[E[E[XIQ2]2IQl]]
E[E[XIQ2]2]
www.MathGeek.com
www.MathGeek.com
Proof. First, note that E[XI11, ... , Y,,] E G since (via Jensen's
inequality) we have:
www.MathGeek.com
www.MathGeek.com
r E[XIY = y] dFy(y) = r
JB JY-l(B)
X dP
E[XIY = y] = r x fx,Y(x, y) dx
JIT€. fy(y)
r . fx,Y(x, y) d
JIT€. X fy(y) x.
www.MathGeek.com
www.MathGeek.com
(The ratio
fx.y(x, y)
fy(y)
is called a condit'lonal densdy of X given Y = y and is denoted
by fXIY(xly).)
1
-y'2K-2-n VI
1
- p2 exp
(y2) (_y2)
2 exp 2(1 _ (2)
~
-(X2 - 2PX Y))
x
JFi!.
x exp ( 21-p2)
( dx
1 1 (y2) (_y2)
y'2K v/l - p2 exp 2 exp 2(1 _ (2)
~
_(x2 _ 2pxy ± p2y2))
X lR x exp ( (2)
21-p dx
www.MathGeek.com
www.MathGeek.com
k~exp( -IYI) dy = 1.
2exp(IYI)g(y) \1
2exp y
I) = g(y).
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
Pe= 7rO r
JS I
fo(x)dx+ 7rl
JSa
r .h(x)dx.
Pe = hI .10
7ro (x) dx + 7rl ha .II(x) dx
+ hI .II
7r l (x) dx - 7rl hI .11 (x) dx
+ hI (
7rl 7rofo(x) - 7rl.h(x)) dx.
that is, we announce that the mean is one when x > 1/2. Hence,
we announce that the mean is one whenever X E (1/2, (0), and
this test minimizes the probability of error.
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
6 Random Processes
6.1 Introduction
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
V(t)X(t
E [j~ )] = {E[X(t).]E[X(t + s)] = 0 if s i= 0
+s E[X2(t)] = (}2 if s = 0
is a constant function of t. D
\Ve will next consider a calculus for second order processes. That
is, we will consider a framework in which we may discuss con-
tinuity, differentiation, and integration of second order random
processes.
lThus, the phrase "stationary Gaussian process" is not ambiguous.
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
6.12 Theorem If, in the context of OUT discussion, E[X(t)] and g(t)
aTe continuous on [a, b] and if K ,is continuous on [a, b] X [a, b]
then g(t)X(t) is L2 integmble on [a, b].
and,
E [i b
g(s)X(s) dS] = E [i b
h(t)X(t) dt] = o.
6.14 Theorem If E[X(t)] = 0, if h is continuous on [a, b], and if K
is continuous on [a, b] X [a, b] then
www.MathGeek.com
www.MathGeek.com
if j i: k
'if j = k,
2. 'if e(.) is any eigenfunction of A then e(·) zs equal to a
linear cornb'ination of the en's, and,
www.MathGeek.com
www.MathGeek.com
for sand t 'In [a, b] (where the ser'les converges absolutely and
uniformly in both variables).
X(t) = L Znen(t),
n=l
Zn = lb X(t)en(t) dt.
Notice that each term in the above series expansion for X(t) is a
product of a random part (that is, a function of w) and a deter-
ministic part (that is, a function of t). As the following theorem
shows, the Karhunen-Loeve expansion takes on a special form
when the random process is Gaussian.
www.MathGeek.com
www.MathGeek.com
L P jmP mk·
(2) -_ '""'
P jk
mEl
P(n)
k
= '""'
L
P .p(n)
J )k·
jEl
www.MathGeek.com
www.MathGeek.com
\Ve will say that a state ak can be reached from state aj if there
exists some nonnegative integer n for which p~~) is positive. A
set A of states is said to be closed if no state outside of A can
be reached from any state inside A. For an arbitrary set A of
states, the smallest closed set containing A is said to be the
closure of A. If the singleton set containing a particular state
is closed then that state is said to be an absorbing state. A
Markov chain is said to be irreducible if there exists no closed
state other than the set of all states. Note that a Markov chain
is irreducible if and only if every state can be reached from every
other state.
A state CYj is said to have period m > 1 if p)7)
= 0 unless n is a
multiple of m and if m is the largest integer with this property.
A state aj is said to be aperiodic if no such period exists. Let
ij~) denote the probability that for a Markov chain starting in
state CYj, the .first entry into state CYk occms at the nth step. Let
=
ijk = L ij~).
n=1
Note that ijj = 1 for a Markov chain that begins in state CYj
then a return to state CXj will occur at some later time with
probability one. In this case, we let
=
J-Lj = L ni?;),
71,=1
and we call J-Lj the mean recurrence time for the state aj. A state
aj is said to be persistent if ijj = 1 and is said to be transient
if ijj < 1. A persistent state CYj is said to be a null state if
J-Lj = 00. An aperiodic persistent state aj with J-Lj < 00 is said
to be ergodic.
=
6.19 Theorem A state CXj is transient if and only 'if L p~~) < 00.
71,=0
www.MathGeek.com
www.MathGeek.com
6.21 Theorem rr
the state aj ,is aperiod'ic then limn--->oo Pl;l ,is e'ither
equal to zero or to fij / Pj .
Note that the result of the previous theorem follows from the
seemingly weaker condition used to define a Markov process.
The theorem says roughly that a conditional probability of a
future event (at time u) associated with a Markov process given
the present (at time t) and the past (at times before t) is the
same as a conditional probability of that future event given
just the present. That is, for a Markov process, the past and
the present combined are no more "informative" than jnst the
present for determining the probability of some future event.
The following corollary to the previous theorem restates this
property in terms of conditional expectation.
www.MathGeek.com
www.MathGeek.com
The following theorem says that the future and the past of a
Markov process are conditionally independent given the present.
www.MathGeek.com
www.MathGeek.com
Martingales 149
6.7 Martingales
2. Xn is Fn-measurable,
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
r
6.28 Theorem If Fn F= and if Z is an integmble mndom variable
then E[Z I Fn] converges to E[Z I F=] a.s.
www.MathGeek.com
www.MathGeek.com
k h(t) dY(t).
if t < al
h(t) ~ {~ if aj-l =::; t
if t ::2: an.
< aj for 1 < j =::; n
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
where the right hand side approaches zero as t - s ----7 0 via the
previolls continuity hypothesis.
www.MathGeek.com
www.MathGeek.com
6.34 Theorem Every zero mean wide sense stationary random pro-
cess {X(t) : t E lR} satisjying the continuity condition given at
the beginn'ing oj this sect-ion possesses a spectral representation
oj the jorm
X(t) = re
Jill!.
21r1t
)" dY(A),
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
kIC(>')1 dF(>.). 2
lIT{r e
21TltA
Q(t) = IC(>') 12 dF(>.).
www.MathGeek.com
www.MathGeek.com
X(t) = re
J~
2mt
)" dY(>').
k h(s)X(t - s) ds
1(-T,T)
h(s)X(t - s) ds.
www.MathGeek.com
www.MathGeek.com
Bn(x) =
(-l)n
1:::1. exp
(X2)
-2
n
d
-p(x)
V nl 2(J dxn
where p is the univariate Gaussian density function given above.
These functions are called the orthonormalized Hermite poly-
www.MathGeek.com
www.MathGeek.com
9= L bn 8n (-),
n=O
00 roo
~ b~ = i-oo Ig(x)12 dx < CXJ
via Parseval's eqnality and since 9 is an element of L2(lR., B(lR.) ,
m).
Consider again the bivariate density function f of the random
variables X(tl) and X(t2) where, for convenience, we let X =
X(td and Y = X(t2). This bivariate density admits a senes
expansion, given via the Mehler series, as
00
www.MathGeek.com
www.MathGeek.com
xp(x)p(y)dxdy = O.
www.MathGeek.com
www.MathGeek.com
Next we show that even if the input random process were ban-
dlimitec!., the output random process is not bandlimited when 9
is not a polynomial. Observe that the support of
is given by the n-fold Minkowski sum of r2 with itself; that is, its
support is r2 EB ... EB r2 = EBnn where EB denotes the lVIinkowski
sum; that is, AEBB = {a+b : a E A, b E B}. Assume again that
the input is bandlimited. It follows that 00 > In( EBnr2) 2: n'\(r2)
where .\ is Lebesgue measure. Further, recalling the Cantor
ternary set, we see that the above inequality can be strict, and
by considering a dosed interval, we see that it can be satisfied
with equality. For the moment, we will measure bandwidth by
the Lebesgue measure of the support of the spectral density.
Thus, we see that the output is bandlimited if and only if there
exists a positive integer N such that bn = 0 for all n > N.
Notice that this condition is equivalent to the nonlinearity being
almost everywhere equal to a polynomial. However, since we are
assuming that the nonlinearity is not almost everywhere equal to
a polynomial, we see that the bn's do not truncate, and thus the
output is not bandlimited. On the other hand, if the nonlinearity
is almost everywhere equal to a polynomial then we see that the
output is bandlimited if and only if the input is bandlimited.
\Ve summarize this result with the following theorem.
www.MathGeek.com
www.MathGeek.com
Assume for now that the mean has been subtracted from the out-
put random process. Observe that this is equivalent to assuming
that bo = 0 since Bo(x) = 1 for all x and thus bo = E[g(X(t))].
Observe, also, that (J"-2nS(w)*n can be viewed as a density func-
tion of a sum of n mutually independent, identically distributed
random variables each with mean zero and variance equal to the
second moment bandwidth of the input B2 [X]. Thus it follows
that
1: w2(J"- 2n S(w)*n dw = nB;[Y]
R(T) = L b~pn(T).
n=1
Further, recall £I·om standard properties of Fourier transforms
that B2 [X] = -pl/(O). Similarly, note that
B [Y] = -RI/(O).
2 R(O)
n=1
00
11,=1
00
www.MathGeek.com
www.MathGeek.com
2. for any real numbers 0 ::; to < tl < ... < t k , the incre-
ments ~V(tk) - Vqt k- 1 ), VV(t k- 1) - ~V(tk-2)' ... , H'(t1)-
n'(to) are mutually independent,
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
which reduces to
s 11
loo te(t) dt + s e(t) dt = >.e(s); 0 :::; s :::; 1.
s
(1)
and that
:s 11 se(t) dt = 11 e(t) dt - se(s).
Thus, differentiating (1) with respect to s implies that
1 d
1
. s
e(t) dt = >.-e(s)
ds
(2)
2If a1(t, s) exists and is continuous and if o:(s) and (3(s) are difIeren-
as
tiable real-valued functions then
d 1;3(S)
----;; . 1(t, 8) dt = 1(3(8),
d3(s)
8)-'-~-- 1(0:(8),
do:(s) 1;3(S)
s)-.-+
a)(t, 8) dt.
C)
de n(8) de ds . 0«8) uS
www.MathGeek.com
www.MathGeek.com
en(s) = A sin C~ 2n
l)11"S)
where
for each n E N. D
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
7 Problems
7.1 Set Theory
1. j is one-to-one.
www.MathGeek.com
www.MathGeek.com
170 Problems
<) Problem 1.11. Show that any uncountable set of positive real
numbers includes a countable subset whose elements sum to 00.
www.MathGeek.com
www.MathGeek.com
Problem 2.1. Let f-L: JPl(N) ---7 [0, 00] via f-L(A) = 0 if A is a
finite set and f-L(A) = 00 if A is not a finite set. Is f-L a measure
on (N, JPl(N))?
<> Problem 2.3. Let D = JR2 and, for each positive integer TI, let
An be the open ball ofradius one centered at the point (( -l)n In,
0). Findliminfn ---7= An andlimsuPn---7= An.
www.MathGeek.com
www.MathGeek.com
172 Problems
www.MathGeek.com
www.MathGeek.com
10 f dJL.
r
J[a,b]
F(x) dF(x).
1: l5(t)f(t) dt = f(O).
1(a, b)
f(x) dg(x) = f(O)
when .f is continnolls at the origin and when a < 0 < b? Why
can't we simply define l5(t) to be the derivative of g(t)?
www.MathGeek.com
www.MathGeek.com
174 Problems
i E, C (b, c) = {I0
for 0 :::; b :::; 1 and 0 :::; c :::; 1
otl' len,VIse.
.
\Vhat is the probability that the roots ofthe equation x2+2Bx+
C = 0 are real?
www.MathGeek.com
www.MathGeek.com
Problem 5.6. Consider real numbers Xl, X2, Yl, and Y2 such
that Xl ::; X2 and Yl ::; Y2. Show that if F(x, y) is a joint
probability distriblltion fundion then it mllst follow that
www.MathGeek.com
www.MathGeek.com
176 Problems
7.6 Independence
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
178 Problems
for all F E F.
www.MathGeek.com
www.MathGeek.com
Moments 179
7.8 Moments
www.MathGeek.com
www.MathGeek.com
180 Problems
www.MathGeek.com
www.MathGeek.com
<) Problem 8.17. Let Xl, X2, Yl, and Y2 be real numbers such
that Xl #- X2 and Yl #- Y2. Consider random variables X and Y
defined on the same probability space such that P(X = Xl) +
P(X = X2) = 1 with P(X = xd > 0 and P(X = X2) > 0 and
such that P(Y = Yl) + P(Y = Y2) = 1 with P(Y = Yl) > 0 and
P(Y = Y2) > o. Prove or Disprove: If X and Yare uncorrelated
then X and Yare independent.
www.MathGeek.com
www.MathGeek.com
182 Problems
and let
w~ ~t,Xf
The random variable W has a density function fw that you do
not need to find. Instead, find an expression for a density func-
tion of T in terms of fw and fz where fz is a density fnndion
for Z.
www.MathGeek.com
www.MathGeek.com
Convergence 183
7.11 Convergence
www.MathGeek.com
www.MathGeek.com
184 Problems
Problem 11. 7. If you toss a fair coin 10, 000 times then what
(approximately) is the probability that you will observe exactly
5000 heads?
www.MathGeek.com
www.MathGeek.com
Conditioning 185
7.12 Conditioning
www.MathGeek.com
www.MathGeek.com
186 Problems
Show that
www.MathGeek.com
www.MathGeek.com
and
f(x, y) = 8x'y
' if 0 <
- x. < _.Y <
- 1 and 0 < - x
{
o otherwIse.
Find E[XIY] and E[YIX].
www.MathGeek.com
www.MathGeek.com
188 Problems
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
190 Problems
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
192 Problems
www.MathGeek.com
www.MathGeek.com
8 Solutions
1.1. Yes, {0} is a set containing one element and 0 is the set
containing no elements.
1.2. Since the only subset of the empty set is the empty set itself
it follows that {0} is the power set of 0.
1.3. Assume that A c B. If A U B is empty then B is empty
and hence Au B = B. Assume that Au B is not empty and let
x E A U B. By definition of union, it follows that either x E A
or x E B. If x E A then x E B since A c B. Thus, x E Band
we conclude that A U B c B. If B is empty then A is empty
and hence Au B = B. Assume that B is not empty and let
x E B. Then x E Au B which implies that B c Au B. Thus,
we conclude that A U B = B.
Assume that Au B = B. If A is empty then A c B for any set
B. Assume that A is not empty and let x E A. Then x E Au B
which implies that x E B since Au B = B. Thus, we conclude
that A c B.
1.4. The first function is not onto and not one-to-one. The
second function is onto but not one-to-one. The third function
is one-to-one but not onto. The fourth function is bijective with
inverse 1- 1 (x) = yIX.
1.5. Choose some b E B and note that since 1 is onto there
exists some a E A such that 1(a) = b. Since 1 is bijective it
follows that 1-1 ({b}) = {a}; that is, 1-1 (b) = a. Substitution
thus implies that lU- 1 (b)) = b.
Choose some a E A and let 1(a) = b. As above, note that
1-1(b) = a. Substitution thus implies that l- 1U(a)) = a.
1.6. There does not exist a bijection from R into S since no
function from R to S can be one-to-one. There does not exist
a bijection from S into R since no function from S to R can be
onto.
www.MathGeek.com
www.MathGeek.com
194 Solutions
Note that this sequence defines a bijection from the union of the
Ai'S to N. That is, the countable union is itself countable.
1.10. Yes. This is the smallest possible algebra or O"-algebra on
n.
1.11. Yes. This is the largest possible O"-algebra on n since it
contains every subset of n.
1.12. Five O"-algebras on n are {0, n}, JPl(n) , {0, n, {I}, {2,
3}}, {0, n, {2}, {I, 3}}, and {0, n, {3}, {l, 2}}.
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
196 Solutions
1.19. Note that IT({A, B}) = {O, 0, A, AC, B, BC, AUB, (AU
B)C, A U BC, B \ A, B U AC, A \ B, AC U BC, A n B, A D. B,
(A D. B)C}.
1.20. Yes, 0 E F and 0 C F.
1.21. No. Let A = lR and let A = {0, A}. Further, let f: lR---+
lR via f(x) = 3 for all x E R Then f(A) = {0, {3}} is not a
IT-alge bra on lR since lR t/:. f (A).
2.3. Assume that lim inf An is not empty and note that
= =
w E lim inf An ::::} W E U n Am
k=l m=k
::::} 3N st WEn= Am
rn=N
=
::::} W E U AmVk
m=k
www.MathGeek.com
www.MathGeek.com
: : } wEn= 00
2.4. Recall that lim inf An consists of all those points that belong
to all but perhaps a finite number of the An's. Let a be a positive
real number. Choose a positive integer N so that liN < Ct. Note
that a ¢:. An when n is an even integer greater than N. Since
there are an infinite number of such n's it follows that Ct cannot
be in lim inf An. A similar argument implies that no negative
real number is in lim inf An. Note, however, that since 0 E An
for each 11, it follows that 0 E lim inf An. Thns, we conclude that
lim inf An = {O}.
Recall that lim sup An consists of all those points that belong to
infinitely many ofthe An's. Note that any real number from the
interval (-1, 0] is in An for any even integer n, and that any
real number from the interval [0, 1] is in An for any odd integer
n. Further, any real number outside of these intervals is not in
An for any n. Thus, lim sup An = (-1,1].
2.5. Note that this exercise asked you to try to find a non-Borel
set. In particular, you could have sucessfully completed this
problem without actually finding such a set!
The purpose of this exercise is to convince the reader that con-
structing a non-Borel subset of the real line is not a trivial task.
Since the construction of such a set at this point would take us
rather far afield, a non-Borel set will not be presented here. For
many examples, see the book Counterexamples in Pmbability
and Real Analysis by Gary Wise and Eric Hall.
A proof for the e.Tistence of a non-Borel set is not quite as dif-
ficult. It follows immediately from the fact that the set of real
Borel sets is equipotent to Itt
2.6. To begin, we will show that any singleton subset of lR is a
real Borel set. Note that, for any x E lR,
n
{x} = = ( x - -, x1 + -1) .
n=l n n
Thus, since {x} is a countable intersection of bounded open
intervals it follows that {x} must be an element of 8(lR), the
smallest O"-algebra containing every bounded open interval.
www.MathGeek.com
www.MathGeek.com
198 Solutions
n=l
ifr = {ao, aI, ... , am}. Since 11(x) - I(Y)1 s Clx - YI for all
x and Y in [a, b] it follows that
Tn
www.MathGeek.com
www.MathGeek.com
Thus,
10= x f(x) dx = +00
and
1°= x f(x) dx = -00
which implies that the first moment does not exist. Note that
Thus, the odd moments do not exist and the even moments are
infinite.
5.5. The only way that a Lebesgue integral of a measurable
fundion can fail to exist is if one encounters a sum of the form
00 - 00. This cannot occur if the measurable function is non-
negative or nonpositive.
5.6. Note that
www.MathGeek.com
www.MathGeek.com
200 Solutions
fx,Y(x, y) = 2
1
J1 :2 exp
(-q(X,2 y))
1!"iT1iT:2 - P
where q(x, y) =
fx,y(x, y) = 2
1
J1 2 exp
(-q(X,2 y))
1!"iT1 iT2 - P
where q(x, y) =
II xy f(x, y) dxdy.
www.MathGeek.com
www.MathGeek.com
and that any integral over 0 is zero. Thns, E[X] satisfies the
second property in the definition of E[XI{0, r2}]. \Ve condnde
that E[XI{0, D}] = E[X] a.s. Note, however, that this eqnality
actually holds pointwise since the only null set in {0, r2} is the
empty set.
5.14. Note that
E[XY] E[E[XYIY]]
E[YE[XIY]]
E[YE[X]]
E[X]E[Y].
www.MathGeek.com
www.MathGeek.com
202 Solutions
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
204 Solutions
[(3) ::::} (1)] Let f(a) = f(b). If a i= b then {a} and {b} are
disjoint yet f( {a} )nf( {b}) is equal to {f(a)} which is not empty.
Hence f is one-to-one.
1.8. Assume that f is onto and that BeY. If b E B then
there exists some a E X such that f (a) = b and hence snch
that a E f-1(B). Thus, b = f(a) E f(f-1(B)). This and
Problem 1.6 imply that fU- 1 (B)) = B.
Next, assume that f(f-1(B)) = B for every subset B of Y.
If y E Y then f(f-1( {y})) is equal to {y} which implies that
f- 1 ({y}) is not empty. Thus, f is onto.
1.9. Assume that the set S is countable and let the sequence
{a1' a2, ... } denote the elements in S. Construct a sequence /3
of 0' sand l' s as follows: Let the n- th term in /3 be 0 if the n- th
term in an is 1 and let then-th term in /3 be 1 otherwise. Note
that }6 is an element of S yet is different from an for each n EN.
This contradiction implies that the set S is not countable.
1.10. Fix n E N and note that every polynomial p(x) = ao +
a1x + ... + anx n with integer coefficients is nniquely deter-
mined by the point (ao, a1, ... , Ll:n) from the countable set
zn+1. Thus, the set P of all such polynomials is countable and
we may list the elements of P as a sequence {P1, P2, ... }. The
fundamental theorem of algebra implies that the set Ak = {x E
lR. : Pk (x) = O} is a finite set for each k. Since a countable union
of finite sets is countable it follows that the set of all algebraic
numbers is countable.
1.11. A point x E lR. is said to be a point of condensation of a
subset E of lR. if every open interval containing x contains 11n-
countably many elements of E. To begin, we will show that any
uncountable subset E of lR. has at least one point of condensa-
tion.
Assume that there exists no condensation point of E. Then, for
each x E E there exists an open interval Ix such that x E Ix
and such that Ix n E is countable. Let J x be an open interval
such that J x C Ix, such that x E J x , and such that J x has
rational endpoints. Note that Jx n E is also countable. Further,
the collection of all such intervals Jx is countable and may be
www.MathGeek.com
www.MathGeek.com
E= U NknE
k=l
which implies that E is countable. This contradiction implies
that E must have at least one point of condensation.
Now, let E be an uncountable set of positive real numbers and
let a be a condensation point of E. If a i- 0 then let (0:, (3)
be an open interval containing a such that CI: > O. Let {Xn}nEN
be a sequen<:e of distind points in (0:, (3) n E and note that
2:~=1 Xn = 00 since Xn > CI: for each n. If a = 0 then since
(0, (3) = Uk=l 0:,
(3) it follows that some interval of the form
(~, (3) contains un<:01mtably many point of E. From this point,
we may pro<:eed as we did when a i- O.
1.12. Since n E F we see that F is closed under complementa-
tion. That is, if A E F then n \ A = Ae E F. Now, let A E F
and B E F. Then Be E F and hence A \ Be = A n B E F.
Thus, F is closed under finite intersections. De lVIorgan's Law
thus implies that F is also dosed under finite unions.
1.13. Re<:all that B(JR) is the smallest IT-algebra on JR <:ontain-
ing all bounded open intervals. Let Q be the collection of all
bounded open intervals of JR with rational endpoints and note
that Q is countable. Further, note that (J(Q) is a subset of B(JR)
sin<:e Q is a snbset of the <:olledion of all bonnded, open inter-
vals. Assume that (J(Q) is a proper subset of B(JR). Then there
must exist an open interval (x, y) that is not an element of (J(Q)
since B(JR) is the smallest (J-algebra containing all such intervals.
Let {Xn}nEN and {Yn}nEN be sequences of rational numbers such
that Xn 1 x and Yn 1 Y with Xn < Yn for each n E N. Note that
since (x, y) = U~=l(xn, Yn) it follows that (x, y) E (J(Q). This
contradiction implies that IT(Q) = B(JR), and thus we see that
B(JR) is count ably generated.
1.14. Consider the IT-algebra F given by the countable and
cocountable subsets of R Note that F contains every singleton
subset of R Assume that there exists a (J-algebra Q such that Q
contains every singleton subset of JR and su<:h that Q is a proper
subset of F. Let F E F with F tj. Q. Note that F can be written
www.MathGeek.com
www.MathGeek.com
206 Solutions
{~n
if x tj C
1(x) = if x = C2n
Cn if x = C2n-l.
Note that 1 is onto and one-to-one. Thus, we conclude that A
and A \ B are equipotent.
1.16. Let 1 : (0, 1] ----7 [0, (0) via 1 (x) = (1 - x) / x . Let y E [0,
(0) and note that 1(1/(1 +y)) = y. Thus, since 1/(1 +y) E (0,
1], we see that 1 is onto. Next, let a, b E (0, 1] with ai-b.
Since (1 - a)/a i- (1 - b)/b we see that 1 is one-to-one. Thus,
we conclude that (0, 1] and [0, (0) are equipotent.
1.17. No. Assume that M is a countably infinite IT-algebra on
a nonempty set 0 and, for each w E 0, let
A",= n
{MEM:"'EM}
All.
www.MathGeek.com
www.MathGeek.com
(-1)k)2 2
( X - -k-,- +y <1
(-I.)k) E U.
( x--k-,y
Since
(_I)k)2
( X - -k-'- + y2 <1
for all k 2 TI, and hence that 2x(-I)k 2 l/k for all k 2 TI.
This last resnlt, however, cannot be trne since the left hand side
alternates sign (or is zero) and the right hand side is always
positive. Thus we conclude that x 2 + y2 must be less than 1.
Hence (x, y) E U and thus liminf An C U.
www.MathGeek.com
www.MathGeek.com
208 Solutions
( _I)k) 2 2x(-I)k 1
x - - - +' 2 = x 2 _ +_ +' 2
( k Y k k2 Y
<
2 21xl 1 2 2
x +-+-+y < x +-+-+y = x +y +- = l-E+-
2 1 2 2 2 3 3
- k k2 k k· . k k
since (x, y) E U and kEN. Thus, we see that
(_I)k)2
(X - -k-'- + y2 < 1
(-1)k)2
(X - -k-'- + y2 < 1
which is true if and only if
2x(-I)k 1
E < k k2·
Since (x, y) E lim sup An we know that (x, y) E Uk=nAk for all
n E N. That is, for all n E N there exists some kEN such that
k 2n and such that (x, y) E A k. Note that (x, y) E Ak if and
only if
2x 1
E < k - k2
www.MathGeek.com
www.MathGeek.com
that n > 2 (x + 1) / t: and let k be any even integer not less than
n. Since (x, y) E lim sup An we see that there exists an (even)
integer rn such that rn 2: n and such that
2x 1
C"<---
~ 'm ln 2 ·
(_I)k)2 + y2 = (-(_I)k)2
- - + (±1)2 = -
1
+1> 1
(x - --
k k k2
(-It)2 :2 :2 :2
<x +y ::::;1.
(x--'-k- +y
Thus, for any n E N we can find some kEN such that k > n
and such that (x, y) E Ak if x < o. Hence, (x, y) E U~nAk
for all n E N if x < o. A similar argument shows that (x,
y) E Uk=n Ak for all 17 E N if x > o. Finally, if (0, y) E S
then (0, y) E U = lim inf An. Since lim inf An C lim sup An
we thus see that (0, y) E lim sup An- Hence, we conclude that
S C lim sup An. Combined with our earlier result we see that
lim sup An = S.
www.MathGeek.com
www.MathGeek.com
210 Solutions
www.MathGeek.com
www.MathGeek.com
r
~1}
f dJ-L +
~2}
r
f dJ-L +
J{3}
r
f dJ-L +
~4}
f dJ-Lr
f(l)f1({l}) + f(2)J-L( {2})
+f(3)f1({3}) + f(4)f1({4})
1 X 1 +4 X 1+9 X 1 + 16 X 1
30.
www.MathGeek.com
www.MathGeek.com
212 Solutions
Thus,
c E[X - (X - c)]
E[X] - E[X - c]
10= (1 - F(t)) dt - iOco F(t) dt
O
-loco (1 - F(t + c)) dt + i = F(t + c) dt
10= (F(t + c) - F(t)) dt + iOco (F(t + c) - F(t)) dt
www.MathGeek.com
www.MathGeek.com
5.3. Note first that Fj(l : (0, 1) ---+ lR exists and is strictly
increasing. Thus, if Z = Fx(X) then it follows that Fz(z) =
P(Z ::::; z) = P(Fx(X) ::::; z) = P(X ::::; Fj(I(Z)) = Fx(Fj(I(Z)) =
z for 0 < z < 1. Thus, Z is nniform on (0, 1).
5.4. Let Y = -In(F(X)) and, as above, note that Fy(y) =
P(Y ::::; y) = P( -In(F(X)) ::::; y) = P(ln(F(X)) :2:: -y) =
P(F(X) :2:: exp( -y)) = P(X :2:: F-I(exp( -y))) = 1 - P(X ::::;
F-l(exp( -y))) = 1 - F(F-I(exp( -y))) = 1 - exp( -y) for y :2::
o where the sixth equality follows from the continuity of the
indicated distribution function. Thus, .!y (y) = exp( -y) for y :2::
o and is zero for y < o.
5.5. Note that Fz(z) = P(Z ::::; z) = P(X ::::; z, Y ::::; z) =
Fx,Y(z, z). Also, note that Fw(w) = P(W ::::; w) = 1 - P(W >
w) = 1-P(X > w, Y > w) = P(X::::; w)+P(Y::::; w)-P(X::::;
w, Y ::::; w) = Fx(w) + Fy(w) - Fx,Y(w, w).
5.6. Assume that X and Y have a joint probability distribution
function given by F. Note that P(XI < X ::::; X2, YI < Y ::::;
Y2) = P(X ::::; X2, Y ::::; Y2) - P(X ::::; Xl, Y ::::; Y2) - P(X ::::; X2,
Y ::::; yd + P(X ::::; Xl, Y ::::; YI) = F(X2' Y2) - F(Xl' Y2) - F(X2'
www.MathGeek.com
www.MathGeek.com
214 Solutions
.fx(X) r ~Ic(x,y)dy
JIR 1["
Y l-X 2 1
j -yl-x2
-dy
1["
~Jl- X2
1["
www.MathGeek.com
www.MathGeek.com
and since
www.MathGeek.com
www.MathGeek.com
216 Solutions
then follows that X-l(A) = VAu (-VA) and hence that IT(X)
consists of all sets of this form. But, any real Borel set B C [0,
00) may be written as VC for the set C = {x 2 : x E B}. Thus,
O"(X) consists of all sets of the form B U - B where B C [0, 00)
is a real Borel set.
7.3. Assume that X is not equal to Y a.s. Then there must exist
a set of positive probability on which X - Y is not zero. Hence,
there exists a set of positive probability on which (X - y)2
is positive. But, if (X - y)2 is positive on a set of positive
probability then E[(X - y)2] cannot be equal to zero. This
contradiction implies that X must equal Y a.s.
7.4. Note that E[X] = 0 and that
VAR[X] = ~ ( x 2 e- 1xl dx = 2.
2 JIT€.
Thus, Chebyshev implies that P(IXI > 2) ::; 1/2. Note, how-
ever, that
P(IXI > 2) = 1 - -1
1
2.
2
-2
e- 1xl dx
.
= e- 2 ~ 0.135.
www.MathGeek.com
www.MathGeek.com
~
2 2
E[IX-Yll r r Ix - yl dx dy
4 Jo Jo
-41 Jor2 Jry2(x-y)dxdy+-41 Jor2 Jor (y-x)dxdyy
4lo
1 2 ( 1
2 - 2y - "2y2 + y2 ) dy
1 10
2(2 2) dy
+-4.0 Y - 1
-y
2
1 .,
4lo~ (y2 - 2y + 2) dy
www.MathGeek.com
www.MathGeek.com
218 Solutions
2
3
(~)n
for 0 < z :s; B. Thus, it follows that
www.MathGeek.com
www.MathGeek.com
1 1
- cos(l) + - cos(2s + 1)E[cos(2X)]
2 2
1
-- sin(2s + 1)E[sin(2X)]
2
1
"2 cos(l).
oo
la P(X > t) dt.
8.2. Since E[(X - m)2] = E[X2] - 2mE[X] + m 2 it follows that
d
dm E[(X - m)2] = -2E[X] + 2m. Setting this latter expression
equal to zero implies that m = E[X] is a critical point. Since
~ . .
dm2E[(X - m)2] = 2 > 0 it follows that m minimizes E[(X-
m)2J.
8.3. Apply the Cauchy-Schwarz inequality to the product XY
to see that
IE[XY]I :::; E[IXYI]:::; VE[X2]/E[Y2].
www.MathGeek.com
www.MathGeek.com
220 Solutions
E[Y] =
1
-b-
- a
lb
a
ydy
a
=--
+b
2
and that
VAR[Y] = -1-
b- a
lb a
y2 dy - (a-+2-b)2 (b - a)2
12
www.MathGeek.com
www.MathGeek.com
e- lt ( e21~z~ 1 )
e,t _ e- 1t
2d
sin( t)
t
Finally, if Sn = Xl + ... + Xn then
www.MathGeek.com
www.MathGeek.com
222 Solutions
1/
"2.
00
-00 (1 + (;os(2tx))f(x) dx
1 1
"2 + "2<I>(2t)
from which the desired result follows immediately.
For an alternate solution (that does not require X to possess a
density fundion), note that
<I>(t) E[(;os(tX)]
E [2 cos
2
C;) - 1]
2E [cos
2
C;)] - 1.
Thus, <I>(2t) = 2E[cos 2 (tX)]-1. Jensen's inequality implies that
E[cos 2 (tX)] ;::: E[cos(tX)J2. Thus, <I>(2t) ;::: 2E[cos(tX)J2 - 1 =
2<I>2(t) - 1 from which the desired result again follows immedi-
ately.
8.14. Note that X is equal to 0, 1, and 2 with probabilities 1/4,
1/2, and 1/4, respectively. Thus,
1'vlx(s) E[e SX ]
2
'LeskP(X=k)
k=O
ens P(X = 0) + e1s P(X = 1) + e2s P(X = 2)
1 1 s 1 2s
_ + _e + _e
4 2 4
~(1 + e )2. s
4
www.MathGeek.com
www.MathGeek.com
Note that
1 s
lvI I (s) = _e 1 2
+ -e S
x 2 2'
that
for n E N.
8.15. To begin, note that
lvIx(s) = E[e SX
] = ~(1 + e + e2s ).
S
www.MathGeek.com
www.MathGeek.com
224 Solutions
0 wp 1/9
1 wp 2/9
X+Z= 2 wp 1/3
3 wp 2/9
4 wp 1/9.
Hence,
~1~
IVjX+Z
()
s = -1 2 8 + -c
+ -c 1 28 + -e
2 3s + -c
1 48 .
9 9 3 9 9
Note also that
and,
27r 1
E[Y] = ( - sin(e) de = o.
21r Jo
X and Yare not independent since, however, Sll1ce P(X E
[1/V2, 1], Y E [1/V2, 1]) = 0 =I P(X E [1/V2, I])P(Y E
[1/V2, 1]).
8.17. Consider uncorrelated random variables X and Y with
joint probability distribution function P(X = Xi, Y = Yj) = Pij
for i, j = 1, 2 and marginal probability distributions P(X =
Xi) = Pi fori = 1, 2 and P(Y = Yj) = qj for j = 1, 2. Note
that Pll + P12 + P21 + P22 = 1, Pi1 + Pi2 = Pi for i = 1, 2,
P1j + P2j = qj for j = 1, 2, P1 + P2 = 1, and q1 + q2 = 1.
] = 2:: i =l 2:: j =l XiYjPij, EX = 2:: i =l XiPi, and
2 2. [] 2
Note that E [ XY
E[Y] = 2::;=1 Yjqj. Since X and Yare uncorrelated it follows
www.MathGeek.com
www.MathGeek.com
1 ( X ) 1
fx(x) = 21/R 211" = 211"(b - a)
fy(y) = 2y11rYfR
1 (
VfY)
-; 1
= 2y11rY b -
1
a
~
2
_1_ (= yexp (_y 1 dy
lyl VI - (Z2/ y 2)
)
11"0"2 llzl . 20"2 .
(since Iz/yl < 1 and Y > 0)
~2
1 exp
11" (J"
(_Z2) 10= exp
~-2
2(J". 0
(-t 2)
~2
2(J"
x t 1 dt
VI -
2 2
(Z2/(t + Z2)) Jt + Z2
(where we let y2 = t 2 + Z2)
= 1
~')
11" O"~
exp ~-2 10= exp
(_Z2)
20" 0
(-t 2)
-')
20"~
dt
www.MathGeek.com
www.MathGeek.com
226 Solutions
1
-exp - 2
(_Z2) -V21ra
1 2
1ra 2 2a 2
_ 1 exp (_Z2)
rrV'iii 2a 2
for z E JR. That is, Z is N(O, ( 2 ).
9.3. Let g(x, y) = x/(x+y) and let h(x, y) = x+y. Note that if
a(b, t) = bt then a(g(x, y), h(x, y)) = a(x/(x + y), x + y) = x,
and if /3(b, t) = t(l- b) then (3(g(x, y), h(x, y)) = (3(x/(x + y),
x + y) = y. Let B = X/(X + Y) and T = X + Y. Then
e-bte-Hbtltl
te- t
for t > 0 and 0 < b < 1. Thus,
www.MathGeek.com
www.MathGeek.com
Thus, a = 1.28.
10.2. Recall the moment generating function lVIx and note that
www.MathGeek.com
www.MathGeek.com
228 Solutions
( 22) (22)
. ~t ~t
Alx(t) = exp -2- + tm1 and },fy(t) = exp -2- + tm2 .
Thus,
. (0"1+0"2
. 2 2) t 2 )
lVlx+y(t) = exp ( 2 + t(m1 + m2)
which implies that X + Y is N(m1 + m2, O"r + O"i).
10.7. Note that
k (>.f1 +
etx (x) (1 - )..)i2(x)) dx
). k f1(X) +
etx dx (1 -)..) ketx i2(x) dx
www.MathGeek.com
www.MathGeek.com
E[X] lvI~(O)
Further,
E[X2] M~(O)
[AO"i lvfl (t) + A(O"it + ml)21\1l (t)
+(1 - A)0"~lvf2(t) + (1 - A)(O"~t + m2)21\12(t)]t=o
AlTl2 + Am 2 l+( 1 -)A21T2 + (1 - A)m 2
2.
Thus,
VAR[X]
E[Z] i: zfz(z) dz
i: 2zj(z)F(z) dz
2 roo z_1_e- z2 / 2 rz _1_e- t2 / 2 dt dz
L= v'2i L= v'2i
~ roo t ze-z2/2e-t2/2dtdz
i-= i-=
j= 1= e- t2/.2ze -z/2dzdt
7r
_1 2 '.
7r -= t
11= 1=
_
7r -=. t
ze-(Z 2 +t 2 )/2 dz dt
11= h=
-
7r -=. 2t2
e-"2w -1 dwdt
2
www.MathGeek.com
www.MathGeek.com
230 Solutions
11(Xl e dt
- _t 2
1["-(Xl
~ 1(Xl e- 1_ dt =
t2 / 2 _ _1_.
1["-(Xl J2 yIK
11.1. Note that the Xn's are mutually independent and that
2:~=1 P(Xn = 1) = 2:~=1 ~ = 00. Thus, the second Borel-
Cantelli Lemma implies that P(lim sup{ Xn = I}) = 1. That is,
there exists a null set A such that for any w E AC, Xn(w) = 1
for infinitely many values of n. Hence, Xn does not converge to
zero for any w E AC. Since P(AC) = 1 it follows that the Xn's
cannot converge to zero almost snrely.
11.2. Since Cx ; (t) = e- 1tl for eachi and since the Xi's are
mutually independent, it follows that
www.MathGeek.com
www.MathGeek.com
P(Zn S z)
P(n(l - max{Xl' ... , Xn}) S z)
P (1 - max{Xl , ... , Xn} S ~)
17,
(1 - -nz)n ----Jo e- z
as n ----Jo 00. Thus, FZn (z) ----Jo 1 - e- Z for 0 < z < 00 as n ----Jo 00.
p(IZI < _2 )
0.91
P(IZI < 2.19)
2<I>(2.19) - 1 = 0.97.
www.MathGeek.com
www.MathGeek.com
232 Solutions
~ (1~0) - ~ (~~)
2 (~(1~0) - ~(O))
2~ (_1
100
) -1.
Note that ~(0.01) = 0.5040. Hence, P(X = 5000) ;::::j 2(0.5040)-
1 = 0.008.
11.8. Note that
E[X~l - 20E[Xnl + 0
2
12.1. Note that X and Yare mutually Gaussian and that X and
Yare uncorrelated since E[XYl = E[(U + V)(U - V)] = E[U 2 ] -
E[V2l = O. Thus, X and Yare independent. Problem 10.6
implies that X and Yare each N(O, 2).
www.MathGeek.com
www.MathGeek.com
L E[XIY](w)P({w}) = L X(w)P({w}),
where JlvI E (j(y) = {0, f2, {a}, {b, e}}. Snbstitntion for !vI
implies that E[XIY](a) = X(a) and E[XIY](b) + E[XIY](e) =
X(b) + X(e). Since E[XIY] is (j(Y)-measnrable it follows that
E[XIY](b) = E[XIY](e). Thus, E[XIY](a) = E[XIY](b) =
E[XIY] (c) = 1 and we see that E[XIY] = E[X] = 1/3 + 2/3 = 1
as required. However, X and Yare not independent since
P(X = 1, Y = 1) = 1/3 yet P(X = l)P(Y = 1) = 1/9.
12.3. Note that E[ZIX] = E[XYIX] = XE[YIX] = XE[Y] =
o a.s. Similarly, E[ZIY] = 0 a.s. However, Z is O"(X, Y)-
measurable and hence E[ZIX, Y] = Z a.s.
12.4. Since E[XIF] is F-measurable it follows that E[XIF] =
alIA + a2IB + a3Ic for some real constants aI, a2, and a:3. Recall
that E[XIF] must satisfy
£E[XIF]d)'= £Xd),
1 rl / 4 2 1
al = )'(A) Jo w dw = 48·
www.MathGeek.com
www.MathGeek.com
234 Solutions
LE[XZIY]dP LXZdP
L IAXZdP
E[IAXZ]
E[X]E[IAZ]
E[X] LZdP
E [( n+1
{;Yk) 2 I ]
-(n+1)0"2Fn
where in the second to the last step we used our first result and
noticed that Y;+ 1 is independent of Xi fori ::; TI.
12.7. Note that
www.MathGeek.com
www.MathGeek.com
E[(X - g(y))2]
E[(X - E[XIY] + E[XIY] - g(y))2]
= E[(X - E[Xly])2] + 2E[(X - E[XIY])(E[XIY]- g(Y))]
+E[(E[XIY]- g(y))2].
The first res nIt now follows since
JAr
E[XIQ] dP = al dP =
JAr JAr
X dP
www.MathGeek.com
www.MathGeek.com
236 Solutions
r XdP
E[XIQ] = _JO-,,-i-,------.,._
P(D;)
for all w E Di .
12.11. To begin, note that
where
E[XY] E[E[YIQ2]Y]
E[E[E[YIQ2]YIQ2]]
E[E[YIQ2]2]
E[X2]
and
E[XY] E[XE[XIQl]]
E[E[XE[XIQl]IQl]]
E[E[XIQl]2]
E[y2].
E[XY] E[E[XYIX]]
E[XE[YIX]]
E[X(a + leX)]
aE[X] + pE[X2]
aE[X] + pVAR[X] + pE[X]2.
Substituting here implies that -3 = 3a + 11p. Solving these
two eqnations yields a = 53/2 and p = -15/2.
www.MathGeek.com
www.MathGeek.com
.fy(y) k f(x, y) dx
l
J
r 8xy dx
y
4y(1 _ y:.l)
www.MathGeek.com
www.MathGeek.com
238 Solutions
lax 8xydy
4x 3
for 0 S x S 1. Thus,
E[XIY = y] r xf(x, y) dx
JJR fy(y)
2 2
1- y Jy
r 1
X
2
dx
21 - y:3
31- y2
21 + y + y2
3 l+y
for 0 S y S 1. Further,
E[YIX = x]
r f(x, y) d
JJR Y fx(x) y
2
2"
lox,Y 2 d
Y
x 0
2
-x
3
for 0 S x S 1.
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
www.MathGeek.com
Index
www.MathGeek.com
www.MathGeek.com
INDEX 241
www.MathGeek.com
www.MathGeek.com
242 INDEX
www.MathGeek.com
www.MathGeek.com
INDEX 243
www.MathGeek.com
www.MathGeek.com
244 INDEX
www.MathGeek.com
www.MathGeek.com
INDEX 245
46
]R.n, set difference, 16
Radon-Nikodym theorem, erA, 216
68 er-algebra, 24
random process, 13.5 countably generated, 41
random sequence, 135 generated, 25, 76
random variable, 70 IT-field, 24
range, 20 er-finite measure, 33
rank, 114 IT-subalgebra, 41
rational line, 174 simple function, 51
real vector space, 59 singleton set, 14
reflexive relation, 19 singular distribution, 72
regression function, 129 size of a subdivision, 50
relations, 19 spectral density function, 155
relative <:omplement, 16 spedral distribution fun<:-
Riemann integral, 47, 51, 55, tion, 155
143 spedral representation, 155
Riemann-Stieltjes integral, spedrum, 155
51,96 square law device, 159
Riemann-Stieltjes sum, 50 St. Petersburg paradox, 97
Riesz-Frechet theorem, 67 standard deviation, 98
right mntinuous fundion, standard Gaussian distribu-
56,70 tion, 110, 111
Russell, Bertrand, 13 standard Gaussian random
variable, 110
sample fnndion, 135 state, 145
sample path, 135 absorbing, 146
sample sequence, 135 aperiodic, 146
sample space, 25 dosed set, 146
Schroeder-Bernstein theo- ergodi<:, 146
rem, 22 rnean re<:nrren<:e tinle,
second order random pro- 146
cess, 139 null, 146
second order random van- period, 146
able, 98 persistent, 146
separable modification, 136, reachable, 146
137 transient, 146
separable random process, stationary Gaussian process,
136 140
sequence of random van- statistical
ables, 116 hypothesis, 132
www.MathGeek.com
www.MathGeek.com
246 INDEX
unbounded set, 36
unbounded variation, 49
uncorrelated random van-
ables, 101
uncountable set, 21
www.MathGeek.com