ECE 534 RANDOM PROCESSES FALL 2011

SOLUTIONS TO PROBLEM SET 2

1 On the sum of a sequence of products
(a) Using (ii), then (iii), and then (iv) yields for xed c > 0,
a
n

n

k=1
exp
_

c
k
_
1
c
k
__
exp
_

k=1
c
k
+

k=1
c
2
k
2
_
exp
_
c lnn +
c
2

2
6
_
= A
c
n
c
0 as n
where A
c
= exp
_
c
2

2
6
_
.
(b) On one hand, by the upper bound on a
n
found in part (a), for c > 1,

n=1
a
n
A
c

n=1
n
c
A
c
_
1 +
_

1
u
c
du
_
=
cA
c
c 1
< +
On the other hand, if 0 < c 1, by fact (i),
_
1 +
c
k
_
1
exp
_

c
k
_
for any k 1, so, applying fact
(iv), a
n
exp
_

n
k=1
c
k
_
exp(c(1 + lnn)) = (ne)
c
. Therefore, for 0 < c 1,

n=1
a
n

n=1
(ne)
c

_

1
(ue)
c
du = +
2 Convergence of alternating series
(a) For n 0, let I
n
denote the interval with endpoints s
n
and s
n+1
. It suces to show that
I
0
I
1
I
2
. If n is even, then I
n
= [s
n+1
, s
n+1
+ b
n+1
] [s
n+1
, s
n+1
+ b
n+2
] = I
n+1
.
Similarly, if n is odd, I
n
= [s
n+1
b
n+1
, s
n+1
] [s
n+1
b
n+2
, s
n+1
] = I
n+1
. So in general, for any
n, I
n
I
n+1
.
(b) Given > 0, let N

be so large that b
N
< . It remains to prove that |s
n
s
m
| whenever
n N

and m N

. Without loss of generality, we can assume that n m. Since I

m
I
n
it
follows that s
m
I
n
and therefore that |s
m
s
n
| b
n+1
.
3 A Gaussian sequence
We begin by considering convergence in distribution. By induction on k, X
k
is a Gaussian random
variable with mean zero for all k. The variance of X
k
, denoted by
2
k
, is determined by the following
recursion:
2
0
= 0 and
2
k+1
=

2
k
+
2
4
. This can be solved to get
2
k
=
2
(
1
4
+
1
4
2
+ +
1
4
k
) so by
the formula for the sum of a geometric series, lim
k

2
k
=
2

=

2
3
. The CDF of X
k
is given by
F
k
(c) = P
_
X
k

k
_
=
_
c

k
_
, where is the standard normal CDF. Since is a continuous
1
function, it follows that F
k
converges pointwise to the CDF F

(c) = (
c

), so that (X
k
) converges
in distribution with the limit having the N(0,
2

) distribution.
The sequence does not converge in p. Let > 0. Consider P{|D
k
| } where D
k
= X
k+1
X
k
.
By the recursion, D
k
=
X
k
+W
k
2
X
k
=
W
k
X
k
2
. D
k
is a Gaussian random variable and Var(D
k
) =

2
+
2
k
4


2
4
. Therefore, P{|D
k
| } = P{
2|D
k
|

} 2Q(
2

) > 0. So P{|D
k
| } 0 as
k so P{|X
n
X
m
| } 0 as m, n . That is, (X
n
) is not a Cauchy sequence in
probability, and hence does not converge in probability. The sequence thus also does not converge
in the a.s. or m.s. sense.
4 A cumulative lottery
(a) For n xed, the possible values of Y
n
are {1, 2, 4, . . . , 2
n
}.
P{Y
n
= 2
n
} = P{B
1
= = B
n
= 0} = (1 p)
n
P{Y
n
= 2
k
} = P{B
nk
= 1 and B
nk+1
= = B
n
= 0} = p(1 p)
k
for 0 k n 1
(b) Let 1 c < 2
n
. The largest integer k such that 2
k
c is k = log
2
c. Therefore, F
n
(c) =
p +p(1 p) +p(1 p)
2
+ +p(1 p)
log
2
c
or
F
n
(c) =
_
_
_
0 c < 1
1 (1 p)
log
2
c+1
1 c 2
n
1 c 2
n
Therefore, F
n
(c) F(c) as n for all c, (i.e. F
n
converges pointwise to F) where
F(c) =
_
0 c < 1
1 (1 p)
log
2
c+1
c 1
That is, if Y has the limit CDF F, then Y has the representation Y = 2
R1
, where R is a geo-
metrically distributed random variable with parameter p, and represents the number of days since
there was a winner.
(c) No, (Y
n
) does not converge in p. because, for example, it is not Cauchy in probability because
P{|Y
n
Y
n1
| 1} P{B
n
= 0} = 1 p for all n 1.
5 Convergence and robustness of the sample median
(a) We show that Y
n
a.s.
c

. It suces to prove that for any c

0
and c
1
with c
0
< c

< c
1
,
P{Y
n
c
1
for all n suciently large} = 1 (1)
P{Y
n
c
0
for all n suciently large} = 1 (2)
Since c

is the unique solution to F(c

) = 0.5, it follows that F(c

1
) > 0.5. By the strong law of
large numbers,
I
{X
1
c
1
}
+ +I
{X
2n+1
c
1
}
2n + 1
a.s.
F
X
(c
1
)
In words, it means that the fraction of the variables X
1
, . . . , X
2n+1
that are less than or equal to
c
1
converges to F
X
(c
1
). Since F
X
(c
1
) > 0.5, it follows that
P
_
I
{X
1
c
1
}
+ +I
{X
2n+1
c
1
}
2n + 1
> 0.5 for all n large enough
_
= 1,
2
which, in turn, implies (1). The proof of (2) is similar, and omitted.
(b) The event {|Y
n
| > c} is a subset of the union of the events {|X
i
| c for all i A} over all
A {1, , 2n+1} with |A| = n+1. There are less than
_
2n+1
n+1
_
such subsets A of {1, , 2n+1},
and for any one of them, P{|X
i
| c for all i A} = P{|X
1
| c}
n+1
. Now
_
2n+1
n+1
_
2
2n+1
because
the number of subsets of {1, . . . , 2n+1} of cardinality n+1 is less than or equal to the total number
of subsets. Thus (b) follows by the union bound.
(c) Note that for c > 0, P{|X
1
| c} = 2
_

c
1
(1+u
2
)
du
2

c
1
u
2
du =
2
c
. By the result of part
(b) with n = 1, P{|Y
1
| c} 8
_
2
c
_
2
=
32
(c)
2
. Thus,
E[|Y
1
|] =
_

0
P{|Y
1
| c}dc 1 +
_

1
P{|Y
1
| c}dc 1 +
_

1
32
(c)
2
dc 1 +
32

2
.
6 Some tilted distributions associated with large deviations
(a) For and xed, the function f

(x) is proportional to e
x
e
x
= e
()x
as a function of x
for x 0. That is possible if and only if < < , and for such , f

is the exponential pdf

with parameter .
(b) For , and
2
xed, the function f

(x) is proportional to
exp
_

(x )
2

2
+x
_
which is proportional to
exp
_

x
2
2( +
2
)x
2
2
_
.
Such a pdf exists for any R, and it is the N( +
2
,
2
) density.
(c) For and p xed, the pmf p

(k) is proportional to (1 p)
k
e
k
(same as ((1 p)e

)
k
) for k 1,
which gives a valid pmf if (1 p)e

< 1, or equivalently, if < < ln(1 p). For such , the

tilted pmf is the pmf of the geometric distribution with parameter p = 1 (1 p)e

.
(d) For , n and p xed, the pmf p

(k) is proportional to
_
n
k
_
_
pe

1p
_
k
whereas a binomial distribution
with parameters n and p is proportional to
_
n
k
_
_
p
1 p
_
k
. The pmf p

is thus the pmf of the binomial

distribution with parameters n and p, where
p
1 p
=
pe

1 p
,
or, equivalently, p =
pe

1p+pe

, which is valid for any R. Note that p

ranges over the interval

(0, 1) as ranges over R, and p

=0
= p.
3