1

Homework 1 - Additional Problems

1.1. Let {Xn }n0 be a random walk on Z with momentum that we discussed in class. That is, let (1/2, 1) be xed and let X0 = 0, and P (Xn+1 Xn = Xn Xn1 ) = , P (Xn+1 Xn = (Xn Xn1 )) = 1 . P (X1 = 1) = P (X1 = 1) = 1/2,

That is, the rst step is like a simple symmetric random walk and all subsequent steps move in the same direction as the previous step with probability or reverse direction with probability 1 . We mentioned in class that {Xn }n0 is not a Markov chain. However, if we let Zn = (Xn , Xn Xn1 ) for n 1, then {Zn }n1 is a Markov chain with state space Z {1, 1}. What are the transition probabilities for the Zn ? That is, give a formula for p(i, j) when i = (i1 , i2 ) and j = (j1 , j2 ) are elements of Z {1, 1}.

1.2. Let {Xn }n0 be the gamblers ruin Markov chain with p = .45 and N = 6 (compare with Example 1.1 in the book). That is, if you currently have i > 0 dollars, in the next round the probability of winning $1 is p(i, i + 1) = .45 and losing $1 is .55, and you stop playing if your fortune ever reaches $0 or $6. a) Write down the transition probability matrix p = (p(i, j))i,j for this Markov chain.

b) Use a computer or calculator to compute p20 . Round each entry to 3 decimal places.

c) Explain in words (in terms of the gambler) what the rst column of p20 represents.

d) Explain in words (in terms of the gambler) what the 4-th row of p20 represents.

1.3. As in the previous problem, consider the gamblers ruin Markov chain with p = .45 and N = 6. Suppose the gambler starts with X0 = 3 dollars. The gambler will only quit gambling if he goes broke or doubles his money to $6. Use a computer to compute the matrix pn for some large values of n, and use this to approximate the probability that the gambler eventually doubles his money. Explain how you got your answer. (Well learn how to calculate this probability exactly later in the chapter, but for now use the above method to give a good approximation.)