# Markov Chain Chap 11.5 - 9

Consider a random walk on a circle of circumference n. The walker takes one unit step clockwise with probability p and one unit counterclockwise with probability q . Modify the program ErgodicChain to allow you to

input n and p and compute the basic quantities for this chain. (a) For which values of n is this chain regular? ergodic? n = odd regular n = even ergodic (b) What is the limiting vector w? (c) Find the mean first passage matrix for n = 5 and p = .5.

> P <- matrix(data = 0, nrow = 5, ncol = 5) > P[1, 2] = P[1, 5] = 0.5 > P[2, 1] = P[2, 3] = 0.5 > P[3, 2] = P[3, 4] = 0.5 > P[4, 5] = P[4, 3] = 0.5 > P[5, 1] = P[5, 4] = 0.5 > coefs <- t(P) - diag(5) > solve(rbind(c(1, 1, 1, 1, 1), coefs)[1:5, ], c(1, 0, 0, 0, 0)) [1] 0.2 0.2 0.2 0.2 0.2 > W <- matrix(data = 0.2, nrow = 5, ncol = 5) > Z <- solve(diag(5) - P + W) > M <- matrix(data = 0, nrow = 5, ncol = 5) > for (i in 1:5) { + for (j in 1:5) { + M[i, j] <- (Z[j, j] - Z[i, j])/0.2 + } + } > print(M) [,1] [,2] [,3] [,4] [,5] [1,] 0 4 6 6 4 [2,] 4 0 4 6 6 [3,] 6 4 0 4 6 [4,] 6 6 4 0 4 [5,] 4 6 6 4 0 |

d times n-d works

I am having a wonderful feeling that I have a tool to calculate probabilities for most of the classical probability problems