Sheldon Ross 10: Exercise 4.08

Question: Consider Example 4.1 in which the weather is considered as a two-state Markov chain. If α = 0.7 and β = 0.4, then calculate the probability that it will rain four days from today given that it is raining today. The transition matrix is defined below

0.7 0.3
0.4 0.6

Analytical Solution

In order to calculate the probability that it will rain 4 days from now, simply multiply the transition matrix with itself 4 times and the result is shown below. The required probability is the first element of the first row which is 0.5749

0.5749 0.4251
0.5668 0.4332

Simulation Solution

I tried visualizing the transitions for about 10000 days using bezier curves. The same is plotted below.


Code

Module[{transitionMatrix = {{0.7, 0.3}, {0.4, 0.6}}},
  MatrixPower[transitionMatrix, 4]]

Clear[markovSimulator];
markovSimulator[transitionMatrix_?MatrixQ, currentState_Integer] :=
    Module[{states = Range[Dimensions[transitionMatrix][[1]]]},
      RandomChoice[transitionMatrix[[currentState]] -> states]];

Module[{transitionMatrix, state = 1, coordinates, simulation,
  transitions, labelCoordinates, nStates},
  transitionMatrix = {{0.7, 0.3}, {0.4, 0.6}};
  nStates = Length@transitionMatrix;
  simulation =
      Table[state = markovSimulator[transitionMatrix, state], 10000];
  coordinates =
      Association@
          MapThread[{#1 -> #2} &, {Range@nStates, CirclePoints@nStates}];
  labelCoordinates = 2 CirclePoints[Length@transitionMatrix];
  coordinates = coordinates[#] & /@ simulation;
  transitions =
      Partition[#, 2] &[Riffle[Drop[coordinates, -1], Rest[coordinates]]];
  Graphics[{{Opacity@0.02, Hue[RandomReal[{0.5, 0.7}]], Thickness@0,
    BezierCurve[{#[[1]],
      Mean[{#[[1]] + #[[2]]}] + RandomReal[{-0.5, 0.5}, 2], #[[
          2]]}]} & /@ transitions,
    MapThread[
      Text[Style[Rotate[#1, #3 + \[Pi]], 16], #2] &, {{"Rains Tomorrow",
      "Doesn't Rain Tomorrow"}, labelCoordinates,
      Range[\[Pi] / 2, 3 \[Pi] / 2, \[Pi]]}]}, ImageSize -> 788]
]

End of the post


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.