# Sheldon Ross 10: Example 4.10 Question: An urn always contains 2 balls. Ball colors are red and blue. At each stage, a ball is randomly chosen and then replaced by a new ball, which with probability 0.8 is the same color, and with probability 0.2 is the opposite color, as the ball it replaces. If initially, both balls are red, find the probability that the fifth ball selected is red.

Analytical Solution

Before solving anything, let us identify the configurations here. Since there are two slots available in the urn, and the number of ways in which the balls can be arranged is

• Red-Red: Let’s call this state A
• Blue-Blue: Let’s call this state B
• Red-Blue or Blue-Red: Let’s call this state C

The system starts with the state A. Lets work this through a flow chart to dissect this problem.

• State A
• to State A: Probability = 0.8 – since both balls are red, to begin with, the probability that the replaced one is also red is 0.8
• to State B: Probability = 0.0 – since we are at state A, the probability that the new ball added makes them both blue is zero
• to State C: Probability = 0.2 – since both balls are red, the probability the replaced ball is blue is 0.2
• State B
• to State A: Probability = 0.0 – since both are blue, to begin with, the probability that the newly added one makes both balls blue, is zero
• to State B: Probability = 0.8 – since both balls are blue, to begin with, the probability that the replaced on is also blue is 0.8
• to State C: Probability = 0.2 – since both balls are, the probability that the replaced ball is red is 0.2
• State C
• When the urn is in state C, the probability of selecting either blue or red is 50-50. Then the ball that is replaced is conditionally dependent on the picked ball
• The probability that the red ball is picked is 0.5
• The probability that a red ball is replaced is 0.8 ⇒ the urn stays in state C with a probability of 0.8*0.5 = 0.4
• The probability that a blue ball is replaced is 0.2 ⇒ the urn goes to state B with a probability of 0.2*0.5 = 0.1
• The probability that the blue ball is picked is 0.5
• The probability that a blue ball is replaced is 0.8 ⇒ the urn stays in state C with a probability of 0.8*0.5 = 0.4
• The probability that a red ball is replaced is 0.2 ⇒ the urn goes to state A with a probability of 0.2*0.5 = 0.1
• Hence from state C, the system
• goes to state A with a probability of 0.1
• goes to state B with a probability of 0.1
• stays in state C with a probability of 0.4+0.4 = 0.8

Now we have enough information to construct the transition matrix.

 A B C A 0.8 0 0.2 B 0 0.8 0.2 C 0.1 0.1 0.8

Simulation Solution

The simulation for the image is given below. Rather interesting isn’t it? We see no transitions between A and B just A to B and B to C. This is also compliant with the transition matrix that we have designed above. Code

```Module[{transitionMatrix = {{0.8, 0.0, 0.2}, {0.0, 0.8, 0.2}, {0.1,
0.1, 0.8}}}, MatrixPower[transitionMatrix, 2]]

Clear[markovSimulator];
markovSimulator[transitionMatrix_?MatrixQ, currentState_Integer] :=
Module[{states = Range[Dimensions[transitionMatrix][]]},
RandomChoice[transitionMatrix[[currentState]] -> states]];

Module[{transitionMatrix, state = 1, coordinates, simulation,
transitions, labelCoordinates, nStates},
transitionMatrix = {{0.8, 0.0, 0.2}, {0.0, 0.8, 0.2}, {0.1, 0.1,
0.8}};
nStates = Length@transitionMatrix;
simulation =
Table[state = markovSimulator[transitionMatrix, state], 5000];
coordinates =
Association@
MapThread[{#1 -> #2} &, {Range@nStates, CirclePoints@nStates}];
labelCoordinates = 2.3 CirclePoints[Length@transitionMatrix];
coordinates = coordinates[#] & /@ simulation;
transitions =
Partition[#, 2] &[Riffle[Drop[coordinates, -1], Rest[coordinates]]];
Graphics[{{Opacity@0.1, Hue[RandomReal[{0.60, 0.65}]], Thickness@0,
BezierCurve[{#[],
Mean[{#[] + #[]}] + RandomReal[{-0.25, 0.25}, 2],
Mean[{#[] + #[]}] + RandomReal[{-0.5, 0.5}, 2], #[[
2]]}]} & /@ transitions,
Text[Style[Rotate[#1, #3], 16], #2] &, {{"Red-Red", "Blue-Blue",
"Red-Blue"}, labelCoordinates, {\[Pi] / 3, 0, -\[Pi] / 3}}]},
ImageSize -> 788] // Framed
]```

End of the post 😉

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.