Sheldon Ross 10: Section 3.6.3 – Uniform Priors

Sheldon Ross 10 Section 3.6.3 – Uniform Priors-01.jpg

A prior in a uniform 😉


Introduction This is a rather interesting combination of the Standard Uniform Distribution and the Binomial Distribution.

As we know the Binomial Distribution has three parameters.

  • Number of trials
  • Probability for 0 (or 1)
  • Number of the occurrences of 0 or 1

For many applications, we assume p to be constant and calculate the probabilities of k occurrences out of n trials. It would be straightforward to calculate and anticipate how the probabilities might look like. How about the case when the probability creating the binomial is itself a distribution? Interesting isn’t it?

Choices on the probability distribution for the p.

  • Since we need a number between 0 and 1, we need a distribution that has finite limits.
    • this makes exponential (and several other) probability distribution function completely useless
  • We can choose distributions like
    • Standard Uniform Distribution
    • Triangular distribution with limits between 0 and 1
      • there is no restriction on the peak position
    • Any other functions whose integral is 1 and the outcomes space is between 0 and 1
  • We will look at the simulations of the Binomial outcomes with randomly varying probability

Calculations

For the calculations let us recapture a few things

  • We will perform n bernoulli trials sequentially
  • We need to calculate the probability that number of successes is k
  • The probability is itself randomly distributed by a uniform distribution

P{ X=k | p=uniform[0,1] } = ∫ Binomial ( n, k, p ) PDF[ uniform[0,1] ] dp

∫ pk(1-p)2n-k dp = Γ(n+1)Γ(n+2) = n!n+1 ! = 1n+1

Voila! 😮 we have removed the dependence of probability on the k. See how the probability is only dependent on the number of trials and for all k: 0 ≤ k ≤ n. Seeing this in another way, any of the k successes amongst n trials have the same probability of happening. This is an astounding result.


Simulations

Let us do some simulations to see the results we obtained in the previous section. The simulation would only take a few lines but is a very powerful illustration of the concept. We will pass two types of the probability functions to construct the Binomial Distribution.

 

Uniform probability distribution from [0, 1]

 

Triangular distribution [0, 1] and peak at 0.5 – Can you guess why the distribution is dome shaped?


Code

AppendTo[$Path, "D:\\Mathematica Files 4K\\mathematicaPackages"];
SetOptions[$FrontEndSession, "EvaluationCompletionAction" -> {"ScrollToOutput"}]
Needs["utilities`"];

Uniform Priors

Gamma[n + 1] / (Gamma[k + 1] Gamma[n - k + 1]) \!\(
\*SubsuperscriptBox[\(\[Integral]\), \(0\), \(1\)]\(
\*SuperscriptBox[\(p\), \(k\)]
\*SuperscriptBox[\((1 - p)\), \(n - k\)]\[DifferentialD]p\)\)

ListPlot[{Gamma[# + 1] / Gamma[# + 2] & /@ Range[100],
  1 / (# + 1) & /@ Range[100]}, PlotRange -> All, ImageSize -> 700]

Module[{p = RandomReal[] & /@ Range[10000], data},
  data = KeySort@Counts[RandomVariate[BinomialDistribution[20, #]] & /@ p];
  BarChart[data,
    ChartLabels -> Placed[{Keys@data, Values@data}, {Below, Above}],
    ImageSize -> 788]
]

Module[{p = RandomVariate[TriangularDistribution[{0, 1}], 100000],
  data},
  data = KeySort@
      Counts[RandomVariate[BinomialDistribution[20, #]] & /@ p];
  BarChart[data,
    ChartLabels -> Placed[{Keys@data, Values@data}, {Below, Above}],
    ImageSize -> 788]
]

End of the post 😉