Gibbs Sampling, some notes.
Gibbs sampling works when you want to
sample from the joint distribution of two variables, when it is easy for you to
sample from the distribution of one variable, conditional on the other.
The example involves a bivariate
Normal distribution, and is described here.
Here is the code
that goes with the note. It implements Gibbs sampling to construct a sample of
draws from the joint Normal distribution, working only with the conditional distributions.
The code does the Gibbs draws, and then displays two things: (i) the marginal distribution of the two variables, overlayed on the actual Normal marginal distribution that
we know they have (see the note); and (ii) the correlation between the two
variables (an indicator of their joint density) as implied by Gibbs sampling
and compares that with their actual correlation (again, for this see the note).