Gibbs Sampling, some notes.

Gibbs sampling works when you want to sample from the joint distribution of two variables, when it is easy for you to sample from the distribution of one variable, conditional on the other.

The example involves a bivariate Normal distribution, and is described here.

Here is the code that goes with the note. It implements Gibbs sampling to construct a sample of draws from the joint Normal distribution, working only with the conditional distributions. The code does the Gibbs draws, and then displays two things: (i) the marginal distribution of the two variables, overlayed on the actual Normal marginal distribution that we know they have (see the note); and (ii) the correlation between the two variables (an indicator of their joint density) as implied by Gibbs sampling and compares that with their actual correlation (again, for this see the note).