Any Bayesian model is described by the variables in the model and the relationships between them. These variables can be observed (data), latent, or fixed to a particular value.

In a complete Bayesian model, any variable that is latent is described by a "prior" probability distribution - i.e. the distribution of the data that the modeller deems to be sensible given background knowledge.

Bayesian inference is then the process of using the data to update the beliefs about these latent variables to arrive at "posterior" (after the fact) distributions. Every prior distribution has parameters of some kind, for example the Gaussian distribution is typically described by a mean and a variance (although it can be parameterised in various different ways).

In a "flat" (non-hierarchical model), all of the parameters would have to be fixed - e.g. in the Gaussian case, a mean and variance would have to be fixed ahead of time, say to 0 and 1 respectively. However, we might think that having a prior mean of 0 is too restrictive, so we might want to further treat this as latent (to be learnt), and instead put a prior of the distribution of the mean (resp. variance). This would allow more flexibility in the modelling, at the cost of needing more data to arrive at a coherent answer. This is the simplest form of hierarchical model.

In general, virtually all Bayesian models are hierarchical in some sort, since this is the method by which complex models are built up. See e.g. http://mbmlbook.com/ for examples.

Note that there are also "quasi-Bayesian" models, in which variables do not have prior distributions, but inference is still performed on them. The classical "E-M algorithm" is one such example.

Hello Eveline Furrer,

Sorry for the late response, I hope this article and attached document could be helpful for you.

DOI:10.3758/BF03196750

Best Wishes