The Bayesian probability measures a "degree of belief". It pretty much matches our every-day intuitive understanding of probability, only that probability in a mathematical sense must satisfy Kolmororow's probability axioms - but this is not important to understand the concept. So when a layman thinks about probability, it will be the "Bayesian probability" right away. The more I belief in something, the more I am convinced of something, the higher the probability. Probability just gives a numeric value for the strength of belief/conviction.
The frequentists interpretation needs some explanation. Frequentists can assign probabilities only to events/obervations that come from repeatable experiments. With "probability of an event" they mean the relative frequency of the event occuring in an infinitively long series of repetitions. For instance, when a frequentists says that the probability for "heads" in a coin toss is 0.5 (50%) he means that in infinititively many such coin tosses, 50% of the coins will show "head".
The difference may become clear when bot (Bayesian frequentist) are asked for the probability that this particular coin I just tossed shows "head".
The Bayesian will say: "Since I have no further information than that there are two possible outcomes, I have no reason to prefer any of the sides, so I will assign an equal probability to both of them (what will be 0.5 for "heads" and also 0.5 for "tails").
The frequentist will say: "There is no answer to this question. A particular event cannot have a probability. The coin shows either head or tails, and unless you show it, I simply can't say what is the fact. Only if you would repeat the toss many, many times, any if you vary the initial conditions of the tosses strongly enough, I'd expect that the relative frequency of heads in all thes many tosses will approach 0.5". Here the frequentist uses sone knowledge about coin tosses (physics, variability). He assumes that the probability is 0.5. Actually someone has to claim that the probability just is 0.5. That's actually as stupid as it sounds. But previous experience may show that this assumption is not bad. The measured relative frequencies from previous experiments can be used as estimates for the probability.
Note that "experiment" can also mean "sampling (with replacement!) from a population", what can be repeated infinitively often. When the long-run frequencies (=probabilities) of selection for all items in the population are equal -we call this a random selection-, it is simple to calculate long-run frequencies(=probabilities) for properties of the sample. For example, if the proportion of males in a state is 52%, the probability of a randomly selected person from this state being male is simply 0.52. The frequentits sais that the "selection of a male person has a 0.52 probability" but he won't say what the probability is that any actually selected person is male (same as for the coin toss). His only claims that in the infinitively long run, 52% of repeatedly selected people will be male.
The Bayesian should also use this knowledge when being asked for the probability that a particular (but yet unknown) person of this state is male. His argumentation will be: I have no information about the sampling, so I can not prefer any particular person being sampled. Given the informaton that 52% of the people are male, my posterior probability for "male" will be 0.52.
Simplistically. Here's my simplistic version (for whatever it is worth):
The frequentist interpretation of probability interprets outcomes in terms of frequency. For example, take flipping a coin. We all know that despite the fact a flip of a fair coin is .5, it is entirely possible to flip 10 coins and get ten heads. The frequentist position holds that probability ought to be understood as the expected value given a large number of idealized flips (i.e., we intepret the probablity of getting some sequence of heads and tails given any number of coin tosses as being equal to what we would expect given that every flip has by definition a .5 chance of being heads or tails). The subjective (including Bayesian) interpretation of probability recognizes that despite the intuitive sense this makes, there is no a priori reason for thinking that given 6 tosses of a fair coin, we should expect 3 heads and 3 tails. In fact, that's probably NOT going to be the outcome. So instead, Bayesians interpret probability as a measure of one's (justified) belief in an outcome given prior knowledge. Instead of deciding that the probability of any sequence of fair coin tosses is equal and is determined by the fact that every toss has a .5 chance of being heads or tails, the probability for Bayesians is that any given toss is based on the assumption that there is a .5 chance and this assumption is held so long as it continues to best predict the outcome of future coin tosses.
Basically, the frequentist interpretation assumes an idealized number outcomes given identically (and idealized) events. Every coin toss has a .5 chance of being heads or tails, every draw from a deck has a 1/13 chance of being an Ace, and the normal distribution is God (yes I exaggerate and pardon the blasphemy). For bayesians, the fact that this is an idealization makes assuming it problematic, so it is understood as uncertain and indeed defined as such. Instead of using the standard probablity assumptions as the actual probability, bayesians use these as a measure of uncertainty. Given that a fair coin toss should, according to subjective beliefs, be equally likely to be heads or tails, we represent this as the belief that it should be. If we continually toss a coin and get heads, we become more and more certain that the coin isn't fair, rather than assuming it is. I haven't slept in 24 hours so if this makes no sense, I'll return to the inquiry later. It's a vital issue and how to simplify it is a key issue I've struggled with before.
I decided to take the opportunity to return to this question and found that 1) Dr. Wilhem unsurprisingly provided a perfect explanation and 2) my response contains about the worse sentence possible. Specifically: "The subjective (including Bayesian) interpretation of probability recognizes that despite the intuitive sense this makes, there is no a priori reason for thinking that given 6 tosses of a fair coin, we should expect 3 heads and 3 tails." The use of a priori and expectation, while simplistically correct informally, is just a terrible just of terminology. Bayesian inference involves a certain equating of expected values with a priori reasoning. That is, the frequentist uses expected values to define probability. The bayesian uses expected values as the basis for the prior belief that an event will have some particular outcome. Thus in a certain, real sense, we can simplistically define the first step in bayesian inference by a priori beliefs regarding expected outcomes. The important point is that this is not, in bayesianism, the definition or approach to probability. Rather (and again simplistically), the chances (more or less expected value) for some outcome is the basis for an initial or prior belief for some particular outcome/hypothesis, while probabliity is defined in terms of these beliefs.
The word probability has been used in a variety of ways since it was first applied to the mathematical study of games of chance. Does probability measure the real, physical tendency of something to occur or is it a measure of how strongly one believes it will occur, or does it draw on both these elements? In answering such questions, mathematicians interpret the probability values of probability theory.
There are two broad categories[1][2] of probability interpretations which can be called "physical" and "evidential" probabilities. Physical probabilities, which are also called objective or frequency probabilities, are associated with random physical systems such as roulette wheels, rolling dice and radioactive atoms. In such systems, a given type of event (such as a die yielding a six) tends to occur at a persistent rate, or "relative frequency", in a long run of trials. Physical probabilities either explain, or are invoked to explain, these stable frequencies. The two main kinds of theory of physical probability are frequentist accounts (such as those of Venn,[3] Reichenbach[4] and von Mises[5]) and propensity accounts (such as those of Popper, Miller, Giere and Fetzer).[6]
Evidential probability, also called Bayesian probability, can be assigned to any statement whatsoever, even when no random process is involved, as a way to represent its subjective plausibility, or the degree to which the statement is supported by the available evidence. On most accounts, evidential probabilities are considered to be degrees of belief, defined in terms of dispositions to gamble at certain odds. The four main evidential interpretations are the classical (e.g. Laplace's)[7] interpretation, the subjective interpretation (de Finetti[8] and Savage[9]), the epistemic or inductive interpretation (Ramsey,[10] Cox[11]) and the logical interpretation (Keynes[12] and Carnap[13]). There are also evidential interpretations of probability covering groups, which are often labelled as 'intersubjective' (proposed by Gillies[14] and Rowbottom[6]).
Some interpretations of probability are associated with approaches to statistical inference, including theories of estimation and hypothesis testing. The physical interpretation, for example, is taken by followers of "frequentist" statistical methods, such as Ronald Fisher, Jerzy Neyman and Egon Pearson.[citation needed] Statisticians of the opposing Bayesian school typically accept the existence and importance of physical probabilities, but also consider the calculation of evidential probabilities to be both valid and necessary in statistics. This article, however, focuses on the interpretations of probability rather than theories of statistical inference.
The terminology of this topic is rather confusing, in part because probabilities are studied within a variety of academic fields. The word "frequentist" is especially tricky. To philosophers it refers to a particular theory of physical probability, one that has more or less been abandoned. To scientists, on the other hand, "frequentist probability" is just another name for physical (or objective) probability. Those who promote Bayesian inference view "frequentist statistics" as an approach to statistical inference that recognises only physical probabilities. Also the word "objective", as applied to probability, sometimes means exactly what "physical" means here, but is also used of evidential probabilities that are fixed by rational constraints, such as logical and epistemic probabilities.
It is unanimously agreed that statistics depends somehow on probability. But, as to what probability is and how it is connected with statistics, there has seldom been such complete disagreement and breakdown of communication since the Tower of Babel. Doubtless, much of the disagreement is merely terminological and would disappear under sufficiently sharp analysis.