During the lecture, the lecturer mentioned the properties of Frequentist. As following
Unbiasedness is only one of the frequentist properties — arguably, the most compelling from a frequentist perspective and possibly one of the easiest to verify empirically (and, often, analytically).
There are however many others, including:
1. Bias-variance trade-off: we would consider as optimal an estimator with little (or no) bias; but we would also value ones with small variance (i.e. more precision in the estimate), So when choosing between two estimators, we may prefer one with very little bias and small variance to one that is unbiased but with large variance;
2. Consistency: we would like an estimator to become more and more precise and less and less biased as we collect more data (technically, when n → ∞).
3. Efficiency: as the sample size incrases indefinitely (n → ∞), we expect an estimator to become increasingly precise (i.e. its variance to reduce to 0, in the limit).
Why Frequentist has these kinds of properties and can we prove it? I think these properties can be applied to many other statistical approach.
Sorry, Jianhing. But I think you have misunderstood something in the lecture. Frequentist statistics, which is an interpretation of probability to be assigned on the basis of many random experiments.
In this setting, on designs functions of the data (also called statistics) which estimate certain quantities from data. For example, the probability p of a coin to land heads is given from n independent trials with the same coin and just counting the fraction of heads. This is then an estimator for the parameter p.
Each estimator should have desirable properties, as unbiasedness, consistency, efficiency and low variance and so on. Not every estimator has these properties. But, in principle one can proof, whether a given estimator has these properties.
So, it is not a characteristics of frequentist statistics, but a property of an individual estimator based on frequentist statistics.
Maik Kschischo Hi Maik! Thanks for your answer. So you mean that here these properties belong to the estimator rather than frequentist itself. Using the example you mentioned, the frequentist is based on many repeated trials under the same conditions so if we want to calculate parameter p and combine the core idea of frequentist (repeated the trails many times) we just need to counting the fraction of heads in these many trails. Then, based on mathematics, as the repeated times go to infinite we will get the properties of this estimator. Is my understanding correct?
Roughly, yes. For this example p, you can show that for infinitely many observations (here coin tosses), the fraction of success (here the mean of a Bernoulli random variable) will converge to the true p in some sense (stochastic convergence). This means, the estimator is asymptotically unbiased. Similarly, the expectation value of the fraction of successes equals the true p, which means it is unbiased. But depending on the numbver of samples, one can choose other estimators with lower variance, but pay with some bias. This means, if you have only a few tosses, other estimators might vary less from sample to sample, but might be biased.
I recommend "All of Statistics" from Larry Wassermann to my students, where this is explained. But other good books on Statistics will also explain this. So, these desirable properties of an estimator are concepts from frequentist statistics, but not every estimator can deliver them all at the same time. Sometimes, one has to choose an estimator which is suitable for a given situation.
There are other approaches to Statistics, e.g. Bayesian statistics. Here, you can include prior knowledge and belief and combine it with data. There, also the interpretation of probabilities is different.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS