The interpretation of the p-value isn't the same everywhere and some even contradict each other.
1. So I have the book Basic econometrics by Gujarati and Porter, and in it, they state "This probability is called the p-value (i.e., probability value), also known as the observed or exact level of significance or the exact probability of committing a Type I error. More technically, the p-value is defined as the lowest significance level at which a null hypothesis can be rejected."
2. Some youtube videos (ex: https://www.youtube.com/watch?v=XyVDXRuA9Oc) claim the p-value is the probability of obtaining a value of X and more extreme.
3. This Minitab blog (http://blog.minitab.com/blog/adventures-in-statistics/how-to-correctly-interpret-p-values) even contradicts the book Basic econometrics of Gujarati and Porter stating "Incorrect interpretations of P values are very common. The most common mistake is to interpret a P-value as the probability of making a mistake by rejecting a true null hypothesis (a Type I error)."
Now my question is which of these is actually correct? Or do they complement each other in a certain way?
The interpretations given by Gujarati and Porter (in Basic econometrics) are correct. The interpretation given in the Minitab Blog, namely, "P value is the probability of obtaining an effect at least as extreme as the one in your sample data, assuming the truth of the null hypothesis," is also correct. But their statement: "the most common mistake is to interpret a P value as the probability of making a mistake by rejecting a true null hypothesis (a Type I error)" is wrong. Their justification for this false statement is as follows: "P values are calculated based on the assumptions that the null is true for the population and that the difference in the sample is caused entirely by random chance. Consequently, P values can’t tell you the probability that the null is true or false because it is 100% true from the perspective of the calculations." Nobody has ever said that the p-value is the probability that the null (H0) is true or false!!! What we say is the following: Assuming H0 is true, if we decide to reject it, the probability of making a type I error is the p-value.
There is an interesting fundamental paper of the American Statistical Association (ASA)
The ASA's Statement on p-Values: Context, Process, and Purpose
are available together with a discussion by Ronald L. Wasserstein and Nicole A. Lazar
from
http://amstat.tandfonline.com/doi/pdf/10.1080/00031305.2016.1154108
There you find an informal defintion
"Informally, a p-value is the probability under a specified statistical model that a statistical summary of the data (e.g., the sample mean difference between two compared groups) would be equal to or more extreme than its observed value."
You may find also a good characterisation of that concept there.
I attach my answer in a file because it contains a diagram which does not display in text mode.
@Manfred Borovcnik I read the paper, but can you please elaborate? According to this statement the statement "p-value = the exact probability of committing a Type I error" is wrong even though I'm still beeing taught p-value = the exact probability of committing a Type I error.
@mousimi it's more about the meaning of p-value itself. I understand I need a p value of as low as possible for it to have as much evidence as possible against the null hypothesis.
@Lall B. Ramrattan I don't see what you are suggesting. I'll try to make things easier, can you interpret (or anyone else) the p values of the independent variables of this screenshots I took of one of my hypothetical calculations. In words. If I were your student, what would you expect of me to say to you the p-value of 0.001 in this hypothetical result really meant? (see statistics.png)
Dear Elvedin,
It seems you are after an interpretation of the significance of regression coefficients. The t-statistics is what is usually reported (in standard form in bracket under the coefficient estimates) and not the p-values. The norm is for the t-values to be =>2 to be significant at the 95 percent confidence level, which is backed up by the p-values.
Your results show that the constant and X1 estimates are significant but X2 is not.
Good luck in your research.
Elvedin, assume that the statistical summary is clear and not disputed.
The specified statistical model corresponds to the null hypothesis.
As the p-value informally is
"the probability under a specified statistical model that a statistical summary ... would be equal to or more extreme than its observed value"
(which is somehow point 2 of your original question where the present data is signified by X)
the p-value would be equal to the smallest alpha level of a test, which would still lead to a rejection of the null hypothesis.
(Thus, if the p-value equals 0.01999..., you could still reject the null hypothesis at the 0.02 level using your data - with the caveat to repeat: alpha level has to be declared before you get the data, p-level is something that emerges after you see your data.)
In fact, you might refer to the p-value as Type-I error (with some reservation as you would have to fix the alpha level before getting the data).
As the ASA paper also states, you cannot interpret the p-values of probability of the null hypothesis is true given the present data (which coincides with the false interpretation of p-value under your point number 3)
If you perform an exact test (a' la Fisher) you get interpretation 2. This is a natural way of thinking about a p-value. You take your data (e.g units cross-classified in a 2X2 table) and calculate how far (left or right) the cross-classification you observe stands under the probability distribution corresponding to the null hypothesis that the observed arrangement is due to pure chance or that any possible arrangement is equally-likely Your p-value then is simply the proportion of all possible arrangements of cell frequencies whose values are at least as far to the left/right (or both) than the value associated with the single arrangement you actually observe.
Interpretation 1 comes from the Neyman-Pearson theory of hypothesis testing which is the mainstream approach found in textbooks, including Gujarati's. Not everyone agrees with it, however, (even in the frequentist field, let alone the Bayesians) also because a null hypothesis can never be proved conclusively but only possibly disproved and so there is no such thing as 'rejecting a true null'. The argument gets somewhat philosophical around the interpretation (or the need even) of a p-values and this is why you find this lack of consensus and apparently conflicting definitions.
I suggest you read some basic and fundamental papers in related to the statistics. It can guide you explore various ways of interpretation of p- value.
@Manfred Borovcnik
@Luca Mancini
YES! This is EXACTLY the answer I was looking for! More precisely this statement "The argument gets somewhat philosophical around the interpretation (or the need even) of a p-values and this is why you find this lack of consensus and apparently conflicting definitions."
Thanks a lot!
The interpretations given by Gujarati and Porter (in Basic econometrics) are correct. The interpretation given in the Minitab Blog, namely, "P value is the probability of obtaining an effect at least as extreme as the one in your sample data, assuming the truth of the null hypothesis," is also correct. But their statement: "the most common mistake is to interpret a P value as the probability of making a mistake by rejecting a true null hypothesis (a Type I error)" is wrong. Their justification for this false statement is as follows: "P values are calculated based on the assumptions that the null is true for the population and that the difference in the sample is caused entirely by random chance. Consequently, P values can’t tell you the probability that the null is true or false because it is 100% true from the perspective of the calculations." Nobody has ever said that the p-value is the probability that the null (H0) is true or false!!! What we say is the following: Assuming H0 is true, if we decide to reject it, the probability of making a type I error is the p-value.
Dear Elvedin, I totally agree with Dimitris. A practical application of the last statement of Dimitris occurs when You run a regression with OLS. You can use the P-value for the selection of the coefficients of the regressors, instead of "loose time" to consult the T statistics. As an example, if the p-value of a coefficient is 0.02 it means that there is a probability of 98% of do not make an error in rejecting the hypothesis that this coefficient has no statistic significance.
A P value, or calculated probability, is the probability of finding the observed, or more extreme, results when the null hypothesis (H0) of a study question is true – the definition of ‘extreme’ depends on how the hypothesis is being tested. P is also described in terms of rejecting H0 when it is actually true, however, it is not a direct probability of this state.
https://www.statsdirect.com/help/basics/p_values.htm
forexample, if p.value =0.24
interpretation
the p.value of 0.24 "means that you fail to reject HO by a 24% while 5% of chance you reject HO.
so, that the chance of error is 5% while 24% of chance you correct
some researchers make mistake for p.value and level of significant (Alpha)
simple definations
1.p.value is probability that tells us much your hypothesis is correct . high p.v meas your hypothesis likely to true while low p.v means your hypothesis is unlikely to true
note :
if p.value alpha ,then you fail reject HO
2. level of significant also called level of risk
In testing a given hypothesis, the maximum probability with
which we would be willing to take risk is called level of
significance of the test. This probability often denoted by “alpha” is
generally specified before samples are drawn.The level of significance usually employed in testing of
significance are 0.05( or 5 %) and 0.01 (or 1 %). If for example a
0.05 or 5 % level of significance is chosen in deriving a test of
hypothesis, then there are about 5 chances in 100 that we would
reject the hypothesis when it should be accepted. (i.e.,) we are
about 95 % confident that we made the right decision. In such a
case we say that the hypothesis has been rejected at 5 % level of
significance which means that we could be wrong with probability
0.05.
I disagree with Liibaan Cali's interpretation, that "the p.value of 0.24 "means that you can reject HO by a 24% while 76% of chance you fail to reject HO."
Suppose we are testing the null hypothesis that the mean of a population is 10 (H0: mu = 10) against a right-sided alternative (H1: mu > 10), and that the value of the sample statistic (X-bar) that generated a p-value 0.24 is X-bar = 11. This means that if in fact H0 is true, there would be a 24% chance of observing X-bar as large as 11, which, by the way, does NOT lead to the rejection of H0 (see T. Wonnacott and R. Wonnacott, Introductory Statistics for Business and Economics, 2nd Ed., 1977, pp. 246-248). If you say that there is a 24% chance of rejecting H0, then you in fact adopt the following erroneous statement: "If you take another random sample of equal size from the same population, the probability of rejecting H0 is 24%." This statement is obviously incorrect, however, as the decision rule (and hence the critical region) remains the same as before, but the value of X-bar in the new sample could be very different, and might lead to a p-value of, say, 0.001, i.e., a strong rejection. The p-value obtained from the previous sample, which did NOT even lead to rejection, is irrelevant, and hence its meaning can't be generalized.
the interpretation of p-value according to models such as OLS, the p-value tells us which is the variable are significant or insignificant
A P value, or calculated probability, is the probability of finding the observed, or more extreme, results when the null hypothesis (H0) of a study question is true – the definition of ‘extreme’ depends on how the hypothesis is being tested. P is also described in terms of rejecting H0 when it is actually true, however, it is not a direct probability of this state.
https://www.statsdirect.com/help/basics/p_values.htm
forexample, if p.value =0.24
interpretation
the p.value of 0.24 "means that you can reject HO by a 24% while 76% of chance you fail to reject HO.
some researchers make mistake for p.value and level of significant (Alpha)
simple definations
1.p.value is probability that tells us much your hypothesis is correct . high p.v meas your hypothesis likely to true while low p.v means your hypothesis is unlikely to true
note :
if p.value alpha ,then you fail reject HO
2. level of significant also called level of risk
In testing a given hypothesis, the maximum probability with
which we would be willing to take risk is called level of
significance of the test. This probability often denoted by “alpha” is
generally specified before samples are drawn.The level of significance usually employed in testing of
significance are 0.05( or 5 %) and 0.01 (or 1 %). If for example a
0.05 or 5 % level of significance is chosen in deriving a test of
hypothesis, then there are about 5 chances in 100 that we would
reject the hypothesis when it should be accepted. (i.e.,) we are
about 95 % confident that we made the right decision. In such a
case we say that the hypothesis has been rejected at 5 % level of
significance which means that we could be wrong with probability
0.05.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Probability