Excellent question, Debra!

There are various definitions for the P value, most of them difficult to understand for non-statisticians. I prefer describing it simply as the probability of getting a false positive result. In other words, statistical theory tells us that with a P value of 0.05, we have a 5% probability that our result (outcome) is statistically significant by chance alone (or 1 in 20 tests would give us a significant value by chance alone). If we set the nominal cutoff at 0.10, we are increasing the probability of finding a false positive result to 10%  (1 in 10).

Of course, if we increase our p value cutoff from 0.05 to 0.10, we are also more likely to find a significant result, and conversely  if we lower the p value cutoff to 0.01, it will be more difficult to find a significant result.

So now that we got all that out of the way, we can now address the question as to "where do we set out p value cutoff." By convention, most researchers set their nominal p value cutoffs at 0.05. However, some disciplines relax that to p = 0.10, while others set it more conservatively to 0.01.

To some degree, these nominal levels are set arbitrarily, since we know what the estimated p values are from running the appropriate statistical test. So in the end, the researcher ends some saying something like "the treatment effect was statistically significant (p value = 0.038)". They can say it was statistically significant because they told us ahead of time than their nominal cutoff was 0.05 and thus the 0,038 was lower than the 0.05.

FYI, I purposes did not go into all the details of the underlying statistical assumptions, such as stating the null hypothesis, etc., because that typically makes it more difficult to understand the concept. I do suggest you read up on the details when you get the time/inclination.

I hope this helps

Ariel   

Ariel:

Thanks for taking the time to break down your answer so that it made it easy for me to logically follow your explanation. Your explanation has helped my understanding of setting the "p value" better, so my reading up on it can now make more sense.

Many thanks,

Debra