I read Aron, Coups, & Aron (2013), Statistics for Psychology, to get explanation about the logic of hypothesis testing.
I got confusing statements about rejecting/ accepting Hypotheses Null and Alternative:
[In almost all psychology research, we base our conclusions on the question, “What is the probability of getting our research results if the opposite of what we are predicting were true?” That is, we usually predict an effect of some kind. However, we decide on whether there is such an effect by seeing if it is unlikely that there is not such an effect.
If it is highly unlikely that we would get our research results if the opposite of what we are predicting were true, that finding is our basis for rejecting the opposite prediction. If we reject the opposite prediction, we are able to accept our prediction.
However, if it is likely that we would get our research results if the opposite of what we are predicting were true, we are not able to reject the opposite prediction. If we are not able to reject the opposite prediction, we are not able to accept our prediction.] (pp.111)
Can you help me to understand?
Null hypothesis testing is design to answer the question of whether an observed difference is likely (to a pre-agreed level of probability) to occur by random chance if in reality there is no effect. These are tests of difference. There is only the need to have no interval overlap to achieve a definitive answer.
Alternative hypothesis testing is design to answer the question of whether an observed similarity (to a pre-agreed level of probability) to occur by random chance if in reality there is the size of effect we expect. These are tests of equivalence. To do this we need to establish that the confidence intervals for our data overlap with a relevant proportion of the confidence intervals for the alternative hypothesis, so both upper and lower confidence intervals need to be considered.
The possibility space for no overlap is unconstrained and larger than that ways to achieve overlap. This is why a negative result in null testing does not imply a positive result in alternative hypothesis testing.
The quotes you have are not wrong but are easily misread. Rejecting the null does not prove our prediction, it just gives use reasonable faith (the pre-agreed level of probability) that our prediction is not due to random chance. If we want to have faith that our prediction is what we want, then we need to test for equivalence to that prediction.
Why is null testing more common than alternative testing? Because alternative testing is much more stringent and tends to require much larger resources.
During early development (which characterises most scientific research)
1) the biggest risk is that you develop something that is not in fact real - there are more false avenues than real ones.
2) there is usually very little to provide a reliable estimate of expected true effect size in order to formulate a reasonable alternative hypothesis.
3) the people that need to be convinced are funders who will support the ongoing development. Their main concerns is that most proof of concept results fail, so they want to find the few that are not likely due to random chance.
This makes alternative hypothesis testing unattractive but null hypothesis test more useful
Later on in development
1) you are hopefully more confident you are not deluded and are exploiting something real.
2) You will have more data to formulate a reasonable alternative hypothesis so you know what you are testing against.
3) the people that need convinced are regulators, purchasers, markerters... people who will be using the tool in the real world in some way. These people need to see that the tool will provide an expected result.
This make alternative testing more important and attractive, as it will show how the thing under development will work in real situations. However, it is not uncommon to encounter a range of realistic alternate testing scenarios in real world development, and to be scientifically rigorous these would need experiments to test them individually. In reality it is common to test a few alternative hypotheses then extrapolate to new situations and hope for the best.
Null hypothesis testing is design to answer the question of whether an observed difference is likely (to a pre-agreed level of probability) to occur by random chance if in reality there is no effect. These are tests of difference. There is only the need to have no interval overlap to achieve a definitive answer.
Alternative hypothesis testing is design to answer the question of whether an observed similarity (to a pre-agreed level of probability) to occur by random chance if in reality there is the size of effect we expect. These are tests of equivalence. To do this we need to establish that the confidence intervals for our data overlap with a relevant proportion of the confidence intervals for the alternative hypothesis, so both upper and lower confidence intervals need to be considered.
The possibility space for no overlap is unconstrained and larger than that ways to achieve overlap. This is why a negative result in null testing does not imply a positive result in alternative hypothesis testing.
The quotes you have are not wrong but are easily misread. Rejecting the null does not prove our prediction, it just gives use reasonable faith (the pre-agreed level of probability) that our prediction is not due to random chance. If we want to have faith that our prediction is what we want, then we need to test for equivalence to that prediction.
Why is null testing more common than alternative testing? Because alternative testing is much more stringent and tends to require much larger resources.
During early development (which characterises most scientific research)
1) the biggest risk is that you develop something that is not in fact real - there are more false avenues than real ones.
2) there is usually very little to provide a reliable estimate of expected true effect size in order to formulate a reasonable alternative hypothesis.
3) the people that need to be convinced are funders who will support the ongoing development. Their main concerns is that most proof of concept results fail, so they want to find the few that are not likely due to random chance.
This makes alternative hypothesis testing unattractive but null hypothesis test more useful
Later on in development
1) you are hopefully more confident you are not deluded and are exploiting something real.
2) You will have more data to formulate a reasonable alternative hypothesis so you know what you are testing against.
3) the people that need convinced are regulators, purchasers, markerters... people who will be using the tool in the real world in some way. These people need to see that the tool will provide an expected result.
This make alternative testing more important and attractive, as it will show how the thing under development will work in real situations. However, it is not uncommon to encounter a range of realistic alternate testing scenarios in real world development, and to be scientifically rigorous these would need experiments to test them individually. In reality it is common to test a few alternative hypotheses then extrapolate to new situations and hope for the best.
Historically, null hypothesis testing as a statistical idea parallels classic Philosophy of Science where nothing is actually proven. Proof is for math. We might think we know that, “all swans are white” but no matter how many white swans we find there might be a black swan out there we haven’t seen. We might think we know Newton’s Universal Law of Gravitation but maybe we just haven’t found the place in the universe where it doesn’t apply. As scientists we always have our best guess awaiting falsification. Statistically, that means we have a hypothesis we’d like to test (e.g., X is different from chance, X is different from Y, X & Y are correlated) but we can never actually prove a hypothesis. So philosophical we’re trying to reject a comparable null hypothesis (e.g., X is chance, X and Y are the same, X & Y have no correlation). When we’re successful, we don’t say we proved our hypothesis, rather we say we can reject the null. When we’re unsuccessful, we don’t say we proved we’re wrong, but rather we say we failed to reject the null. Mainly we just need to flip around our common sense way of speaking for the statistical way of speaking. Hope that helps some, Aftina. ~ Kevin
Hypothesis testing is done because one uses only a sample of a big population. That the researcher would like to test if data taken from the sample are sufficient enough to claim some characteristics of the population. Meaning to say- that the researcher would verify the sufficiency of the sample statistics ( characteristics of the sample) to the population parameters ( characteristics of the population). There are 2 types of hypotheses: the null (ho)and the alternative (ha). Normally the ho is the working hypothesis and at the end of the analysis ho can be rejected or accepted. If ho is rejected then ha is accepted. If ho is accepted then ha is rejected. Ho is rejected if the p value is less than or equal to alpha value. If the p value is greater than alpha value then ha is accepted. Alpha is called as the maximum probability of commiting type 1 error. In hypothesis testing we deal on errors namely Type 1 and type 2. Why do we have errors? It is because of the follwing situations: in reality ho is true or false and at the end jo can be accepted or rejected. if ho is accepted and ho is true then it is a correct decision. If ho rejected and ho is false then it is also a correct decision. In research if ho is false it has to be rejected and if ho is true then it has to be accepted. However, it is possible that a true ho is rejected and we call it type 1 and a possibility that a false ho is accepted which is called type 2.
In hypothesis testing, you need to establish your ho and ha as well as your claim. The claim might be ho or ha depending on the statement of the problem. However, if ho is the working hypothesis then ho is tje claim.
If ho is rejected and the claim is ho then one would conclude that there is sufficient or enough evidence to reject ho. If ho is accepted and the claim is ho then one would conclude that there is NOT enough evidence to reject ho.
- Badges
- Science topic
- Similar topics
- Philosophy
- Philosophy Of Science
- Reasoning