Friends, we have heard about 'lies, damned lies and statistics'. So what are the thoughts that come to your mind. I remember that when I was doing my studies, and I got a significant value, I was very happy. But now, my feelings changed a bit...

It is in fact usual it happens

Dear Miranda: I am not an expert on statistics. It will depend upon your hypothesis. It essentially means that you are rejecting the null hypothesis. So you should not lose heart but reexamine your hypothesis and the assumptions that you have started with. Remember Thomas Edison “I have not failed. I've just found 10,000 ways that won't work.”.

@Prof Hatem, do you mean, it's usual that we feel happy to get a significant value of p, when we do a research? :)

Statistics are based on the idea that we cannot perfectly observe the object of inference, and, thus, must draw inference through probability statements (be it Information theoretic or p-values or other). However the meaning of the p-value (as the probability of a type-I error) is only accurate if the statistical assumptions of the analysis are effectively met. So, it seems to me that uncertainty over what a given p-value means should only be related to uncertainty as to whether the statistical assumptions of the analysis were met.

Why the doubts now, Miranda? With that population, with that sample, at that time, using the best method available then, you found something statistically significant. That is so unexpected that it demanded a report.

It is not as if you ran a thousand research studies, and only reported the one that gave p

Dear Ian and friends, 'the doubts now' is because I am slightly better informed. O, most of you know that I'm careful and honest, but when I was a student I never heard of 'lies, damned lies and stats'. More to follow, soon...

Dear Miranda

What I mean that whatever you get aresult in the research should mean something

Recently, I came across one interesting article. It says a few things that needed hard thinking. To me, it's mind boggling. What is your input to these things? I quote:

'The smaller the studies, the less likely the research findings are to be true.

The more quantities there are to find relationships between, the less likely the research findings are to be true.

The greater the flexibility in designing studies, the less likely the research findings are to be true.

The more financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true.

The hotter a scientific field, and the more scientific teams involved, the less likely the research findings are to be true.'

Dear Dr Mirandah, for me significant or non significant is not important and do not be happy or sad for each of them. The point for me is presenting real and true data in publication, because both of them considered as result and you have to show the truth of the research. TQ

Thanks Dr Heshu, yes. I do just that:

'The point for me is presenting real and true data in publication, because both of them considered as result and you have to show the truth of the research. '

Thank you friends for your views:

@Dan: 'Statistics are based on the idea that we cannot perfectly observe the object of inference, and, thus, must draw inference through probability statements (be it Information theoretic or p-values or other). '

@Ahed: 'we have to be rational and aware of interpreting our data'

@Prof N: 'reexamine your hypothesis and the assumptions that you have started with'

@Ian: 'It is not as if you ran a thousand research studies, and only reported the one that gave p

Dear Miranda,

Let figures and true results lead the study. As long as your sample represents the universe and you have true data, the results will be accurate. You found results were statistically significant, I can`t see any problem. Thank you.

@Prof Mahfuz, I did learn something good. Now for my research on musical mnemonics, I do a lot MORE than 1 study :))

But about the statements I quoted, does anyone have some great ideas to discuss?

I think that the study design needs to evaluate the adequacy of sample size. If the sample is adequate, even a minimal statistical significance demonstrates something, expecially if the study was methodologically sound and is in accord with the feelings that spurred it. However, a low significance needs a confirmation by further studies on a large sample. Despite these considerations, your finding is a "scientifically decent" finding.

I love the cartoon at http://johncarlosbaez.wordpress.com/2013/09/11/why-most-published-research-findings-are-false/ which explains the issue so clearly. Although I do not agree with the title of the page, we do also need to ask questions like: "How important is this result to the world?" Green Jelly Beans may have a significant, but unimportant effect.

Dear Miranda,

and other RG-friends,

as some other here I too am no expert in statistics . So I better should shut up in this thread. But reading

Nageswara Posinasetti's post / reply above · I also would like to state, that outcome of your statistical calculations will not only depend upon your hypothesis, but as well as the number of (valid) data you put into the calculation (not knowing if that is 100% right, but it's what I remember from my some 30 years ago...). I have had colleagues which tried - when failing a successful attempt - to use other unless their hypotheses fitted into the results(or vice versa).

The statement by Thomas Edison

“I have not failed. I've just found 10,000 ways that won't work.” I found really neat and true for most situations of our life too [ok, not 10,000 but at least 25 (:-))],

regards,

Wolfgang

PS: Nasty counterquestion: Can you calculate statistically a p-value in a study comparing (e. g. therapy) data on two patients, one of which is "healthy" - the other ill and has been treated? (I really was asked this when I was a student!).

Dear Profs Enzo, Ian: thanks.

Now, Ian, you have helped me to locate the source of the statements I saved, but forget to bookmark!

I'm glad that you disagree with the title; I believe that many of us have integrity. We also prefer to engage in open discussions about our problems and doubts, rather than keeping quiet.

Dear Profs Wolfgang and Shafig, thanks for your views. I'm glad that I asked this question and got all your input. This is the reason I stick with RG; the learning I get is GREAT :)

@Wolfgang, the questions we were asked during the defense of thesis are meant to make us sweat ;)

Dear Miranda Statistical significance is the number, called p-value tells you the probability that a result is observed, given that certain assumptions (the null hypothesis) is true. If the p-value is small enough, the experimenter can safely assume that the null hypothesis is false

Dear Miranda,

If the design of the experiments and the sample size is appropriate, then the conclusions is OK. You should note however that we pose a statistical hypothesis not to accept but to reject if possible. Rejecting is more sure than accepting! You should also have in mind the little book, How to lie with Statistics.

http://lib.freescienceengineering.org/view.php?id=728185

Dear Miranda,

only very briefly: I am no Professor at all (not only due to my lousy statistics knowledge, I guess...(:-)) so it suffices to call me since I also am a very informal person), Secondly: "sticking to RG can become addictive...especially for learning" but you are right with "GREAT" (:-)) all best wishes and regards,Wolfgang

Dear Miranda, first of all you make your prayers! Then:

1)You check if your number of data exceeds the 'magic number' of 30 (don't ask why)

2)You check every computation to be accurate, just for reproducibility reasons

3)You try to see if your data follow any one of the most known probability laws, ie normal or in general exponential family, uniform etc and take care about the continous or discrete random variables you have used

4)You use a second statistical package, for extracting the same numbers

5)If all above are OK, then you can modestly claim that the error of type I or alpha that you make by rejecting the null hypothesis when that is true, is less than 5%, so we can reject the null hypothesis in a relative safe.

And after all: Good luck! (In Statistics it is always possible to get in trouble, even if you did all as you better know to do ...)

Basically, a significant result means that you can reject the "null hypothesis" that the relationship you observed was solely due to chance (at p,.05, there is only one chance in twenty that a relationship this strong could have occurred randomly).

What you have to avoid is assuming that your substantive hypothesis was correct, just because you minimized the possibility that the result was due to chance. In particular, there might have been any number of other things that caused relationship. So, you do have an increased reason to believe that a relationship does exist, but not necessarily for any of the substantive reasons you have stated as the basis for your original hypothesis.

Also, the size of your sample affects your ability to find a significant relationship (which is known as the "power" of the test). For example a correlation coefficient (or a chi-square value etc. etc.) might not be significant with only 50 observations, but it almost certainly would be with an N of 5000.

This raises the question of "statistical significance" versus "substantive significance," because a large sample can show statistically significance for relatively small effects. Thus, these effects are not zero, but they may not be large enough to make a meaningful difference. This is especially important when you are working in an applied area, and you want to know which variables have a major impact on the situations that you are trying to change or improve.

As per my experience p-value is always debatable, my request never generalize p-value you should interpret basing on the results but not just below 0.05, sometimes even you get below 0.05 either you can accept or reject null hypothesis. As mentioned by Mahfaz it should be statistically significant. That is wonder of statistics.

Thanks @Costas, Gonzalo, Wolfgang, Demetris, kamal, David, Jetty.

Yes I agree: 'Rejecting is more sure than accepting! '

Yes, Prof Kamal: time is a better test :)

'If you have significant results and you publish the study, this is good. In the future another author will collect the data from many publications and pooled the data and analyzed it to see if this significance remains valid'.

@Wolfgang, the learning on RG is great, even on Q and A. But I can't get addicted, too busy...

Usually I report p-value, and also effect size. Thanks to all of you :)

I think the most important concept here, which was touched on by some of the contributions above, is that a p-value is a function of sample size. No specific ('simple') hypothesis is exactly true. If you collect enough information, you can reject (fail to accept) at a given level. On the other hand, i have seen small sizes used which virtually guaranteed that a null hypothesis would not be rejected. The power is sometimes considered, but usually just to pick a test to use, rather than in analyzing results. Thus picking a level for decision making should at least mean tailoring the level, based on the sample size, not just using 0.05 all the time. The link below considers a practical interpretation of a p-value.

This is why I think it is much more important to use confidence intervals than hypothesis tests, whenever possible. Just having a standard error for a point estimate can be much more useful than an hypothesis test.

Article Practical Interpretation of Hypothesis Tests - letter to the...

Thanks very much for your paper @James.  I have downloaded it.  Dear friends, you can see from the tone of this question that it is REFLECTIVE or metacognitive.  But some very superior person has come along and put downvotes on many posts.  Dear downvoter, why don't you share your views that we would like to hear?  Don't you ever doubt your own ideas and thoughts, and don't you ever question the wisdom of doing something?  Is it stupid to ask ourselves such questions?  We wish to hear from you.  Thanks.

If I did a research, analyzed the results, and got a p-value of less than 0.05, I reject the null hypothesis and accept the alternative one. This means that there is a significant effect of the independent variable on the dependent variable.

Dear friends and colleagues, as with regard to numerous "downvotings", guessing in a "batch" style, in this thread (concerning also my own replies) and with regard to Miranda's post above ("waiting for the downvoter's reply") I would like to inform you about the following (apologize for lengthiness):

Tuesday, 29.07.2014 I wrote to the Community Support of RG the following:

Downvoter is a coward! Hater! "Small" man! Some of them need real help! Seven contributors to this thread were downvoted. As @Wolfgang suggested, I will not  be surprised at all if I et downvoted. As RG Community support mentioned: "Downvoting a question is an anonymous action that signals poor quality or irrelevant content to the ResearchGate network. ...The same principle applies to low-quality contributions! ", my question is :"How to prevent malicious mass downvotes?" RG may include filter that prevent downvote without  at least, one answer to the thread! It may be a part to a solution. Downvoting an answer/question will still remain anonymous, but I do hope that it will not so massive!

My protest and disagreement will be done by upvoting the downvoted threads! Where is the end!

Dear Ljubomir,

I know a very easy way in order to make harmless a down-voter: all the other participants of the tread should up-vote the down-voted persons’ comments. Thus, the down-voter attains even the opposite of his/her original intention: deteriorate anonymously others.

You read my mind dear @Andras! I do support your proposal,yes: "all the other participants of the tread should up-vote the down-voted persons’ comments." I doubt that this proposal will be accepted!

Dear friends, thanks very much for support and solidarity.  What we find isn't: "Downvoting a question is an anonymous action that signals poor quality or irrelevant content to the ResearchGate network. ...The same principle applies to low-quality contributions! "  

But we found downvoting practiced by malicious people to put others down. (I'm glad you noticed that even Prof Kamal was downvoted!  I would say that Profs Kamal, Ljubomir are among the supporters and upvoters of MANY RG members!  Why do people downvote even those who upvote them?)

Dear Miranda, returning to your original posting on the thread, I copy here a link to an article published in Nature early this year, which is relevant. In a very nice way it discusses the motivation behind using P-value (as introduced by Fisher) and the pitfalls of its use and interpretation.

http://www.nature.com/news/scientific-method-statistical-errors-1.14700

Thanks dear Bukovinsky.  It's a great link!

'The irony is that when UK statistician Ronald Fisher introduced the P value in the 1920s, he did not mean it to be a definitive test. He intended it simply as an informal way to judge whether evidence was significant in the old-fashioned sense: worthy of a second look. The idea was to run an experiment, then see if the results were consistent with what random chance might produce. Researchers would first set up a 'null hypothesis' that they wanted to disprove, such as there being no correlation or no difference between two groups. Next, they would play the devil's advocate and, assuming that this null hypothesis was in fact true, calculate the chances of getting results at least as extreme as what was actually observed.'

Another point of view of the great scientist Bruno de Finetti.

http://ecodevoevo.blogspot.it/2012/01/probability-does-not-exist-part-i-very.html

Thanks very much Maurizio, for the link!

'PROBABILITY DOES NOT EXIST

The abandonment of superstitious beliefs about the existence of the Phlogiston, the Cosmic Ether, Absolute Space and Time, . . . or Fairies and Witches was an essential step along the road to scientific thinking. Probability, too, if regarded as something endowed with some kind of objective existence, is no less a mis-leading misconception, an illusory attempt to exteriorize or materialize our true probabilistic beliefs.

'In other words, the idea is that probability is not part of the real world, only of one's belief in the nature of that world.'