Hello Onur,

Really important question. I hope you forgive the brevity of my comments and read the 3 reference I indicate

Although my interests are public health related, before getting in the specifics of the model itself, I would first consider these points: 

The p-value inferential mechanism is not well-understood and thus worth summarizing. Under the null, the coefficient of the linear model is zero, tested against two other possibilities: a positive or a negative coefficient, given the data. The p-value (probability value) is the probability of obtaining a positive or negative estimated coefficient or other quantity when the null hypothesis is true. It is not the probability that the null is true. The interpretation of p-values requires knowing:

·       The distribution function (which we take to be continuous and thus is a density function), normalized so that the area under it is exactly 1.00. You have that distribution.

·       A choice regarding the one sided or two sided test itself (the two-sided test rules in two alternative; the one-sided test rules in only one alternative), This is something that you develop from theory (it seems that you are not doing data mining as you have a very clear idea as to the the stochastic process you intend to deal with).

·       A test statistic that  determines the probability associated with the p-value itself calculated from the data.

·       State a theoretical p-value (this is the choice of the probability number that – according to the researcher -- rules out the null as being due to chance).

BUT: Sterne and Smith (2001) state that:

 If only positive findings are published, then they may be mistakenly considered to be of importance rather than being the necessary chance results produced by the application of criteria for meaningfulness based on statistical significance. As many studies contain long questionnaires collecting information on hundreds of variables, and measure a wide range of potential outcomes, several false positive findings are virtually guaranteed. The high volume and often contradictory nature of medical research findings, however, is not only because of publication bias. A more fundamental problem is the widespread misunderstanding of the nature of statistical significance, and hence the use of the p-value alone.

It follows that the p-value tells about one third of the answer. I suggest you consider the fuller aspects of the p-value by considering type 1 and Type 2 error rates. (a and b are probabilities alpha and beta)

Decision Rules to be considered extend the p-value as follows:

The null H0 is true (there is no effect) against the alternative HA is true (the effect occurs):

Reject the null Type 1 error, probability = a. HA is true with probability b.

Accept the null with probability 1 - a. Type 2 error, probability 1 - b.

My comments are a short but the idea should be useful after a bit of work I would encourage you to read these references before relying on the p-value alone. These papers give you the remaining third of the full answer:

Sterne JAC and Smith GD, Sifting the Evidence - What is wrong with significance tests? BMJ 322:226-231 (2001)

 Ioannides JPA, Why Most Published Research Findings are False, PLOS Medicine, 2(80:2124 (Aug 2005) http://dx.doi.org/10.1371%2Fjournal.pmed.0020124

S Goodman and S Greenland, 2007, Assessing the Unreliability of the Medical Literature: A response to why most published research findings are false, Johns Hopkins University, Dept. of Biostatistics Working Paper No. 135, 2-28, 2007.

Best regards and thanks for stimulating some thinking!

Paolo

Dear Professor,

Thank you very much for your informative and illuminating comments. I will carefully examine the papers that you suggested.

Yours Sincerely,

Onur