Hello Emad,

I would always start with exploratory data analysis, mainly be creating scatterplots, see the pattern, potential outliers, potential nonlinearity.

Then, what did you do? I assume you did a traditional moderator analysis with a product term? If yes, what was the p value? If it is close to .05, increasing the N would *probably* add power to show a significant effect--but not necessarily: If there is not true moderation in the population, the coefficient of the product term will converge to zero with increasing N. But even if a huge N finally leads to a significant moderator effect, the effect itself will be practically irrelevant despite the statistical significance.

Perhaps you have to finally deal with the fact, that there is no moderation.

Speak with you supervisor if this would be a problem (which it should not be as dissertations should do proper scientific research and not deliver some significant results at all costs).

Best,

Holger

Did you you collect 201 responses, analyzed the data and then collected another 299 responses or only 98?

Thanks, Rainer Duesing for your question.

I collected 201 responses initially; another 98 in the second round; all together, 299.

I did the analysis with 201 responses, which revealed insignificant moderation.

I repeated the analysis with 299 and ended up with the same results.

thx

Thanks, Holger Steinmetz for your comments.

In fact, I started with data screening (missing values were handled, normality outliers, and CMB were all done). I ended up with 299 usable responses.

below P value results in bold, which showed highly insignificant moderation

AI x SCREEN -> SCP 0.400 AI x SCC -> SCP 0.245 AI x SCI -> SCP 0.248 AI x SCA -> SCP 0.275

thx

As Rainer Duesing 's question was getting at, you can't combine data like that within a frequentist statistical approach. You would treat as two separate studies. On getting results that support the model that you are testing (i.e., getting a non-significant result), the value of the study depends on how strong the study is of a test of this model (i.e., Popper's LSD). On removing outliers based on "normality", I'd be careful since as @Holger says this may bias results. Better to use more robust statistical methods. Only remove data that are not part of population of interest (e.g., I do stuff with response times, so if someone sneezes I am not interested in testing cognitive processes while sneezing).

Thanks, Daniel Wright . that's really helpful

Emad Isaid you cannot gather samples, check for significance and continue to gather an additional sample, add them up and check again. That is cheating. There are methods for sequential analyses in the frequentistic realm (but I dont think you used them) or you need Bayesian methods.

You may ask 'why' this is cheating. Parameter estimtates are fluctuating around the true value. Now, imagine an extreme example: you conduct your test after each new response/participant and check for significance. At some point the estimated value may be very large, just by chance, and significant. You would stop your analyis at this point, since it is significant. But this introduces a bias to yor analysis, since you would only stop at random large effects, but not at random small effects and this is a form of capitalizing on chance.

Therefore, you cannot simply combine your two samples, if you already conducted your analysis at the first sub-sample.

Thanks, Rainer Duesing for your input.