Hi.
Just looking for some advice/help.
We've run an interrupted time series analysis and wantvto report the difference between the point estimate and counterfactual at 1 year and 2 years post-intervention.
We want to report such with a p-value and CIs. But such needs to be derived from post-hoc estimation. This is currently done using the emmeans package in R. Said package offers a pairwise comparison with Tuckey adjusted p-value. I wondered whether directly reporting the above p-value was the right approach or can you use that based on the z-score of the estimate and work out the p manually? The former [Tuckey adjusted] yields non-significant results whereas a manual value derived from the z-score of the output is statistically significant.
Assume we're not the first to attempt to report such and wondered if anyone could share how they derived p-values and CIs previously for a point estimate to counterfactual comparison at a specific time point.
Thanks Adeyemo Samuel.
My concern is that I used the emmeans package, which computes multiple pairwise comparisons and give me a p-value of 0.14.
Just using the z-score from such it would give a p-value of 0.03...
Do you know of a good example of posthoc estimation in R? Are there good alternatives to emmeans/resources for posthoc estimation that have been based previously in ITS
Pairwise comparison is a statistical method that compares the pre- and post-intervention outcomes of each group in an interrupted time series (ITS) analysis. In this method, each group is compared with itself before and after the intervention, and the differences between the pre- and post-intervention outcomes are analyzed to determine whether there was a statistically significant change.
To perform pairwise comparisons in an ITS analysis, you can follow these steps:
It is important to note that pairwise comparison is just one of several methods for analyzing ITS data, and it has some limitations. For example, it does not account for potential confounding factors or time trends that may affect the outcomes. Therefore, it is recommended to use multiple methods and conduct sensitivity analyses to ensure robust and reliable results.