Erikson, Jay and Saad:

I think all the suggestions of using the paired t-test forgive an important aspect of the question asked by André: "What is the best way to test the intervention's effect accounting for socio-demographic factors (age, gender, number of sessions attended etc?". This is not answered with the paired t-test. 

Some of the interventions before have considered this aspect. 

Jorge Ortiz

ISSUE: Statistical test for the significant effect of stimulus

PAIRED T-TEST: This test is a test of location, i.e. comparing meanS between two groups or two samples. The samples could have the same size or difference sizes. Generally, this type of test is used where the data are taken from the same condition. In the present case, you data comes from different condition. Therefore, paired t-test may not be appropriate. In addition, the t-test assumes that the data is normally distributed. What if your data is not distributed? You would need to verify whether the data are normally distributed.

ALTERNATIVE TEST-1: You are seeking to verify the effect of the intervention, the test's must have a counting BEFORE the introduction of the stimulus (intervention or treatment) and the AFTER measurements. This type of measurement involves two elements: (i) the value or data measurements themselves, and (ii) time factor. The first alternative is the Two Counts Poisson Distribution Test.

(1)   Z = (R1 - R2) / sqrt((R1 / t1) + (R2 / t2))

... where Ri = frequency count which is determined by Ri = Ni / ti; and ti = time period. N = number of counts. Set your confidence interval, i.e. CI = 0.95; then the hypothesis statements follows: H0: Z(obs) < 1.65 means that there is no significant effect. HA: Z(obs) > 1.65 means that there is a significant produced by the intervention. Use the Z-table for this test.

ALTERNATIVE TEST-2: the second alternative may be a test for inconsistency among the two samples (before and after). One means of accomplish this task is to construct a 2 X 2 table and take the counts for the two samples.

---------------------------------------------

                       class1          class2

Sample1         a                    b

Sample2         c                    d                      ... where n = a + b + c + d

----------------------------------------------

(2)   X2 = [(n - 1) (ad - bc)2] / (a + b)(a + c)(b + c)(c + d)

This second test is based on chi-square distribution. using the degree of freedom (n - 1) above to compare to the critical value in the chi-square table. The rationale of this test is that if the two samples are consistent, it means that they came from the same population, i.e. no significant effect to differentiate sample1 from sample2. Use the chi-square table for this test.

ALTERNATIVE TEST-3: A third option is to use the F-test for two counts Poisson distribution. recall that from the above definition N = counts of the predefined effect. The frequency of these counts occurrence is R = N / t. The F-test is given as:

(3)   F = A / B

(3.1)   A = (N1 + 0.50) / t1

(3.2)   B = (N2 + 0.50) / t2

... where the degree of freedom is defined as df1 = (2N1 - 1) and df2 = (2N2 - 1). Use the F-table for this test.

DEMOGRAPHIC FACTORS: These are nominal data. You can use categorical counting, i.e. define your category of interest (male or female). See chi square routine above.

REFERENCES: See Kopal Kanji's book on relevant statistical tests.

http://fcc-statistics.wikispaces.com/file/view/100+Statistical+Tests.pdf

This is to theoretical for me and not helpful. Just needed to know whether paired t-test would be enough (as I do not have a comparison group) and what test to use to adjust the diference between post intervention and pre intervention measurement for confounding factors

No, paired t-test is not enough and may not be appropriate. The test routine described above are quite simple to follow. They are no more complicated than your proposed paired t-test. If you have your data ready, put it through excel. The calculations should not be more than 10 minutes. Cheers.

Hi Andre,

It depends on what you want to know.

If you want to see whether there is a change from time before intervention and after yes a simple t-test will tell you whether there has been a change or not. If you want to see if your intervention has a bigger effect than additional variables (socioeconomic etc) you will need to multivariate tests.

The type of multivariate test I/you would choose depends somewhat on the format of your outcome variable? Is it a BMI change i.e. a continuous variable or is it a categorical variable Response achieved or not. If continuous you could use a multivariate general mixed model. Or you could use a multiple or logistic regressions if you want to compare the effects of the other variables on your outcome and the odds/ risk ratio for achieving a particular outcome or not for example.

I also know some good text books on this, but I guess that is not why you are posting here....

Dear Dr. Renzaho,

It would be useful to know a couple of things to give you a better answer:

1. Would you please tell us what your hypotheses/research questions are?

2. Could you say a thing or two about the characteristics of your data (e.g., scale of measurement, distribution)?

Simplest answer [assuming stat assumptions are met]: For pre-post comparisons, paired t-tests are often appropriate. If you want to account for the influence of demographic variables, however, then ANCOVA or SEM might be your best bet.

best,

Carl

If your dependent variable (say Y) is numeric, calculate DY = YAfter - YBefore as a change measure in your DV.

If you only need to compare a location parameter of DY by sex (for example) a t-test o a Mann-Whitney test is ok. 

But if (as you say in your question) you need to compare DY accounting for age (numeric), sex (dichotomous), number of sessions (random integer), etc., you should use a mixed model. 

best,

Jorge Ortiz

Hi,

If you want the comparing means of 2 paird (matched) samples.  Example: weight before and after a diet for one group of subjects.  Then you can take that the parametric test when the dependent variable is normally distributed - then you have to do the Paired t - test for your data.

If you want to compare the means of two independent groups, and your independent variables are categorical or nominal then you have to an independent t - test for your analysis.

In your study more than 3 independent groups and your dependent variables are contiunous and indepent variables are categorical then you go to one way ANOVA test.

For your study,

you want to compare more then 3 measurements (5 sessions) on the same subject, dependent variables are continuous / scale and independent varaibles are in time variables, then you have to do the repeated measures ANOVA.

Very careful in choosing the statistical tests.

All the best for you.

The paired t-test is a good choice.  Even if there are differences in socio-demographic factors, each subject is acting as its own control.  I would use graphs to show effect differences based on socio-demographic factors.  Testing for differences in subgroups usually does not add much to understanding the data. 

Hi André,

If you decide to apply a paired t-test you compare After - Before mean values (= intervention effect) and with a significant result you say the intervention has been effective. But this does not take account of sex, age, etc. 

If you apply a paired t-test for the men group, with a significant result you say the intervention is effective for this group and the same for the women group. But this does not allow you to compare these two effects and you can not test if the effects are different for these two groups.

I interpret "accounting for" sex as to compare these two effects. The paired t-test is not the way to this end. Please, see my previous answer for a more convenient solution.

Jorge Ortiz

Hi André,

I’m agree with most of the answers about that paired t-test is enough, but there is something relevant in what Jorge Ortiz said, you must question yourself if a significant result of a paired t-test is enough for conclude that the intervention has been effective.

It depends the hypothesis that you want to prove, usually paired t-test is formulated under the null hypothesis that the mean of differences is zero (0), but according to the characteristics of the intervention, you can change this hypothesis to prove that the intervention has been effective if the mean difference is greater than an expected value. In this case, the paired t-test is one-tailed and if the result is significant you can conclude on the basis of effectiveness of the intervention. An easy way to confirm this, is to calculate the confidence interval for the mean of differences, if the interval includes the expected value, the intervention effectiveness is not significant, but if the confidence interval excludes the expected value, then the intervention effectiveness is significant.  With the advantage that the CI gives you more information than a simple p-value

You can run this paired t-test in any statistical software (even with Excel data analysis complement), including the expected value of the mean of differences, be careful on this because some packages makes this automatically to prove that the mean of differences is cero.

Good luck!

thanks heaps, Frederico

Thank you all, your answers are very helpful, especially  Mark Farmen's answer where each subject acts as its own control. ready to analyse the data now.

Hi André,

I would use paired 'T' tests to make overall comparisons of your BMI scores, etc.

To see whether there was a difference due to gender - I would compare the differences (ie post minus pre) in BMI scores for the males, with the differences in BMI scores for the females - using a normal unpaired 'T' test (though if your numbers are small (eg

Hi, André!

Unfortunately, without a control group for counterfactual evidence, it's really hard to make causal inferences. No statistical test is going to solve that.

If want to say something about how effective the intervention was, you will have to make stronger assumptions: e.g., that pre-intervention measures would remain stable without the intervention (or with random or a priori known fluctuation).

If this assumption is somewhat acceptable, the paired t-test will give the significance of the difference for the pooled data.

If you are interested in controlling for co-variates (age, gender, etc.), your best bet is to build a regression model. The most simple model (but not neccessarily adequate) would be something like:

postMeasure = B0 + B1*preMeasure + B2*gender ... + e

The intercept (B0) will give you the expected difference between before and after, together with a test for significance. The main slope (B1) will give you variable treatment effects in function of previous weight.

I believe that  accounting for the socio-demographic characteristics as intervention could be done by test of difference of means between independent samples.It would be impractical (although still physically possible) to make pairs like "male before-female after", "20 years of age before- 21 years of age after" and so on.

I'd tend to agree with Senthilvel, I've been PI on many health intervention projects where paired and independent t-tests have been sufficient, extending this with one-way and repeated measure ANOVAs should suffice for what you need.

Hi again André,

Just after read the other answers and re-read your question, is clear what you have:

First, you have a paired-design with the same individuals measured pre and post intervention, to evaluate the effectiveness of this intervention, I suggest that a one tailed paired t-test is suitable under the null hypothesis that the mean of differences is equal to an expected value and the alternative hypothesis that the mean of differences is greater than that value (calculating the difference: Post - Pre).

Second, if you want to evaluate the effect of other variables (covariables), like socio-demographic factors, the proposal of Erickson makes more sense than other, you can look for a regression model (generalized linear mixed models procedure), that involve all the covariables. But I think that it would be easier if the response (y) of the model is the differences between post and pre intervention, then the model will evaluate directly the effect of the covariables over the variable that can be interpreted like effectiveness.

Sorry if I’m not agree that you can use in this case a repeated measure ANOVA, but I didn't see a repeated measures design in your approach, and this answers can make confusion.

Best regards.

Dear colleagues, I think that everybody is free to express their opinion, but in an academic forum is healthy to discuss our differences and to explain in which point we are agree or not. I received two downs to my previous answer and I read it again and again to see what is wrong with it, which part I wrote something incongruous or I offended someone. The only part that I noted that is against the opinion of someone is related to repeated measures ANOVA, but if you read the André’s question he only did two lectures (pre and post) and this not corresponds to a repeated measures design.

It is supposed that this forum is to help and orient researchers about their questions, and is etic that the answers will be oriented in that order and not to create confusion.

Best regards.

Erickson is incorrect.  If you participants serve as their own controls a paired t-test is appropriate.

If you want to control for confounding/other risk fsctors use some sort pf multivariate regression

Hello, Jay!

I agree with you. Indeed, this is what I said: we need stronger assumptions. If you assume that each subject can serve as a control for himself, the paired t-test is OK.

The problem is: in a fully randomized experiment with control and treatment groups, this can be somewhat taken for granted thanks to randomization. Without a control group, we have to assume "that pre-intervention measures would remain stable without the intervention (or with random or a priori known fluctuation)."

Is this assumption acceptable? I don't know. BMI and habits are somewhat stable, but we can't be sure that our sample happened to regress to the mean, and we interpret this as an outcome of the treatment.

Hi Ericson:

What can be a better control than the same person pre and post

J

To: André Renzaho

Paired t-test is standard and well accepted. However, is the normality assumption valid for your data? If not, should use a  nonparametric test such as the Wilcoxon signed-rank.

Erikson, Jay and Saad:

I think all the suggestions of using the paired t-test forgive an important aspect of the question asked by André: "What is the best way to test the intervention's effect accounting for socio-demographic factors (age, gender, number of sessions attended etc?". This is not answered with the paired t-test. 

Some of the interventions before have considered this aspect. 

Jorge Ortiz