Two general points

1 fixed effects are interpreted just as in standard regression

2 if I have enough occasions, I would allow the effects of time to be random at the higher level - ie random slopes for time. The differences between the level 2 individuals may change over time, for example increasing differences between them.

This is a simple step by step analysis:

http://www.bristol.ac.uk/media-library/sites/cmm/migrated/documents/repeated-measures.pdf

This is a good site in general

http://www.pilesofvariance.com/

More specifically I would not the fit the model that you are. It is generally accepted that you must have main effects and lower order interaction in the fixed part before entering higher-order interactions: so I would usually fit a sequence of models of growing complexity.

Model 1 : to get the nature of what needs to be explained- the unconditional model

Fixed + (Random)

1a Intercept + Time + (random intercept)

1b Intercept + Time + (random intercept + random slope of time)

Then introduce the variables to see how the unexplained variation at each level changes

2a 1b + Experiment

2b 2a + gender

2c 2b + Experiment * Time

2d 2c + Gender * Time

2e 2d + Gender*Experiment

2f 2e + Gender * Experiment * Time

Of course you could do things in a different order depending on your research questions with the full modelling looking like

2f Intercept + Time + Experiment + Gender + Experiment * Time + Gender*Time + Gender+Experiment + Gender * Experiment * Time +

(random intercept + random slope of time at level 2) + random term between occasions.

It maybe you also need to consider non-linear effects of time - so include Time + Time*Time.

Once you have got a model that you you think is appropriate (this should have course been spelled out in the protocol!) then the easiest way to interpret the results is to draw a graph of the predicted values based on the fixed part estimates and plot these against time. You should see in your case 4 lines representing the 4 groups FemExper MaleExpe, FemControl MalControl. the confidence intervals could also be plotted around these lines.

What package are you using, and are you assuming days of opiod use is Poisson? Also, how many measurements do you have per person?

Should follow standard significance observations for focal parameter (variable) effect estimates.

Hope this helps

Matt

Two general points

1 fixed effects are interpreted just as in standard regression

2 if I have enough occasions, I would allow the effects of time to be random at the higher level - ie random slopes for time. The differences between the level 2 individuals may change over time, for example increasing differences between them.

This is a simple step by step analysis:

http://www.bristol.ac.uk/media-library/sites/cmm/migrated/documents/repeated-measures.pdf

This is a good site in general

http://www.pilesofvariance.com/

More specifically I would not the fit the model that you are. It is generally accepted that you must have main effects and lower order interaction in the fixed part before entering higher-order interactions: so I would usually fit a sequence of models of growing complexity.

Model 1 : to get the nature of what needs to be explained- the unconditional model

Fixed + (Random)

1a Intercept + Time + (random intercept)

1b Intercept + Time + (random intercept + random slope of time)

Then introduce the variables to see how the unexplained variation at each level changes

2a 1b + Experiment

2b 2a + gender

2c 2b + Experiment * Time

2d 2c + Gender * Time

2e 2d + Gender*Experiment

2f 2e + Gender * Experiment * Time

Of course you could do things in a different order depending on your research questions with the full modelling looking like

2f Intercept + Time + Experiment + Gender + Experiment * Time + Gender*Time + Gender+Experiment + Gender * Experiment * Time +

(random intercept + random slope of time at level 2) + random term between occasions.

It maybe you also need to consider non-linear effects of time - so include Time + Time*Time.

Once you have got a model that you you think is appropriate (this should have course been spelled out in the protocol!) then the easiest way to interpret the results is to draw a graph of the predicted values based on the fixed part estimates and plot these against time. You should see in your case 4 lines representing the 4 groups FemExper MaleExpe, FemControl MalControl. the confidence intervals could also be plotted around these lines.

Thanks Kelvyn Jones . Would you still include the main effects of randomization and gender when those are not of interest for our research questions? Other than seeing that randomization failed/was successful at baseline.

Can you explain why order matters? (" Of course you could do things in a different order depending on your research questions with the full modelling looking like "). I wasn't involved in the creation of this study, the protocol I was given doesn't have enough detail to go off of.

If you add interactions you also have to have (in my view) the main effects as it is very difficult to interpret without. It is then very clear what each term means eg intercept is baseline for males who are in the control group ; and then everything else is a differential from that so that in the final equation the parameter associated with Exper is the differential at baseline for females, and that associated with Exper*Fem*Time is the differential change for those who are females in the experimental group. The plot of the predictions should be very insightful. I would code time so that baseline is zero, so that the intercept is the value of of the response at the outset.

https://stats.stackexchange.com/questions/11009/including-the-interaction-but-not-the-main-effects-in-a-model

https://psych.hanover.edu/classes/ResearchMethods/Readings/Main_effects_and_interactions.pdf

https://www.theanalysisfactor.com/interactions-main-effects-not-significant/

I think it is revealing to build the model in stages as you can see how terms change as you include new/ more parameters. My preferred strategy is to have a bottom-up exploratory approach informed by theory, but often the theory does not guide on specifics: see last chapter of this

Book Developing multilevel models for analysing contextuality, he...

Hope this helps

Hi. I'm late to the party but a very good set of notes on these models is at this link:

https://www4.stat.ncsu.edu/~davidian/st732/notes.html

Best wishes, D. Booth

The redundancy for males is because there is just one degree of freedom for male/female. You find the female coefficient (slope) and you know that the effect of male. In other words for both of the redundancies you know that the effect of being female is relative to the effect of being male.

Can you tell me how you coded you variables? Without that I cannot know what the regression equation is. Or better yet, could you attach the output from the analysis. Tay will help me to explain why time is negative. Clearly it is inversely relative to some other variable.

Patrice Showers Corneli thanks for your help. Attached is the fixed effects output, is that enough to help explain why time is negative? Female = 0, male = 1 and control = 0 and experimental = 1

I am putting my responses in reverse order as I begin with remarks that are most recent in mulling this analysis.

I think that the problem may be that you do not need the time variable as it seems completely confounded with the dependent variable.

Am I right that you are that you are really wondering what the effect of the treatment is over time and that you are controlling for gender? If so then why not test the proportion of days out of 30 days that each person uses opioids as the observations?

--------------------------------------------------------

Earlier:

I am missing something here because the model that the table implies a model I have never seen.

Are the randomization controls those who did not get the treatment and the randomization =1 the ones who did get the treatment or is it the other way around?

More to the point, why does a single variable (time) have more degrees of freedom than the following models which are more complex and should have more degrees of freedom? This should be closest to the smallest model with no parameters.

-----------------------------------------------------------------

Even earlier:

Usually a linear model is a variation on the following. This would be a first good step.

Generally you should also have main effects for randomization and gender so that your output reveals the extent to which these two variables, by themselves affect (or do not affect) the outcome (as Kelvyn points out). Then you should add a two-way interaction term for each of gender*time and gender*randomization before you do the three way one.

What you are really aiming to do when you do any kind of regression, is to determine what amount of the data's variance is attributable to each of the three explanatory variables and their two-way and three-way interactions. So if you fail to put in all three main effects (separate terms) and all interaction terms, then you cannot know all sources of the variance and you will misinterpret your analysis.

So you have two categorical explanatory variables and one continuous explanatory variables (time).

So you should have:

Main effects:

Time

Gender

Treatment

Two way interactions:

Time*Treatment

Time*Gender

Treatmentl*Gender

Three way interaction:

Time*Treatment*Gender

The linear model then is:

OPIOID.Use.Days = T + G + Tr + T*Tr +T*G + Tr*G + T*Tr*G + ɛ

where ɛ is the random error (residual variance not explained by the model).

Right now you leave some of the variance unexplained because your model does not account for all of the possible sources of variance. It is the variance around the mean that tells you the significance value and the significance value may change if you do not have all of the main effects and the interactions.

You presently have :

OPIOID.Use.Days = T +T*Tr + T*Tr*F + T*Tr*M + ɛ

which I don't understand because the Gender variable is coded as a single indicator variable. So you should not have a different coefficient for the males separately from females. And you should have a single three-way term.

So maybe you have somehow separated the response variable into two groups. One that continues to use and one that does not continue to use at all after the treatment starts because that set did not need the treament.

Can you unwind a little. I am guessing but is the response and time something like How many days of did you consume opiates in the last month, and then you asked this every month of the same person for a number of months. So time is a repeated measure of same person Yes?

Can you then give counts of just the predictors so that we can see that they are all there

ControlMale ControlFemale ExpM ExpF

Time 1 ? ? ? ?

Time 2

Time 3

.....

TIme Final

You may have to change Time into a categorical variable. I am asking this to make sure about Patrice's point about time and response confounding.

If I am correct then you have a response that is a count which cannot go negative and is likely to be positively skewed. Such data is usually analysed by NBD/Poisson mixed model. It would be useful to see a histogram of the response separately for each of the 4 groups across all time periods.

It would then be good to go on and give us the mean (and median) for each of the 4 groups for each time period separately, as that is what you are modelling in the fixed part . the median mean will inform about the skewness.

I also re-emphasize my earlier points about building up from a simpler model to see what is going on.

Thanks Kelvyn for the clarification on design and on the proper distribution for the response. Certainly the responses are not normally distributed but Poisson or logistic.

My head was spinning yesterday and you have very nicely clarified it. Yesterday I tried to graph the data from the intercepts and the slopes but that did not help.

I note here that preparation of the data to check for peculiar outcomes could take considerable time but should prevent spurious bias. End up with a matrix that makes sense. Generally I would say that cleaning the data and exploratory analysis typically account for at least 80% of any analysis that I do,

Get familiar with the data using exploratory analysis including summary tables and especially graphical representations of the data. Scatter plotting the responses against each explanatory variable will give you a very good sense of the effect of each variable on the response variable. You will also readily see before the analysis where you expect to find major interactions both two and three way.

Now you should proceed as Kelvyn suggested getting to know your data further with simple models (regress each explanatory variable against the responses (main effects) and keep the table of slopes and significance. Than, proceed with a stepwise regression from the simplest models until you get to the full model (forward stepwise) and/or proceed from the full model backwards to the simpler.

You should notice the interactions as the coefficients (slopes) for each explanatory variable change depending on the order the you enter them and depending on how many variables you include.

All of this is exploratory analysis leads to a robust model that you can finally plug in and run in no time.

I think answers for my questions at the start of this thread could make clear what is being asked. One reading is that a glm with Poisson residuals would be a start, but I couldn't tell from the question.

Thanks for all of the help. I haven't had any formal instruction on how to use multilevel modeling framework as repeated measures, so I've been trying to teach myself (my masters program focused on the ANOVA framework).

Yes we ask how many days out of the past 30 they have used opioids. This was asked at 0, 4, 8, and 12 months. Kelvyn is correct, gender is a predictor (not being controlled for). I did explore the data before I ran the analysis (attached is the frequencies and distribution by gender/randomization), it was positively skewed and had many missing data points which is why I wanted to try multilevel modeling/ mixed effects model. I thought that it modeled individual's slopes and so the skewed data would not be affecting the results as much (this is incorrect right?).

I haven't rerun the model yet to account for the variance I wasn't explaining before; trying to figure out why time is negative (seemed incorrect) before re-running so I know I was entering time correctly. But it looks like linear mixed models won't work. How would you run this analysis so that Poisson regression can be used? Would generalized linear mixed models would need to be used?

Thanks again

A linear mixed model won't handle the skew - it handles the nested structure but still assumes normality of residuals. So it does look like a count model is more appropriate. My guess is that there is also overdispersion so you'd need to consider a model with a dispersion parameter (sometimes termed quasipoisson) or a negative binomial type model. Overdispersion is often common when you get clumpy data like this because the days of opiod use are not randomly dispersed.

You also have a lot of zeroes - that's common in data with low mean counts but you might also consider a zero-inflated or hurdle model. My experience is that these are easier to handle in other software than SPSS (e.g., I use R). However, it is hard to be sure from graphs alone - so it is possible that a negative binomial or over dispersed poisson model would be just fine.

Olivia, thanks so much for your beautiful tables and graphs and for your clear restatement of the problem. I'll take a look. I am enjoying the process.

Dear Olivia;

You don't need prediction here, because your data insufficirnt and also your model just built to appear the effect of time as random factor (WITHIN SUBJECT). So, i suggest you consider female as standard, and from your results it showen that just the final interaction is significant. This mean that as concludion or final expalnation, the male respond to the drug over the time.

https://academic.oup.com/icvts/article/26/4/539/4904208

http://www.bristol.ac.uk/media-library/sites/cmm/migrated/documents/repeated-measures.pdf

To better understand the structure of the data, I redid your fine tables so that visual comparisons could be easily seen by me.

I found an article which discusses this problem: Article A Tutorial on Count Regression and Zero-Altered Count Models...

The code to analyze different types of RM count data can be found here: https://www.researchgate.net/profile/David_Atkins2/publications (the link in the article is corrupt)

Well done. Thanks for sharing.