You can define before and after as a dummy variable (0, 1) and apply ANOVA to see how much variance in each dependent variable represented by this dummy variable (independent variable. You can use minitab or spss.

You may conduct factor analysis if variables are intercorrelated in order to catch a latent variable covering the concept.

Thank you. Anova will not be of much help here, because the design is unbalanced and incomplete. Technically, the generalized linear model (GLM) with AIC will be able to solve the choice of the minimal number of significant variables, and therefore, my question was more about how to ignore the differences between stations/years in the population means when answering the before-after question.

For example, would it be wise to normalize all data with zero mean unit variance for each station and year, and thus bring them all to the same level that would allow testing the before-after difference?

I am also concerned with the number of observations in the dataset that might not be sufficient to have 4-5 explanatory variables in the model. Since this will be a re-sampling of the population, the year must be kept in the model regardless of its significance...or?

I think you can keep "stations" and "years" data in the model along with the before /after variable. Also you can normalize the variables if it is technically and logically necesarry.

Yes, ANOVA will not be particularly good here.

Without knowing more about the structure of your data (though you described it very well for the purpose of your question), I am taking a bit of guess. Hopefully, this is helpful.

So, it seems like you want to test for an overall effect of the event (pre-event versus post-event) while accounting for random differences due to station and year (and perhaps species ID)? The first thing is that with only two repeated points, the treatment (pre versus post) has the same number of levels for year (n=2). If you had multiple years before and multiple years after the event, including year as a random effect--or, better yet, including some ecological relevant factor that varied by year as a random or fixed effect--would be a good idea. With your current design, you're effectively considering "year" in your comparison before and after the event. Thus, now you're only wanting to know the impact of the event from pre to post-sampling periods. What about simply specifying station as a random effect (as I think you implied) and then specify time period as a within-factor measure within station. This will account for the clustering and unbalanced nature of your data somewhat. Of course, since each station has its own species composition, I think it's good to nest species ID within each station ID, rather than specifying species ID as a crossed random effect. Then, since you want to test for an effect of time (or event), specify that as a fixed term also. So, if you're using R (and lme4 package), the specification would be something similar to:

lmer(y~time+(time|station/species),data) # with appropriate error structure

You also mentioned 4-5 explanatory terms. Are these available for each site. What are they? The primary thing, which you already know, is to ensure that your global model (all terms) is run on a dataset that has no missing values. This will ensure that the reduced models are run on the same dataset and therefore can be compared. This is probably obvious, so please forgive me if I'm addressing issues with which you're already familiar. From your description, you have between 240-720 rows of data (3 stations x 2 periods x 4 species x 10-30 individuals/species). You can definitely include more fixed effects than "time."

Re: missing data. Don't worry about the missing data. Your question is not addressing changes in species-specific values before and after the event. You're effectively asking if the event influenced the response of all species, after accounting for variation due to station and species ID.

Re: inter-correlation of variables. Are these the response variables? You can run separate model selection on each of these variables (if this makes sense biologically). An alternative option is to conduct a PCA (like Ufuk suggested), or an ICA (indepedent components analysis), to simplify your analysis. Interpretation might be more difficult though.

Anyway, these are my thoughts about your model.

İf you have missing observatory data, you can check the book of Montgomery "Design and Analysis of Experiments". Montgomery advises some experimental design prcedures for limited trials in experiments.

Dear Elena, in my opinion you can apply split -split-plot design with three random factors (3*4*2) (missing values will be treated properly). In this design, set the less important factor as your main-plot, the next as subplot, and the one you need the highest precision of calculations as sub-subplot factor. If your interest is change in time, then set it as sub-subplot factor. In this design you can measure the interactions too, fortunately.

Notice that you can't use repeated measures for such data.

If you need more info, just let me know.

Cheers

The split-split plot design can be implemented using a hierarchical random effect (species ID nested within station ID). The two time points at which species at each station are sampled is where the problem arises. Repeated measures designs are designs in which "subjects" are measured repeatedly. In order to decide if a repeated measures (time as a within-"subjects" factor) is appropriate, you need to determine simply if any subjects overlap between the two time points. In Elena's case, the "subject" is not the individual but rather than population of a species at a station. If one assumes that absolutely no individuals are measured longitudinally at both time points, there is no reason to adopt a repeated-measures design. However, if there is a possibility that that any part of the population is measures longitudinally, it is necessary to account for that clustering of the data. Thus, it most certainly helps (and, more importantly, does not hurt) to specify time as a within-"subjects" factor (within species, nested within specific station), which is essentially a repeated measures design.

Thank you very for all your suggestions! I realize now that was not very specific when describing the data and the model that would be most relevant (to my understanding). All 4 species occur at all 3 stations and the measured parameters are stable isotope signatures (d13C and d15N), body weight and C:N ratio. We are interested to see if there is a change in the animal d15N as a result of a phytoplankton bloom. In each species, the d15N values are positively related to body weight and C:N ratio and negatively to d13C (all correlations are significant). Therefore, when testing the event (=bloom) effect on the d15N, the body weight, C:N and d13C must be included in the model as potentially important covariables.

I would like to ask again - what will happen if I remove the station effect by calculating z-scores (zero mean, unit variance) for each station (and year?) and, of course, species, and use these z-scores as variables? Then, I don't have to include the station term in the model, or?

I am not sure I need to include any interactions as long as the effect of the dummy variable for "before-after" as a main factor is significant... Again - neither year nor station (as well as any interactions that involve these factors) are of interest. The species effect is going to be significant and therefore, a separate model needs to be fitted for each species anyway.

Regarding the number of variables: altogether, it is going going to be at least 5 variables (the body weight, C:N and d13C, year, "before and after") for model with 70-100 observations... it is OK, but I cannot afford any interactions.

Elena, the analysis of variance is not what you think it is. You have to extract the significant effect of any sources of variation from your data,because if you don't, it would be considered as error. So, even the interested effect would not show significant effect. To include your covariates you should do an ANCOVA.

Besides, you didn't explain your variables properly here. Do you have separate "before and after" data for each year?

I am talking GLM (using both categorical and continuous variables), with log-link (ANCOVA does not allow this error structure) and I am aware that all significant terms must be included.

As soon as the data are normalized to the station mean, the effect of the station would naturally become insignificant and therefore omitted from the model...The same goes for year. What would be methodologically wrong with this approach?

I think I have specified that before and after data are available for each of the two years (except one species, where it is availble for one year at one station and 2 years at 2 other stations).

By doing so, in my opinion, you can't remove the different effect of different stations on different species (station*species interaction), and it would added to your error term.

Thank you, Ehsan!

The station*species interaction I can live with:) Because the effect of continuous variable (weight or C:N ratio) would differ among the species, to my understanding it would be more straightforward to run GLM for each species separately anyway.

What would be other objections to such normalization?