Based on your description, the collaborator is correct. ANOVA can only be used with a single DV, and these three different 'factors' presumably measure different things.

You would either need to conduct 3 separate repeated-measures ANOVAs (one for each factor) or conduct a repeated-measures MANOVA.

actually the problem is the kind of data, anova usually works fine with cuantitative factors, if you have qualitative or mixed ones anova usuall is not accurate, for qualitative factors use experiments design with works with matrixes and hilbert spaces, for mixed variables its ever more complex. good day.

Blaine Tomkins , Ivan Avila Caceres Thank you so much！I think I'd better use MANOVA.

Qualitative differences between factors mean their absolute means aren't on the same "Ruler". A simple main effect of factor simply tells you "On average, your participants scored higher on factor 2 (say, "motivation"), than on factor 1 (say, "control")"—but that doesn't yield a theoretically meaningful insight, because those factors tap different psychological processes, even though you averaged both on a 1–7 scale. You've essentially correlated apples and oranges.

Aside from ANOVA (separate repeated-measures) -> use repeated-measures MANOVA to test all three factor scores across your three conditions in one go. [It's more powerful and error-controlling than doing three separate ANOVAs or interpreting the ill-defined main effect of "Questionnaire Factor".]

On SPSS >> Analyze → General Linear Model → Repeated Measures

> define Condition > add your three DVs > request Multivariate tests, and interpret Pillai's Trace/Wilks' Lambda.

I disagree with the previous answers. A two-way ANOVA approach is not wrong per se. It is true that a comparison between the factors might not be useful without other considerations, but there are situations where this might be very insightful.

If you only have raw data, I would agree that differences are not really meaningful, BUT if you use standardized scores (from a psychological questionnaire for example), a difference could be interpreted with regard to the relative values.

Just to give a really artificial example: your three factors are measured on an IQ scale, i.e. mean = 100 and sd = 15. Your test scores from your participants are normed with regard to a norming sample. Then, all three factors are on the same scale, so to speak. Imagine you only have 1 condition (not 3 like in your experiment), then you could analyze if this condition impacts the three factors differently, e.g. F1 mean = 85, F2 mean = 95, F3 mean = 120. All comparisons between the 3 factors and also comparisons to the norming sample would be appropriate and meaningful. So, it depends how your variables look like, if it makes sense.

If you think that you cannot compare the three factors to each other, you should better use 3 separate repeated measures ANOVAs (or linear mixed models aka. Multilevel models) to analyze your data. It is not(!) recommended to use MANOVA, just because you have several DVs. Only if you have a truly multivariate hypothesis (which you don’t have obviously) and you are interested in differences between groups/conditions on the newly, artificially created variate, which explains maximum variance between the DVs, a MANOVA is the model of choice.

It is a myth that MANOVA protects against alpha error cumulation and it does not answer the research questions you are typically asking. Also, the power argument is not necessarily a good one. You are talking explicitly about factor scores. Typically, factors are rotated orthogonally, so that factor scores are uncorrelated in the first place. But a MANOVA only has more power, if all DVs are correlated with each other. Otherwise you have no benefit from using a MANOVA.

Therefore, use univariate approaches and correct for multiple testing, if needed. See the article form Huang F.L. (2020) MANOVA_ A Procedure Whose Time Has Passed? for more information on that topic.

Rainer Duesing Interesting. Does the same hold true for multivariate regression?

I would say yes and no. Since all are linear models, everything said about MANOVA also hold for multivariate regression. For example, if you want to conduct a MANOVA within R, you need to formulate it as multivariate regression in a first step and then analyze the results with a Manova() command, for example from the car package (maybe there are other possibilities, but is the one I know). Here for a simple two group MANOVA, i.e. Hottelings T^2 test:

# linear model

HT2

Rainer Duesing

I think we are arguing about two different points.

=> If we have a case in which each factor is conceptually distinct + we are not testing a single "super–factor/composite" -> The multivariate test then doesn't address any substantive question beyond "Do these three things jointly change?"

->So-> it would be more appropriate to separate DVs, as MANOVA (in such case) doesn't meaningfully control Type I any better than a proper familywise correction on univariate ANOVAs.

->Likewise-> if we don't have a true "multivariate hypothesis" or joint research question about multiple DVs as a vector (e.g., "Does our training program shift the entire profile of cognitive abilities?") => But a different question (e.g. "How does Condition A vs. B vs. C affect each psychological factor?") -> That is inherently a univariate question per factor.

->Also-> in your orthogonal rotation example; in such case with uncorrelated DVs, MANOVA's multivariate statistic will be reduced to something akin to the sum of independent tests (no power advantage at all) -> It would be better off running three focused repeated-measures ANOVAs (or a linear mixed model) with a straightforward Bonferroni/Holm adjustment.

=> However-> When your research question genuinely concerns the profile of multiple related constructs, and you'd rather let the data tell you which combination of factors shifts with Condition, a properly run RM‐MANOVA is not only defensible but often preferable to a battery of univariate tests.

->First-> it significantly protects against α‐inflation > If your multivariate Pillai's Trace (or Wilks' Λ) is non-significant, you stop (no need to probe univariate F's at all). That's a real safeguard against chasing noise in each of the three separate ANOVAs.

->Second-> it conserves power with orthogonal factors -> Even if your population factor structure was rotated orthogonally, in any real dataset, the sample scores will rarely be exactly uncorrelated. MANOVA capitalizes on whatever residual covariance is present, however small > Suppose each factor shows a modest Condition effect (say, F= 3.5 p≈0.07 univariate), but all in the same direction. MANOVA can aggregate that pattern into a single significant multivariate signal (even though no individual ANOVA crosses the 0.05 threshold).

->Third-> You can articulate a genuinely multivariate research question: (e.g "Across these three facets of psychological functioning: control, motivation, and inhibition > does greenspace exposure produce a profile shift compared to urbanspace?") > That profile‐shift question is about a composite of DVs: MANOVA directly tests whether the mean vector differs.

->Fourth-> MANOVA yields distinct canonical variates (linear combinations of your three factors that maximize group separation) > This can reveal, for example, that a particular blend of high F1 + low F3 best distinguishes Condition A from C, even if neither factor alone was a standout.

->Thus-> it's standard to use RM‐MANOVA for multiple ROI or scale subscore analyses (particularly in psychometrics and neuroimaging) precisely because it guards against "fishing" across many related outcomes and illuminates multivariate patterns.

I think I am fine with the first part of your answer Mohamed Abdel-Maboud , but I do not follow your rational for your last 4 points.

MANOVA tests one very sepcific hypothesis: do groups differ on the linear combination of the original DVs? The differences in the original variables do not matter directly and are not part of the hypotheses (why ANOVA is not the correct follow up, see below).

I could imagine some scenario, where it could be of interest, to combine several DVs, but I never saw this in a published article in my research field psychology (maybe there are other disciplines where this is more common). So 'yes', you can articulate a genuinely multivariate research question, but most of the time, researchers in my area are interested in questions like: Do the groups differ on variable A, or variable B, or variable C, since all DVs represent other aspects of the same or even different constructs. Therefore, why starting with a multivariate approach in the first place, if this is not my research question.

And no, it is not a safeguard against alpha inflation for univariate ANOVAs, since ANOVAs are not the appropriate follow up analyses in the first place! To preserve the multivariate analyses structure (and otherwise it would be pointless), you should use discriminant analyses, to investigate, which predictors contribute most to the group separation. But with DA you cannot simply answer the question of mean differences between groups. Therefore, this is not a correct argument. Even the other way around: with low correlated DVs and many of them, you might increase type II error. When there might be only a true difference only in one or two DVs the maximization across all DVs might not detect this. (but I would love to see or conduct a simulation study on this point....)

And even if you would use ANOVAs as follow up to a significant MANOVA, because of correlated DVs, you might end up with insignificant ANOVAs. The p-values of the univariate analyses are in no way influenced by multivariate approach. Therefore, why not looking at the univariate analyses, in the first place. And you cannot compare this to follow up / post hoc tests after an ANOVA with, for example, individual t-tests. Follow up tests use the error term of the ANOVA, which should be more robust than pariwise error terms that you would use for t-tests. Therefore, it makes sense to conduct an ANOVA before pairwise comparisons, because you use information fom the ANOVA. But not MANOVA before ANOVAs, because the ANOVA results are independent of the MANOVA results.

I think your second point is more a bug than a feature: if the true factors should be uncorrelated, but you use sample score and find some spurious correlations, what are you interpreting then? There should be no correlation at all and what you have found is an artificial byproduct of your sampling, therefore, you are fitting noise. That is also a drawback of MANOVA: the created variates are very sample dependent, since they are constructed to maximize the group differences. The variates are atheoretical and may change from sample to sample. So what are we comparing them? Two MANOVAs with the same variables on two different samples may create quite different variates.

So, if you really want to keep a multivariate structure, it may be more helpful to use structural equation modeling, where you can directly measure the weights of each variables as well as controlling for measurement error.

As mentioned above, my view may be very narrow from a psychologist's perspective, but in my reasearch field I haven't found an really good application for MANOVA. It is more a historical relic.

Rainer Duesing

I appreciate this discussion; thus, I would like to follow up on a few points.

=> [MANOVA only tests a linear composite; the original DVs don't matter]

=Comment> Every multivariate omnibus does test a linear combination, but that's precisely its strength > it asks "Is there any pattern in how these outcomes, taken together, shift with Condition?" If your theory is about a profile, say greenspace boosts control and inhibition but not motivation, MANOVA is the only test for that joint hypothesis.

-> Contrast with ANOVA: Three separate ANOVAs only answer "Does Control change?" "Does Motivation change?" "Does Inhibition change?" They never answer "Do these three change in concert?"

=> [Researchers in psychology never really have multivariate hypotheses]

=Comment> Modern cognitive and personality research frequently posits that interventions or traits have profiles of effects. For instance >

In emotion regulation -> one treatment may reduce negative affect but leave positive affect unchanged—MANOVA directly tests that dissociation.

In neuropsychology -> a drug might improve memory and executive function but not processing speed—again, a profile question.

Nonetheless, Journals of clinical psychology, developmental psychology, and even social psychology regularly publish RM-MANOVA for symptom-profile or scale-profile analyses.

=> [MANOVA doesn't protect against α-inflation unless you follow up with DA]

=Comment> That's a misunderstanding of error control. The multivariate test controls the family-wise error across all DVs at your specified α (e.g., 0.05) for the omnibus hypothesis. If Pillai's Trace is non-significant -> you stop—no risk of chasing individual false positives.

-> Discriminant Analysis (DA) is not a required "follow-up" > DA estimates the weights that maximize separation, but you still need univariate tests or contrasts to interpret which original DVs drive the effect -> SEM can model latent factors, but it doesn't replace the need for an omnibus test of the observed profile.

=> [Low-correlated or orthogonal factors kill MANOVA's power]

=Comment> Even orthogonal factors, when measured, rarely come in perfectly uncorrelated > Measurement error, sample idiosyncrasies, and subfactor overlap all generate residual covariance that MANOVA leverages.

-> Not to mention aggregating minor effects > If each DV shows a sub-threshold effect (e.g., p = 0.10 univariate), but all in the same direction, MANOVA can reach significance by pooling evidence—univariate tests would each miss it.

=> [Sample-dependent canonical variates are atheoretical and unstable]

=Comment> You can compute canonical variates in one sample, test their applicability, or correlate them with external criteria in another (just as you would check factor structures or regression coefficients).

-> Also, you can specify a priori contrasts on your DVs (via the GLM's custom hypothesis feature) that correspond to hypothesized profiles, rather than let SPSS data-dredge for you. That attaches the multivariate test to substantive theory.

=> [SEM is better for multivariate structure]

=Comment> SEM and RM-MANOVA answer related but distinct questions.

-> SEM models latent constructs and corrects for measurement error, but you still need to establish whether the observed scale scores jointly differ by Condition in the first place.

-> RM-MANOVA is the classic, robust omnibus when you have fully observed repeated-measures subscales and want to test profile shifts before committing to a full latent-variable model.

-> Not to mention the Practical advantages of RM-MANOVA > Robust to sphericity violations (no ε corrections needed) > Single omnibus at α = 0.05 avoids the conservatism of Bonferroni > Built-in diagnostics (Box's M, Levene's test, univariate follow-ups, profile plots) + Canonical insight: the first canonical root tells you the strongest axis of change, and you can interpret its weights to see which DVs matter most.

=> Therefore, when your research question is "Does the pattern of multiple outcomes shift across conditions?" not just "Does A shift?" or "Does B shift?" > A well-planned RM-MANOVA (or its custom-contrast analog) remains the most direct, consequential, and error-controlled approach. Dismissing it as a historical relic overlooks both its methodological virtues and its frequent, appropriate use in contemporary psychological science.

As I already mentioned and acknowledged, IFF you have a multivariate hypothesis, the MANOVA approach is totally fine in my opinion and you should use them, but when do you really have such a case?

To use your example, what is the point of knowing “if I look at motivation, control and inhibition together, I know that something happens on a linear combination of these variables, but I cannot say on which variable and how much, but surely, there is something.” I think this result is not very satisfying and in reality, as an applied researcher, you want to know, which of your DV is really affected by your intervention. You do not stop at the multivariate hypothesis, since most of the time what really matters are the univariate DVs.

Same with the other examples. A MANOVA does NOT tell you that there is a reduction in negative affect and no change for positive affect. It only tells you that on the composite variable something happens. And a drug might improve memory and executive function but not processing speed is not answered by a MANOVA but by the univariate post hoc analyses.

I think the whole discussion might become a little bit lengthy and I am willing to continue, if wanted maybe via PM). But let me cite some influential references, that MANOVA may not be the best choice under many circumstances (and again, there are situations, but I think they are rather rare).

Tabachnick & Fidell (2014):

“But there are no free lunches in statistics, either. MANOVA is a substantially more complicated analysis than ANOVA. There are several important assumptions to consider, and there is often some ambiguity in interpretation of the effects of IVs on any single DV. Further, the situations in which MANOVA is more powerful than ANOVA are quite limited; often MANOVA is considerably less powerful than ANOVA, particularly in finding significant group differences for a particular DV. Thus, our recommendation is to think very carefully about the need for more than one DV in light of the added complexity and ambiguity of analysis and the likelihood that multiple DVs may be redundant. Even moderately correlated DVs diminish the power of MANOVA.”

“MANOVA also is wasteful if DVs are uncorrelated- naturally, or if they are factoror component scores. The multivariate test has lower power than the univariate and there is little difference between univariate and stepdown results.“

Admittedly, Tabachnick and Fidell themselves ague that a MANOVA protects the family-wise error, but this is not necessarily true. See for example Keselman et al. (1998).

“In an overwhelming 84% (n = 66) of the studies, researchers never used the results of the MANOVA(s) to explain effects of the grouping variable(s). Instead, they interpreted the results of multiple univariate analyses. In other words, the substantive conclusions were drawn from the multiple univariate results rather than from the MANOVA. With the discovery of the use of such univariate methods, one may ask: Why were the MANOVAs conducted in the first place? Applied researchers should remember that MANOVA tests linear combinations of the outcome variables (determined by the variable intercorrelations) and therefore does not yield results that are in any way comparable with a collection of separate univariate tests. Although it was not indicated in any article, it was surmised that researchers followed the MANOVA-univariate data analysis strategy for protection from excessive Type I errors in univariate statistical testing. This data analysis strategy may not be overly surprising, because it has been suggested by some book authors (e.g., Stevens, 1996, p. 152; Tabachnick & Fidell, 1996, p. 376). There is very limited empirical support for this strategy. A counter position may be stated simply as follows: Do not conduct a MANOVA unless it is the multivariate effects that are of substantive interest. If the univariate effects are those of interest, then it is suggested that the researcher go directly to the univariate analyses and bypass MANOVA."

And it seems also not true that MANOVA generally protects the univariate analyses. As Frane (2015) argued:

“Myth 1: MP (MANOVA Protection) Never Requires Nominal Alpha Level Adjustment

Many authors (e.g., Enders, 2003; Grice & Iwaski, 2007; Harris, 2008; Huberty & Morris, 1989; Kellow, 2000; Share, 1984) have identified the common misconception that MP always controls the FWER without nominal alpha level adjustment. In fact, Bird and Hadzi-Pavlovic (2013) noted that in practice MP is typically applied without adjustment. The gatekeeper does cap the FWER at α when all univariate null hypotheses are true, but a real effect in any variable eliminates this protection. Thus, unless m = 2 (in which case a real effect in one variable would eliminate the possibility of more than one Type I error), MP requires nominal alpha level adjustment to control the maximum FWER.”

Thus, you need to adjust the alpha levels of the post hoc univariate analyses. But if so, why bothering with MANOVA again in the first place. Frane (2015) showed that in typical situations, adjustment of the univariate analyses works fine.

And for a situation without a clear multivariate hypothesis, he states:

“Myth 3: MANOVA Is Always a Prerequisite for Multiple Univariate Tests

Many believe, as some textbooks have implied (e.g., Christensen, 2007), that an omnibus F-test is always necessary when there are multiple univariate hypotheses—even if the researcher is only interested in the univariate results. However, there is no apparent justification for such a rule (as noted by Barnette & McLean, 2005; Grice & Iwaski, 2007; Huberty & Petoskey, 2000; Keselman et al., 1998)."

And MANOVA is not without assumptions itself. It does not require sphericity, but multivariate normality and homogeneity of the variance covariance matrix. I don’t know if this is generally a good trade off.

Frane, A. V. (2015). Power and Type I Error Control for Univariate Comparisons in Multivariate Two-Group Designs. Multivariate Behavioral Research, 50(2), 233–247. https://doi.org/10.1080/00273171.2014.968836

Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S., Cribbie, R. A., Donahue, B., Kowalchuk, R. K., Lowman, L. L., Petoskey, M. D., Keselman, J. C., & Levin, J. R. (1998). Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses. Review of Educational Research, 68(3), 350-386. https://doi.org/10.3102/00346543068003350

Tabachnick & Fidell (2014). Using Multivariate Statistics (6th ed.)

Of course, MANOVA alone is not enough to directly assert the precise effect of each single factor after asserting the composite effect; then, the univariate post hoc analyses come into play. However, using a series of repeated ANOVAs is not much better. I'm not saying MANOVA is perfect; there is hardly an ideal statistical test. Outside the context of a multivariate hypothesis, I won't argue as much.

But to claim it as a "relic", we must have either disregard it because of overwhelming disadvantages, or have developed a comparable tool that completely overshadows its advantages.

=> Perfect orthogonality is rare in real data (psychometric subscales almost always correlate to some degree) -> so MANOVA typically does leverage shared variance for extra power. Simulation studies (e.g., Shieh, 2008) show that when dependent measures share even modest correlations (r ≥ .3), MANOVA's multivariate test can detect effects that three separate ANOVAs miss, by pooling evidence across outcomes.

=> Even Tabachnick & Fidell (2014) concede that "MANOVA protects against inflation of α and is recommended when sphericity is in question" (p. 204). The multivariate test (e.g., Pillai's trace) guards the overall Type I rate at α if all nulls are true. You only proceed to univariate follow-ups after clearing that omnibus gate, avoiding the need for Bonferroni–style penalties on every individual test -> I will still argue that repeated-measures ANOVA requires ε‐corrections (GG, HF) if Mauchly's test fails, but RM-MANOVA is immune to sphericity assumptions. When you have more than two levels (as in most within-subject factors), this is a non-trivial benefit.

=> MANOVA shines precisely in a scenario where (a) you have more than one correlated outcome, (b) you care about the joint profile of effects, or (c) sphericity is in doubt. A significant multivariate F is not a useless black box -> it's the required omnibus gatekeeper that both protects your α and pools weak but consistent signals across DVs for maximum sensitivity.

=> In clinical trials, neuroimaging, and psychophysiology, MANOVA remains the standard for multivariate endpoints (e.g., symptom clusters, brain‐region activations), precisely because it balances power, error control, and interpretive clarity. MANOVA yields canonical variates that show which combination of DVs best separates groups, often revealing synergistic effects invisible to univariate contrasts (this is very important in complex clinical settings).

=> In short, to discard MANOVA as a "relic" it must either has fatal flaws (across-the-board) that render it useless in modern data (i.e. disadvantages so severe that in every plausible scenario you'd be better off not using it); or -> There exists a universally superior method that replicates all of MANOVA's strengths (omnibus multivariate testing, sphericity immunity, canonical variate insight) while also fixing its weaknesses (e.g. sample-dependence of variates, reliance on multivariate normality) with no new trade-offs -> In practice, neither condition is met > No single method has fully eclipsed MANOVA for the classic problem it solves (simultaneous testing of multiple correlated outcomes under repeated-measures or factorial designs) > Alternatives like multivariate linear mixed models, multilevel SEM, or generalized estimating equations each bring their own complexities or assumptions (e.g. choice of random‐effects structure, large-sample requirements, distributional guesses) and so are more complements than outright replacements.

>Correspondingly, MANOVA's disadvantages (sensitivity to covariance heterogeneity, need for post-hoc univariate tests, potential power loss with uncorrelated DVs) are real but not fatal in most applied settings. Many of these can be mitigated by robust or permutation-based variants (e.g., Pillai's trace, permutational MANOVA).

=> Again, I'm not saying MANOVA is perfect, I simply view it as one of several tools in the modern quantitative toolkit, each with its own strengths and appropriate contexts.

Shieh, G. (2008). Detecting interaction effects in moderated multiple regression with continuous variables: Power and sample size considerations. Organizational Research Methods, 12(3), 510–528. https://doi.org/10.1177/1094428108320370

Vallejo, G., & Ato, M. (2012). Robust tests for multivariate factorial designs under heteroscedasticity. Behavior Research Methods, 44(2), 471–489. https://doi.org/10.3758/s13428-011-0152-2

Caldwell, A. (n.d.). Mixed ANOVA. In SuperpowerBook. Retrieved May 14, 2025, from https://aaroncaldwell.us/SuperpowerBook/mixed-anova.html

https://sites.google.com/view/shinystatkln/home