I'm beginning to feel sorry for this poor student. She asked a relatively straightforward question and produced a largely irrelevant debate about whether everyone should be doing "cutting edge" work as opposed to following well established standards.

As far as what is "blindingly obvious," I would once again refer the student to the preferences in her own field, rather than to others' claims about what is or is not obvious in the eye of the beholder.

First, determine the exact numerical value of "strongly agree", "very emotional", and "most of the time" (by exact, I mean infinite precision). When you realize this is impossible, then you will receive two sets of answers: one from those who took statistics courses in which they learned that because Likert-type data is interval or ordinal (both are wrong, but both are taught), then there exists a collection of statistical methods that are somehow appropriate for translating individual interpretations of a question that is judged conceptually/linguistically by each participant who is forced to take this concept and provide an exact value, and which take these responses that are different for every participant no matter what the responses given but are assumed to be equivalent.

Then there are those which don't make such idiotic assumptions, but aren't generally taught. And when those of us who try to point out the ways in which these generally superior methods should be used and which should be used when, our responses are ignored, criticized, and otherwise rejected in favor of using some intro stats non-parametric test.

What you need, first of all, is a thorough and in-depth study of the responses. You can't possibly hope to analyze responses which are based on different interpretations and biases for each subject and each question without analyzing the responses to look for e.g., middle response biases, hedging, etc. After running various elementary tests to get a feel for both individual responses for all questions and single questions for all participants, you can determine the most ideal method to analyze your response data, whether it is fuzzy numbers, IRT, or nonlinear multidimensional data analysis.

I'm glad you asked this question. There are too many such questions (i.e., questions on how to analyze Likert-type response data) and too many ways to approach the matter poorly. I think I will try to put together a guide of sorts for such questions.

I strongly agree with Andrew Messing - actually I hope you visit my question on rationality of respondents and enrich the discussion (https://www.researchgate.net/post/Are_respondents_rational_And_if_yes_in_what_way), because I do have the same conclusions about assumptions on equivalent responses.

But maybe it is possible to get more detailed information on this second - "criticized" - approach. "After running variuos elementary tests to get a feel ..." - could you develop it?

Putting aside the validity arguments from the previous two respondents, the technique you can use to rate these adjectives is Friedman's ANOVA as this will allow you to compare repeated ordinal data (although, paradoxically the new versions of the algorithms only work when you call the data scale). In the new versions of SPSS there is a very useful feature called the Model Viewer. If you double click on the significance box and select pairwise comparisons it will carry out the post hoc testing for you with appropriate adjustments to the significance values.

Slight correction: Friedman's ANOVA does not work well with a small number of choices. It might be better to rank the 9 adjectives as this is essentially what the Friedman test will do anyway.

Putting aside validity and soundness, you can use simple addition and add up the responses and then divide them by pi. If you wish to put aside validity, why propose anything? You can rate Likert-type responses using Friedman's ANOVA or a magic 8-ball. Validity and soundness is the only thing that renders one superior to the other. If you have to "put[] aside validity arguments", then you have put aside any possible answer with any possible mathematical, statistical, or otherwise relevant answer.

This is not to say that Friedman's ANOVA is a bad method, merely that by prefacing it with "putting aside the validity" of "arguments" is to render it pointless. I can put aside validity and enter values into a calculator and it will logically be at least as sound as any argument for which we "put[] aside the validity" of other arguments. "Garbage in, garbage out."

I don't think the provious responses were about validity, instead they are part of a very old and tired argument about whether you can sum the items that use Likert scoring to form a scale. I presume that was the original design for your measure, since it is based on an existing scale. If so, you should follow the same practice, which was probably to calculate a Cronbach's alpha for the original scale.

From, there, you can do a repeated measures ANOVA to compare the two scales across your three conditions.

This is pretty a common form of analysis, and it is likely to be what is used in peer reviewed journals in your field, and that should be the standard you apply to your work. So, look for similar studies in the literature to be sure you are doing what is typically expected from your fellow scholars.

I can't speak to both "provious [sic] responses", but I can speak to mine, and it absolutely concerns validity and soundness. Your "tired" responses are akin to the "sorcery" and "cult" of statistical significance, only promoters of these typically have far more to defend such positions than illogical and baseless promotion of crud uses of Likert scales of the type you seem to promote (if you have trouble figuring out the quotations, try doing a little research and finding some of the hundreds of criticisms of Fisherian statistics dating back to Fisher),

If you wish to address very "old and tired arguments", you could at least marshal some, or if you don't know any you could attempt to make some kind of rational, logical argument for mapping disparate subject responses to linguistic, semantic content onto infinitely precise numerical values and then applying statistical tests that assume normal distributions over an uncountably infinite interval based on subjective interpretations of less then a dozen, single data points.

But you didn't. You offer nothing other than regurgitated arguments. Actually, you don't even do this. You have to REFERENCE the regurgitated arguments. You seem incapable of producing them. I'm sure your expertise in sociology has given you more than enough experience in fuzzy set theory, linguistics, (real) statistics, etc. But you could at least indicate this familiarity rather than insult those who possess it while relying on non-specific references to support your "argument".

My, my what a flame.

So, you are arguing that Cronbach's alpha produces biased measures, or do you have some other definition of validity in mind?

And my recommending that this researcher should use the standards in her field is hardly a regurgitation. I suppose you could just reject her whole field, which would be your privilege, I guess.

Cronbach's alpha? Seriously? In general, yes it does. This can be easily modeled, but need not be given that

1) Likert scales are attempts to represent with LITERALLY infinitely numerically accurate responses among subjects asked to attempt to pick at best a (known to be generally biased) fit given even ideal survey design. Essentially, subjects are asked to provide vague responses that are treated as infinitely accurate and tested using statistical measures that assume uncountably infinite intervals (if you don't know the difference between denumerable/countable infinite sets and uncountable, I'd be happy provide an explanation).

2) Not only is it idiocy to assume that distances between responses are not only infinitely precise but exactly distanced, but moronic to think that participants will somehow respond using the same interpretations of a linguistic question with conceptual (and hence necessarily anything but exact) responses as if these were all equal and exact. I take it you didn't read Lakoff's criticism of fuzzy set theory in response to this very problem EVEN THOUGH fuzzy sets were designed specifically to account for this issue. That would require knowledge of linguists, logic, and mathematics.

3) Recommending bad methods for bad reasons just because of somebody's background isn't excusable because of that background. Plenty of sociologists continue to rely on bad methods and worse math, but that doesn't excuse the poor results. Luckily, you need not trust me. I have about a century of unanswered criticisms to the standard methods employed in your (and mine, embarrassingly) field

I'd rather keep her field, yours, and mine, and improve the methods and research designs, rather then (for no apparent reason) be given the choice to either use bad methods or reject the field wholesale. Why you feel the choice is between bad methods or no field is beyond me.

It's nice that you would like to help her field by replacing what I presume is their accepted way of doing things with your own preferred approach, but that is an issue for the history of science.

Since the number of researchers who take Cronbach's alpha seriously is so large, I would take that as evidence that its persistence is due to more than sheer inertia, but rather that it has substantial weight. If the levers that subsequent researchers have used to try and dislodge it haven't been up to the task, that says as much about the alternatives as it does about alpha.

Meanwhile, we are dealing with the case of someone who is a graduate student, rather than a cutting edge methodologist or a historian of science. So, my recommendation that she should determine the standards in her field still stands. Whether she chooses to stick with what is widely acceptable to her peers or decides to be an innovator is ultimately up to her.

1) "Her field" is a scientific field. If you would like to examine the highly sophisticated use of statistics in musicology and related fields, I can refer you to some titles that don't require that one disregard soundness and validity.

2) As the researchers who took eugenics so seriously were vast (and included the founders of modern statistics), who cares how many people misuse methods, models, and techniques in some science? Classical physics not only involved hundreds upon hundreds of some of the greatest scientific minds the world has ever known, but some of the greatest achievements in the sciences ever- right up until it failed on a fundamental level. Scientific methods aren't about using what methods are taught, but defensible. Unless we use your approach: screw validity, use dogma.

3) You are absolutely correct about choices here. We are all free to evaluate arguments for and against particular methods, approaches, theories, etc., and to propose our own. We can even (apparently) blatantly disregard the fundamentals of all the sciences by  "putting aside validity" (because, apparently, for some scientific research revolves around dogma, tradition, and cultic-like practice).

4) Bandwagon fallacies aside, can you present an actual defense of any approach to Likert-scale data that doesn't rely on using statistical methods related to or based upon the fact that measurement errors were/are a given and thus statistical methods required to yield a single value from these errors that best represented the true value? In case it isn't blindingly obvious why this relevant, these tests that are designed to yield a "best" value from multiple errors are used in the case of Likert-type response data to take sets of individual measurement errors and use a large number of such errors to approximate...error (if this doesn't seem to make sense, look up the reason behind the full name for e.g., Pearson's correlation coefficient).

I'm beginning to feel sorry for this poor student. She asked a relatively straightforward question and produced a largely irrelevant debate about whether everyone should be doing "cutting edge" work as opposed to following well established standards.

As far as what is "blindingly obvious," I would once again refer the student to the preferences in her own field, rather than to others' claims about what is or is not obvious in the eye of the beholder.

Very true. I spent hours creating a document (that so far involved translating about 6 languages) to thoroughly explain the issues involved. I will finish it, but alas too late (I suspect).

The difficulty in your experimental design is that with it you want to compare impressions of pieces of music between-subjects. Further, the adjectives may overlap in their meaning, thus you are repeatedly asking for the same impressions, but not so clearly that we could call it a repeated measure. There may also be carry-over effects and other surveying effects such as fatigue or learning... 

Were the orders of adjectives randomized for each individual?

So I recommend to look at the original publication of the Geneva Emotional Music scale and follow their advice on how to calculate the scores. Look up papers that have cited that paper and see how the scale was used there. Possibly also conduct a within-subjects version of the study.

I don't fully understand what you want to test, but it sounds like you want to know whether one group's distribution of answers is the same as another group's.  First, look at the data.  Check the distributions of the groups.  Do they look different? If so, how - is it the same shape but just a shift?  Do the ranges seem to differ?  Or is the shape entirely different.  That will point you to an appropriate, and likely nonparametric, test.  (Likert scale data is typically not normally distributed.) Don't focus so much on the test, because statistical tests are just checking whether differences you see in the data can be explained by sampling error or not.  I do not recommend applying numbers to the responses and calculating things such as mean or variance, as you are assuming that the respondents were using that interval scale when they probably weren't.

I just came across this, thank you for this enlightening and entertaining discussion.