Here are some of the most cited papers in this topic and their take on it:

"Parametric statistics can be used with Likert data, with small sample sizes, with unequal variances, and with non-normal distributions, with no fear of 'coming to the wrong conclusion'.

Article LIkert scales, levels of measurement adn the “laws” of statistics

It is, therefore, as the intervalists contend, perfectly appropriate to summarize the ratings generated from Likert scales using means and standard deviations, and it is perfectly appropriate to use parametric techniques like Analysis of Variance to analyze Likert scales.

Article Resolving the 50-year Debate Around using and Misusing Likert Scales

• Now I want to analyse if there is a significant difference between the groups, per statement.

The correct approach would be a chi-squared test or Fishers exact test.

• Though, can I use ANOVA anyway changing/quantifying the data?

Yes. Assign some sensible(!) quantitative utility value to each answer. Such an utility value could for instance be the money people would be willing to pay, depending on their degree of agreement with a statement, or the time they would be willing to spend, something like this. Whatever it is, it must be some kind of "currency" that can be sensibly exchanged between the degrees of agreement.

Any "arbitrary" coding of the ordinal values with some numbers (like -3, -2, ...  +2, +3, or 0, 1, 2, ...5 or any other way) plugged into an ANOVA will give you as arbitrary results.

• Especially, does anyone know if and how I can do this in a simple way in Excel

The Excel-function youu can use is LINEST. It is a bit cumbersome since you have to manually set up the design matrix that will have 4 columns with 0/1-coding for the levels of agreement.

I would advise _against_ using anova for a 5-point likert scale.  (Various reasons I won't list now.)

Luckily, if you just want to compare among groups, question by question, you can use a Kruskal–Wallis test for each question. 

(This treats the Likert items as an ordinal scale, which I favor. This is different than Jochen's approach above.) 

Post-hoc analysis (to compare among the groups) can be done with Dunn test.

I'm not a fan of using Excel for any statistical test, but the Handbook of Biological Statistics has a link for an Excel spreadsheet to do the K-W test.  Also link to do it in R.  The R analysis includes the Dunn test.

http://www.biostathandbook.com/kruskalwallis.html

Link for R analysis:

http://rcompanion.org/rcompanion/d_06.html

Comment on Salvatores post: It is absolutely unreasonable to expect equal shapes of distributions of Likert-type variables, therefore K-W tests should *not* be interpreted as tests on the medians. They test the stochastic difference, and I can not even write down the null that is tested (I would have to search and look it up; for the two-sample case, the MW-test is Pr(|y-x|>0)=0.5). If you think that this is a sensible hypothesis to test, K-W is ok. My problem would be to interpret an effect size what is actually neccesary to judge the significance of the result. But this is the same problem I have with Chi²-like tests.

Simone listed SPSS as one of the topics, so presumably uses that software.  Assuming your sample size is large enough, and there are enough observations falling in each of the 5 response categories, you could use the PLUM procedure to estimate an ordinal logistic regression model (which treats the outcome categories as ordinal).  See the first link below for more info. 

On the other hand, depending on what the response distributions look like, ordinary ANOVA probably yields a better model than many people might think.  See the second link below, for example.

Finally, I agree with Jochen's earlier comments on using the Kruskall-Wallis test for this type of outcome variable. 

HTH.

http://www.norusis.com/pdf/ASPC_v13.pdf

http://www.ncbi.nlm.nih.gov/pubmed/20146096

Hi Bruce, you say:

"On the other hand, depending on what the response distributions look like, ordinary ANOVA probably yields a better model than many people might think. See the second link below, for example."

But such a distribution you have in mind is defined over a numerical variable. Likert is not a numeric scale. One could name tha values A,B,C,D,E. Now let's say the frequencies you find are 5,24,40,24,5. Is this a "symmetric" distribution? Is this even a resonably good approximation to the normal distribution? I think you would say: yes, but I say there is no way to measure the symmetry (nor any other property of a distribution*) here, because the differences between A,B,C... are not defined. To give an example:

Consider a biochemical assay where some concentration of some analyte is measured by the intensity of a color-reaction. The scientist is evaluating this "by eye" and rating the intensity on a Likert-like scale ("uncolored", "a bit blueish", "light blue", "blue", "dark blue"). Consider he rated 98 wells or cells for that color and found the distribution above. Now here we have some nice biochemical system and we can think of a function relating the color intensity to the underlying concentration of the analyte that should actually be measured. This function does not need to be linear, it can have any arbitrary shape: logarithmic, some power function, sigmoidal, logistic, ...

So on the scale of the analyte the distribution may look entirely different. If we knew the functional relation beween the color grade and the analyte concentration** we could assign sensible numerical values to the categories (the precision will depend on the function relating the color intensity to the concentration, for sure). And then we could see what we can know about the concentration and about the difference in the concentration between two groups and so on.

What you seem to propose is to assign a series of integers as values (like 0,1,2,3,4, or -2,-1,0,+1,+2, whatever). By doing so you effectively look at the ranks of the values and then you propose a standard analysis of the rank-distributions (which may then in fact be called symmetric or approximately normal etc.). But what you analyze is not the variable you are interested in - it is the distribution of the ranks of the values (of the ordinal variable) you recorded. That is perfectly fine so far - but: I think it is almost completely unclear what a difference in ranks tells you, or what the precision of the estimates tell you (and hence p-values and all that). A "mean rank difference" of 0.1 might be very significant between the rank #1 and #2, but it may be completely irrelevant between the ranks #4 and #5. If some linear model estimates a "slope" (how large is the rank-difference if the predictor is increased by one unit) - what does this mean? How can I judge the relevance?

I think I am missing something important here.

---

* maybe except for "uniformity"

** where this is possible people use a calibration curve to get values for the concentration directly; but there are cases where this is not possible, like for imunno-stainings of cells in tissue-sections. So we know that there must be some relationship but we are unable to technically determine this relationship. The same - only much more complicated - is the case for questionaires: we assume some kind of relationship between, say, "attitude" and the answer given to some appropriate question, but we are unable to determine the quantitative relationship.

Jochen, have you read the "laws of statistics" article by Norman?  He includes a section (starting at the bottom of page 628) addressing ordinal outcome variables like Likert-type items. 

No I havent. I will see if I can get it.

Do you have any reference to the statement that the analysis of ranks using linear models "usually work very well"? What is "well"? How can the performance of such analyses be determined? Has this been studied somewhere? I mean "real" cases, not some made-up mathematical simulations. Is there any performance measure of such analysis based on real data, possibly using actually replicated studies or some kind of meta-analysis (I don't know if a meta-analysis would be adequate)?

Ok Bruce, I found the paper.

On page 629, 2nd paragraph, they state:

The numbers ‘‘don’t know where they came from’’. What this means is that, even if conceptually a Likert scale is ordinal, [...], this is irrelevant to the analysis because the computer has no way of affirming or denying it. [...] And all the computer can do is draw

conclusions about the numbers themselves. [...] After all, someone has to decide whether the analysis done on the numbers reflects the underlying constructs, and Gaito provides no support for this inference.

So yes, that's very true (except on thing: a computer does NOT draw a conclusion! - it is the scientist drawing a conclusion!!). And I am concerned about the last sentence, not about the applicability of some calculation method. I can add biligels and devide the sum by gotions wen both quantities are represented by some numeric value. This may well work numeriacally, but how do I interpret the calculated values?

Take a simple case and think of a log-normal distribution, like for a variable like "income". One can stress the CLT in larger samples to get a reasonable test of the mean difference in incomes. But here the mere difference does not tell me the relevant story. 100 bucks more is a big deal when the mean income is generally low, but it is not worth to mention when the incomes are very high. Here one can oversee and understand this relation, so a difference of 100 bucks is interpretable, but what when the underlying relation or distribution is unclear? Is a mean rank difference of 0.2 of any relevance? Or it is utmost important? A p-value does not give me this information, and even if the calculation of a p-value is "robust", I still don't know what to think.

The situation becomes really dangerous when multi-factorial models, possibly including interactions, are employed. An interaction, for instance, would be a product of ranks - what the heck is this?

So it is all fine as long as I "just want a p-value". But as soon as I want to interpret the p-value (or, to step into the hypothesis-testing procedure of Neyman/Pearson, as soon as I have to set up some utility function and think about relevant effect sizes to decide for sensible error-rates) I get lost - unless I either assume a linear relation between the ranks and the underlying variable or I know/assume another relationship and use this (i.e. I simply work with "low-granularity data", allowing some 5 or so different values from a continuous scale).

Good morning, Jochen.  I am pressed for time today, so won't write a lengthy comment.  However, I do not think Norman is endorsing Gaito's argument that the numbers don't know where they come from.  For example, later in that same paragraph, Norman says:

'This is almost a "reductio ad absurbum" argument, and appears to solve the problem by making it someone else's, but not the statistician's problem. After all, someone has to decide whether the analysis done on the numbers reflects the underlying constructs, and Gaito provides no support for this inference.'

Rather, Norman is relying on the robustness of the common parametric tests to violation of their assumptions--or "heroic assumptions", as he calls them earlier.  He cites several old articles that show the robustness of parametric tests, and includes his own little demonstration (see Table 1).  One recent demonstration of that robustness that Norman could also have cited is the 2012 simulation study by Rhemtulla et al. linked below. 

Cheers,

Bruce

http://psych.colorado.edu/~willcutt/pdfs/Rhemtulla_2012.pdf

Hi Bruce,

yes, I read this: 'After all, someone has to decide whether the analysis done on the numbers reflects the underlying constructs, and Gaito provides no support for this inference.'

But that's just my problem. To my understanding, statistics should provide "tools to turn data into insight". Claiming that the calculations are robust in some mathematical sense does not solve the main puzzle that the calculated quantities are not interpretable. The problem seems to be solved for the statistician, but it is not for the scientist. I still wonder if it is a good idea to advice (or at least to approve) methods that give statistically robust values when the main problem ("how do the ranks quantitatively relate to the variable that should be measured?") is not clear, not understood, or unrecognized.

It is a bit like advising a t-test on presumably skewed and/or heteroscedastic data. There are three interesting points about that: i) the test is in fact "correct", because it assumes a conditional normal distribution and homoscedasticity, and the results confirm to these assumptions; ii) the p-value and the confidence intervals will not be interpretable in the usual sense based on frequensitic properties (but who will be aware of that?), and iii) the tested mean difference is likely to have no practical relevance that could be derived from the estimate alone (for a gamma- or log-normal distributed responses one may find that the relative difference, like the log ratio, is some interpretable effect measure).

I always fall back to the testing problem. There are two strategies: Fisher and Neyman. Fisher's significance test requires to interpret the efect size. The tested null hypothesis is (well, should!) not be rejected besed only on a p-value. Neymans hypothesis tests require to state some relevant effect size (or: two alternative hypotheses; one could be a zero-effect, the other some finite(!) non-zero effect) so that some utility function can be used to chose somehow optimal error-rates (and it is not about rejecting a null hypothesis but rather to decide between these two alternatives).

Surely, both test strategies depend on some opinion about the effect size, and this is not a statistical problem. This has to be judged by the researcher. But I think the statistician should take care that the researcher gets some effect size measure that he can interpret in some sensible way.

I am facing the same issue. I will completing a survey and I will be using a median split to "transfer" metric scale into categotical.

Yuliya Lynch, you clearly shouldn't do that.

Jochen Wilhelm , it’s not suitable in many situations, but I think it will be the best solution for me. Please, provide more details if you can.

You say you have data on a metric scale. This contains a lot of quantitative information. Dicotomizing this will ignore almost the complete information. The information content of the resulting binary variabla if far lower. It is usually a bad idea to ignore information. The other point is the choice of the cut-off for dicotomization. Although median is a common choice, it must not neccesarily make much sense scientifically. Based on the (arbitrary!) choice of the cut-off you will get different results, come to different conclusions - and all depends on an (eventually) arbitrary choice. That's not good scientific practice.

I also wonder why you have metric data when you plan an analysis based on binary information. Why wasn't the survey constructed to provide the kind of data you like to analyze?

While developing Likert type scales we consider these as summated scales, then why not ANOVA.

Yes, you can use ANOVA after obtaining summed up score of all statements (reverse the score of a statement according to positive or negative nature of the statement) of each individual of the group. It will give you variance within the group & between groups. Make sure that data meet the assumptions for the application of parametric statistics. But for each statement separately there are many theoretical consideration for using ANOVA on categorical data. Therefore not recommended for each statement.

I can see using ANOVA on interval and ratio data but clearly not ordinal data and this is what the Likert Scale is all about... without a valid zero and without a valid constant separation between categories, then the variance is meaningless. If the variance is meaningless, then the ANOVA is meaningless as well.

If we treated Likert scale as interval scale by putting neither in the place of neutral or moderately i hope that the data itself divide into two category were we can do all parametric tests

Hope this helps you

Let's go back to the basics... the reason for using ANOVA is to "Reject" or "Fail to Reject" a statement about the Mean values in a number of similar(?) populations. This statement is called the "Null Hypothesis" and the focus is on the Means of the populations. Are they similar or are they not? If we fail to Reject the Null Hypothesis, we have not "Proved" that the Means of these populations are equal... only that we can't tell there is a difference at a defined degree of confidence.

If the Mean is not a good Measure of Central Tendency of the data in all the populations under investigation, then ANOVA should not be used. Furthermore, the Mean is considered to be one of THE best measurements of central tendency of any data set (population) if the population is symmetrical (near Normal) distributed. The Mean is very sensitive to outliers and thus if your data set contains outliers, then the mean should not be used ... the Median is the better choice. Use the KWH test for Medians is my recommendation.

The scale does depends does on the outlier it is constructed based on need of the study when scale which is based on interval how does it will not symmterial since it divides, Moreover, all likert scale are primary following interval principal,hence, therefore we can do all parameters test since we calculate mean in any interval scale as per paper above shared. In marketing research researchers largely use parametric test if we cannot use this test mean all marketing researcher doing wrong research it is actually a myth created some people to create unnecessary such kind of debate without any value.

Let me explain in simple terms a scale from 1-5 with equal intervals is termed as interval scale in which the mid point is already exist at 3 for 5 point and at 4 for seven point scale but researchers use different labels for their convenience for example moderately or neutral with the same interval,whereas in interval scale were common label is neither, therefore due to the change in the label researchers get confused the facts is that the mid value remains at the same interval i that is 3 or 4,so by this we can say that likert scale is same as that interval the difference is in the label we use then finding median will not be any use we have to do parameteric tests to get correct results.even reliability based on alpha value also uses mean so how that is validity methods for validating a likert scale finally labels does not creates any difference in the data distance since code are same so we have to only use parameteric tests to get best results

Ordinal scales are developed for different reasoning.

While in theory possible (number based) to apply annova, the outcome would mislead the researcher on the key essence of constructs of the research. hence suggest differential treatment.

Likert scale is not an ordinal scale it an interval sir which is when used with different labels it is leads to a name called likert in which median is same as of interval scale only change in the labels and pure ordinal scale is only ranking scale were we do not have any median even it is not know untill a tools is applied hence we can do non parametric test on such scale. By using a modified label without changing the interval how it can become ordinal scale when median is predefined and there is no zero numerical exist as per research paper and as per my knowledge doing non parametric test on likert data without any numerial change of distance will lead to unquantifiable results and wrong interpretation. For example if a researcher uses netural as label

in 1-5 scale of equal interval research if heir she code as 0 by replacing 3 then I hope we can strongly use non parametric test by just changing label the distance does not change and will lead to wrong results. It is like changing the product from one containers to other and saying by changing to another container is had developed that container quality natural according me it is not logically.

Here are some of the most cited papers in this topic and their take on it:

"Parametric statistics can be used with Likert data, with small sample sizes, with unequal variances, and with non-normal distributions, with no fear of 'coming to the wrong conclusion'.

Article LIkert scales, levels of measurement adn the “laws” of statistics

It is, therefore, as the intervalists contend, perfectly appropriate to summarize the ratings generated from Likert scales using means and standard deviations, and it is perfectly appropriate to use parametric techniques like Analysis of Variance to analyze Likert scales.

Article Resolving the 50-year Debate Around using and Misusing Likert Scales