Dear Saleh, formally your scale may have ordinal level. However, important is how the item responses are formulated. One aim in the construction of a Likert scale is that the "distance" between each item category is "equivalent". Therefore, in a good Likert scale the possible answers are usually formulated symmetrically: 'strongly disagree", "disagree", "agree neither nor disagree", "agree", "agree strongly". A good Likert scale will present symmetry of categories and therefore, equidistant attributes will typically be more clearly observed or, at least, inferred. When a Likert scale is symmetric and equidistant it will behave more like an interval-level measurement and therefore, in practice, Likert scales are often viewed as an interval scale.

• Carifio J, Perla RJ. (2007). Ten common misunderstandings, misconceptions, persistent myths and urban legends about Likert scales and Likert response formats and their antidotes. Journal of Social Sciences 3(3): 106-116.

http://www.john-uebersax.com/stat/likert.htm

Dear Saleh, I'm not sure if it's just convenience or laziness. No offense, but each textbook on statistics will provide you information on your question.

Dear Michael, no offense taken. I'm by no means an expert in statistics, but every single book on statistics says that for ordinal scale data we use the non-parametric tests (Mann-Whitney U,Wilcoxon signed rank, Kruskal-Wallis, Friedman). A statistician told me that there are cases when parametric tests can be used for ordinal scale data under certain conditions. She, however, couldn't find me any reference. If you can provide me with any reference that explains this exception I'd really appreciate it. 

Saleh,

Maybe this site might help: http://www.pmean.com/09/LikertAnova.html

Good luck!

Dear Waheeda,

Thank you so much for this.

Regards,

Saleh.

Dear Saleh, formally your scale may have ordinal level. However, important is how the item responses are formulated. One aim in the construction of a Likert scale is that the "distance" between each item category is "equivalent". Therefore, in a good Likert scale the possible answers are usually formulated symmetrically: 'strongly disagree", "disagree", "agree neither nor disagree", "agree", "agree strongly". A good Likert scale will present symmetry of categories and therefore, equidistant attributes will typically be more clearly observed or, at least, inferred. When a Likert scale is symmetric and equidistant it will behave more like an interval-level measurement and therefore, in practice, Likert scales are often viewed as an interval scale.

• Carifio J, Perla RJ. (2007). Ten common misunderstandings, misconceptions, persistent myths and urban legends about Likert scales and Likert response formats and their antidotes. Journal of Social Sciences 3(3): 106-116.

http://www.john-uebersax.com/stat/likert.htm

Dear Michael,

Thanks a lot for this. It was really helpful.

Dear Saleh, additionally, another short advice. It would be more helpful, if you choose or add topics like "Statistical Analysis" which is more strongly related to your question and achieves a broader audience. Kind regards, Michael

HI The scale items in a questionnaire are summed to become one construct (so in your case there will be multiple question items measuring Task achievement (As well as the other constructs). These constructs become a dependant variable and so your IV will be your manipulations and your DV's will be Task achievement, Coherence and cohesion, Lexical resource, and Grammatical range and accuracy and you run your ANOVA on those IV'S and DV'S -

Papers employing questionnaires often do this. If you are interested you can look at my paper which is an example of this: Ali, N., Rose, S., & Ahmed, L. (2014). Psychology students’ perception of and engagement with feedback as a function of year of study. Assessment & Evaluation in Higher Education, (ahead-of-print), 1-13.

I can also discuss the process with you if you like

Hi,

I understand a reference is always needed, so here goes another.

Dear Saleh

I struggled with the same problem and besides the excellent articles cited in this discussion would like to add some more.  Boone & Boone, 2012.  Analyzing Likert Data. Journal of Extension, Vol 12(2).  This is my favourite as it explains in a clear, concise way the difference between Likert-type and Likert - scale data.  Wigley III, 2013. Dispelling three myths about Likert Scales in Communication Trait Research. Communication Research Reports, Vol 30(4), 366-372.  Also Allen & Seaman, 2007.  Likert Scales adn Data Analysis.  Quality Progress, Vol 40 (7), 64-65, and Norman, 2010.  Likert Scales, levels of measurement and the "laws" of statistics.

Hope this helps.

Regards

Hennie

Hola Saleh tu pregunta es valida y lo importante es que tu instrumento tanto preguntas como respuestas este bien construido que sea fiable, valido y pertinente, y cuando eso ocurre es valido usar la ANOVA en análisis cualitativos y la prueba de fuego es usar K-W y compáralos resultados si esta bien hecho saldrán iguales.

Usa el traductor 

In this study, the Ho is that the mean value is not different between both groups. If you can obtain a cuantitative valuation concerning teachers' writting hability, ranging between 1 and 9, for 4 categories you can obtain the mean value for each teacher and treat it as a quantitative variable. Then, you can compare means.

A couple of thoughts about your instrument and your prospective analyses. First, is your rating scale truly analytic, or is it a multiple-trait or adapted holistic scale? A score range of 0-9 is pretty restricted and may not produce the kind of variance that would justify a parametric test like ANOVA (which presupposes an interval scale, as your query suggests you know). Many researchers have used the (extraordinarily problematic) Jacobs et al. analytic essay scoring rubric for assessing ESL writing samples; I would recommend a more contemporary version, such as the one in Ferris and Hedgcock (2014) Chapter 06. Both of the preceding offer a score range of 0 to 100, which will produce a more generous score range (but require greater control when it comes to ensuring interrater reliability). Second, I would argue that any Likert or Likert-type scale is always - by definition - an ordinal scale (and thus produces less robust and less reliable values than a true interval scale). True that the numeric "distance" between scores on a Likert-type scale is uniform, but there is really no way of ascertaining the respondent's or test-taker's certainty, reliability, or truthfulness in selecting a 5 over a 4, and so on. Indeed, a single participant's 4 isn't likely to mean the same thing across items, and her 4 is even less likely to mean the same thing as another participant's 4. Finally, I might suggest avoiding parametric tests in instances where we can't truly satisfy ALL of the assumptions associated with them (and the right scale is pretty crucial). I'd suggest Turner (2014) for some potential alternatives.

References

Ferris, D. R., & Hedgcock, J. S. (2014). Teaching L2 composition: Purpose, process, and practice. New York, NY: Routledge.

Jacobs, H. L., Zingraf, S., Wormuth, D., Hartfiel, V., & Hughey, J. (1981). Testing ESL composition: A practical approach. Rowley, MA: Newbury House.

Turner, J. L. (2014). Using statistics in small-scale language education research: Focus on non-parametric data. New York, NY: Routledge.

Dear John,

Thanks a lot for this. My rating scale (see link) is referred to as 'analytic' in language testing jargon, but it may well be multiple-trait. Originally I compared all the scores using non-parametric tests, and the Turner (2014) reference was of great value. For each criterion on the analytic scale I ran 8 non-parametric tests comparing all the possible interactions:

1. Natives' scores vs non-natives'.

2. Scores of samples with short average sentence length vs long.

3. Natives' scores on short samples vs their scores on long.

4. Non-natives' scores on short samples vs  their scores on long.

5. Natives' scores on short vs non-natives' scores on short.

6. Natives' scores on long vs non-natives' scores on long.

7. Natives' scores on short ns non-natives on long.

8. Natives' scores on long vs non-natives' scores on short.

 However, a statistician suggested it would be better to aggregate my scores to get a mean for each rater, check the residuals, then treat the summary scores as continuous. This would allow us to use the mixed ANOVA to check whether there was a significant interaction between raters' L1 (natives/non-natives) and the mean average sentence length of written samples (short/long), and if a significant interaction did exist, follow-up multiple comparison tests (post hoc) would be run.

Right now I have both the non-parametric and parametric results, and there is not much difference between them. I personally feel a lot more confident with what I had done previously (followed Turner's (2014) instructions and treated my data as non-parametric). Most people on here, however, seem to agree with what the statistician told me. Moreover, statistics in not one of my supervisor's strengths, so she has no opinion on the matter. My main concern is how my examiner may view this.  

Regards,

Saleh.

http://www.ieltsessentials.com/PDF/Writing%20Band%20descriptors%20Task%201%20and%202.pdf

Whether or not your model is unreliable given non-equidistant data depends also on the other related aspects of the normality assumption, such as skew and kurtosis. Lantz has written a paper specifically on non-equidistant Likert data that I think you'll find informative [1].

The take-home message is that ANOVA seems realtively insensitive to violations of the assumption of equidistance given that the other assumptions of normality holds; otherwise, one should look to robust alternatives.

1. Lantz B. The impact of non-equidistance on Anova and alternative methods. Electron J Bus Res Methods [Internet]. 2014;12(1):16–26. Available from: http://www.scopus.com/inward/record.url?eid=2-s2.0-84912137275&partnerID=40&md5=e284285938f919aba5e5acbf52a90cfc

following