Interval means the distance between 2-3 is the same as between 7-8 (and the other comparisons). That is different than being discrete versus continuous and different from any assumptions about the distribution from which the data were drawn. So binary data are interval (0-1 is the same as, well, 0-1), but neither discrete nor drawn from a normal distribution.

As far as why do people still us data that do not even come close to meeting the assumptions, I think there are four main reasons. 1. Reviewers let them get away with it (and if the reviewer mentions it the editor often lets the author get away with it). 2. There either are not methods well-suited for the situation and paraphrasing Tukey (I think this is from his Sunset Salvo paper) an approximate answer to right question is better than the right answer to the wrong question. 3. The appropriate procedures will be unknown to 99% of readers for some models. I sometimes am told to put descriptions into supplementary materials and add a footnote essentially saying the approximate methods lead to the same conclusion as more appropriate methods. On the one hand if the readers won't get what is being said the communicative aspect of publication is affected, but as an archive for research this is wrong (Duncan Luce wrote a nice editorial on this). 4. fINALLY, for most situations there is not a perfectly suited procedure.

I'm sure there are lots of other reasons. wrt factor analysis with a 1-10 scale, the issue I often see that bugs me is when some of the variables have all 9s and 10s and are still included in those models.

ps. You should write to Survey Monkey and tell them to change that. That will confuse lots (or worse, not confuse someone!).

pps. Add another issue. Some of the methods for ordinal discrete data use a lot of parameters (e.g., between each value, see e.g., Section 4.3 of Patrick Mair's recent MOdern POsychometrics with R). This can create a few problems and if the "solution" is collapsing categories that is also not ideal.

1-10 are not interval data, but many data that appear to be interval really never are. A lot of data can only have whole numbers as values and some mathematical operations are not allowed.

Daniel Wright, it might be a good idea to keep variables with only 9 and 10 scores, but not within a factoranalytic framework. In a IRT framework it might be the easiest or hardest score and might be needed to calculate Warm's theta for extreme values. However asking so many ordinal categories might be beyond people capacities to use. It has been shown mathematically that more categories will give higher reliabilties. If you need so many categories, you would need check if people can really differentiate between them. You should ask: why did you score x and not x-1 and x+1. I would not always be able to give the answer myself. Could I assume respondent could do it?

In IRT it would require too many paramaters, so you might need to use another approach.

Hello Nick,

You ask interesting questions, and they are ones that I tussled with some years ago. First, I think that most people would agree that data from a single Likert-type item should probably be regarded as ordinal (and no higher) in nature, but that when responses to a number of Likert-type items are combined, the aggregate of those responses can be regarded as being at the equal-interval level of data - at least when it comes to the kind of statistical analyses that are applied to them. And, in my experience, reassurance is provided by nonparametric and parametric analyses producing almost identical results on aggregated data.

WRT to SurveyMonkey, therefore, my hunch is that their advice is acceptable if they are referring to the aggregate of a number of Likert-type items.

As far as the issue of using individual Likert-type items in factor analysis is concerned, I confess that I'm not sure at a deep, deep, deep level whether that is valid or invalid. However, I can certainly say that it seems to work. Time and again, I have found that putting individual Likert-type items into factor analyses has produced outcomes that have intuitive appeal - even when guarding against being wise after the event, and certainly when I have entertained some a priori hunches.

And, for what it's worth, I share Daniel W's being bugged (see above) concerning researchers working with data in which there is extreme skewness such as most responses being 9s or 10s on a 10-point scale.

Robert.

Robert Trevethan, the aggregate of individidual Likert-type items do not necessarily become interval data. One has to 'proof' a sumscore may be created. For an average the sumscore is required In SPSS reliability has an additivity test. This should indicate additivity. The whole world of IRT modelling is concerned with this idea and unidimesionality. The aggregate can not be 'regarded' as interval data. This would be measurement by fiat. I am quite sure my mentor (Ivo Molenaar) would turn in his grave if I would do that.

I am not concerned with skewed items if I use an IRT model. I really need difficult items to get a good estimate of the extreme values with warm's procedure for items. I can not calculate person-parameters in the same way. Disgrarding easy items is less of a problem, but would also make it hard to estimate the lower bound scores for the scale. In quite a lot of IRT models the assumption is made the scale has a mean of 0 and a standard deviation of 1, but it also assumes the scale does not have a lower and upper bound. In IRT models the scores generally are between -4 and 4, but I have seen other ranges. And yes, it might be strange to expect people to score more then 4 times a standard deviation below or above the mean of 0. But how would Andrew Wiles (the mathematician solving Fermat's last theorem) score on a mathematics test? Would you include the question to solve Fermat's last theorem in a test? If you leave it out, could you really calculate the maximum score on a mathematics scale?

In normal distibuted data the mean and standard deviation are mathematically unrelated, but in a binomial distribution they are mathematecally related (mean = p, variance = n*p*(1-p). In norminal data the relation between mean and standard deviation is very complex (see article by Alberto Maydeu-Olivares) .

It would seem there are different approaches that can be used to tackle the issue(s) that Nick's original questions posed, and some of those approaches will reflect quite different commencing perspectives and "follow throughs".

I could well be wrong, but I did not perceive Nick's "domain" to be related to IRT, but rather to identifying relationships among several variables.

Nick, I'm sorry if I headed off in the wrong direction - but I'm not sure that I did. Regardless of that, I hope you've received some helpful advice and I wish you all the best with your research. Ah, and maybe others will contribute to this series of posts.

Robt.

Thanks for endorsing my most recent post, immediately above, Nick. I have been thinking that others might contribute to this thread, but by now (more than a week later) I think I might add a few thoughts in case they are helpful.

Going back to your original concerns, I suspect that there is a continuum of views concerning whether data obtained from Likert-type scales (as opposed to Likert-type items, i.e., individual items) are ordinal or equal interval in nature. At one end of the continuum are people who might be described as shrill, adamant purists - and at the other end are people who might be described as ignorant, careless slobs. (Please smile.)

As someone who attempts to work with a middle-ground, practical framework that seems to have been supported by a wide range of empirical research, I certainly regard individual Likert-type items as being no better than ordinal in nature. I also regard an aggregate of Likert-type items, although by no means actually equal interval in nature, at least to be amenable to the kinds of statistics that are appropriate for equal-interval data. And, in my view the more items used to validly create that aggregate, without redundancy among them, the better. My reason for this is that many constructs of interest have different facets that are worth tapping, even if those facets reside within a single factor.

That aside, I recommend that you look carefully at your data at all times, and even run pilot investigations to see what kind of data you are likely to obtain in your final study - and whether your questions/items might need adjusting in light of your initial investigations. The importance of this has just been highlighted for a colleague and me in some research we are currently writing up for publication. In that research, we found that, compared with a scale having 7 response options and words above every odd-numbered option, a scale having 9 response options with words only above the extreme options produced inferior data that included, inter alia, spurious variability, a greater tendency for scores to clump at certain points along the range of scores, outliers, and poor test-retest reliability.

This reminds me that you might find the following chapter helpful:

Krosnick, J., & Presser, S. (2010). Question and questionnaire design. In P. V. Marsden & J. D. Wright (Eds.), Handbook of survey research (2nd ed., pp. 263–313). Bingley, UK: Emerald.

I hope you are finding the issues you are confronting to be intellectually invigorating.

Robt.

Thanks for your response Robert. Reading around it seems that the views expressed in many papers and some on this thread agree with your own regarding the type of data obtained from Likert scales and ways to approximate interval level data. However I’m struggling to visualise how this works in practice. For example is we visualise a typical latent construct (say with 4 indicators). would each of the four indicators be themselves composed of data from several meaned/summed subscales?

Sorry, Nick, but I can't quite get on board with you concerning your questions. Might it help if you provide a "name" for your "typical latent construct" and specify what you mean by "4 indicators"? By "indicators", I think you don't mean Likert-type items, but I can't work out what else you might be referring to. Sorry.

I wonder whether factor analysis (if you can get enough participants) would help you sort some of these things out. Are you familiar with factor analysis: what it can achieve, how it is done, etc.?

Robt.

Apologies (I am a newbie - no surprise? :)

Using the example latent variable n1 which I will call 'trust' in the picture I have uploaded (grabbed from a CFA youtube tutorial). I would envisage X1-X3 to be three indicators of trust (jndividual question items), measuring (for example) perceived 'honesty' of the target (X1), perceived 'competency' of the target (X2) and perceived 'reliability' of the target (X3). Each of these would using a Likert scale produce a score of (lets say) between 1 and 5.

If I survey (say) 200 people, I will have the scores of between 1-5 for each. Are we saying that

-The average/aggregate of the 200 scores for each of the three indicators may be considered as interval level data for the purposes of factor analysis etc?

-Each of the three indicators must themselves be represented by more than one sub-scale and the average/aggregate of these used in the factor analysis

-none of the above

thanks

Hello Nick,

Thanks, that helps a bit, but I suspect that my advice runs the risk of being awry because I'm not fully aware about the nature of your intended research - what your research questions are and which variables you intend to use in addressing those questions.

Given the information you've provided, it seems that you might be measuring trust with three items. If you are able to be certain that your three items all tap aspects of trust (factor analysis in conjunction with other items might help you to feel confident about that), I think you could simply add the scores you get from the three items to represent the single construct of trust. If you wanted, you could use that total as the variable you worked with from then on (scores could range from 3 to 15 in that case - each of your 200 people would have a score somewhere between 1 and 15), or you could obtain an average of the scores (which means your trust variable would range from 1 to 5 - and each of your 200 people would have a score somewhere between 1 and 5). Either way would be essentially the same for statistical purposes.

I just hope his is helpful - but for some reason I don't feel extra confident about that. I have a disconcerting feeling that you're not quite on top of the nature of factor analysis and when and how it should be used. Because of that, I think that the "None of the above" response might be best WRT your suggestions above.

Are you able to get some time with a tame statistician? Feel free to keep asking questions . . .

Robt.

thanks. yes I am working my way on my own through stats books so sorry if my comments are off-beat. I am learning about the application of EFA and CFA (Researchgate and Youtube help reinforce guidance from Field, Byrne and Brown etc). Im assuming that you use the summative scale data for hypothesis testing AFTER you have determined the factor structure before moving onto test hypothesised relationships between the latent variables. Although the example below doesnt appear to be a solution for multivariate examples i.e. testing relationships between multiple latent constructs.

https://www.researchgate.net/post/What_statistical_analysis_do_I_use_for_a_Likert_Scale

Hello Nick,

I think that, from your comments above, you are getting the right end of the stick. Yes, certainly, the items that are chosen to be summed on a Likert scale to obtain a total for a latent construct should be determined after processes such as factor analysis have been applied.

About 10 years ago, I had an article published about how to develop scales. It applied to people's assessment of their own foot health, but the principles are widespread. If you want to read it, here's the reference:

Trevethan, R. (2009). Self-assessment of foot health: Requirements, issues, practicalities, and challenges. Journal of the American Podiatric Medical Association, 99, 460–471. https://doi.org/10.7547/0990460

Here's another article that might help you, available open access: Article Best Practices for Developing and Validating Scales for Heal...

One problem with all of this is that it might require too extensive an amount of effort for your current research interests and aspirations. Creating scales, including use of factor analysis, takes a fair amount of time and effort, and also often requires large participant numbers. And that's just at the piloting stage.

The RG site that you referred to at the bottom of your last post contains some good information, but also some information that I think isn't extra good. WRT the latter, I think you're right in that the recommendations aren't sophisticated enough, e.g., to use t-tests. My hunch is that something such as multiple regression might be better in your research - though I don't know enough about your research to make confident recommendations.

As always, I hope this is helpful.

Robt.

Very interesting thread.