Dear Hamid Pashaei ,

This kind of response profile is quite normal for a single 5 category Likert question. In terms of your analysis you need to decide whether you are looking for an association or a dependency. An association might lead you to a chi squared test or a ganeralized linear model. A dependency might lead you to a binary logistic regression or an ordinal logistic regression.

Obviously, having a high proportion of your responses in one category will reduce the sensitivity of your analysis. But the larger your sample size the more sensitive your analysis will be.

MEANINGFUL ANALYSIS should come from meaningful data. The data is not meaningful if it came from "obvious" question to which the answer may seem consistent. If you get 90% of the responses are the same answer, you need to go back to the survey (instruct) and ask whether that question is too obvious. An obvious question or a question that reflects a general public sentiment will most likely return the same answer, i.e. "do you eat when you are hungry? do you drink when you are thirsty?" The answer to this type of questions will most likely be predominantly the same. You may get 90% or more same responses.

BINARY ANALYSIS FROM CONTINUOUS DATA is a sign of defective instrument. If the result of the continuous data became categorical (excessively divisive), this should raise flag that the instrument may be the cause.

SCALE IN SURVEY may be reconsider. The verbal description may be replaced with numerical continuous scale: 1 = lowest, 2 = low, 3 = medium, etc. or using the interpretive note of:

• Definitely Not
• Probably Not
• Possibly
• Probably
• Definitely

However, these interpretive notes should be used internally by researcher. Response scale should be used and explain to respondents as levels: (1,2,3,4,5). Your analysis may become more meaningful if the response definition is change as such.

Did you pilot your research instrument?

Doing a pilot study is important, so you can find out how participants might react to your items.

My suggestion is to treat these 8 respondents as a pilot study - change the scale - and re-pilot to see if the ceiling effect dissipates.

To solve the crux of the issue - Can you not rather use a sliding bar rather than each option? This way you only give them the two extremes - Definitely Not, Definitely ... and then they can place the bar on the slider for however strongly they feel.

This avoids using words that will receive a large number of responses.

I hope this helps.

From only the first 8 respondents in your pre-test, it might be too soon to conclude that upwards of 90% in a decent-sized sample will all choose the "Possibly" category. But if that result does hold for a larger pre-test sample, then your pre-test has given you the very valuable information that this item does not have much, if any, meaningful variance and therefore will not serve the purposes of your research as you had hoped.

As suggested by Paul L., you might be able to obtain more variation in responses if you provide verbal descriptors for only the two extreme categories -- just ask the respondents to choose a number from 1 to 5, where 1 is described as Lowest Probability of purchase and 5 is the Highest Probability, but categories 2, 3, and 4 are not labeled in words. Another approach would be to omit the Possibly category from your scale, thereby reducing the choices to 4 and forcing uncertain respondents to choose either "Probably Not" or "Probably". Unfortunately, there is no way to know whether either of these approaches will increase the variation (without also creating other problems, like increasing the Don't Know or Refuse to Answer responses) unless you do more pre-testing on these alternatives.

Assuming that this is only a pretest and not the final application of the survey, you have just demonstrated the value of a pretest. The question, as currently designed, is not providing you with enough information to be useful in other types of analyses. One possibility is to increase the number of items to the scale (e.g., moving to a 10-point scale); people still would tend to give answers in the middle, but there hopefully would be more variation in their responses. Whether that variation would be meaningful, however, is another question.

It seems that people are picking the middle option because they don't know what they would do. (As a side note, I think "possibly" is probably not the best word choice. Based on the wording, it is a somewhat positive response, but as the middle of five items, I think people are treating it as a neutral response.) One thing you might investigate is whether they really don't have any strong feelings one way or the other, or they have conflicting feelings, leading to a overall undecided responses. Each topic could be pursued further, to determine why they don't care or what the conflicting feelings are. Maybe you might also want to investigate what new information or change in features would create the greatest change in their feelings. This might require focus groups and/or another pretest before you are ready to field the final instrument.

Nope Nopes.

Could you add a comment box where you ask people to explain their answer? Could you make it a 10-point likert scale to add a bit more variation?