When checking the reliability and validity of these scales, I suggest that you apply the same rating range but alter the wording of the item to suit the purpose of your study. If the item is based on a 7-point Likert scale, keep it that way.

During the assessment of the adapted items with different Likert scales (for validity and reliability), you must standardize them into Z-scores, which is the norm - so that you can apply between-subjects analysis without having to fret about differences in the range of values across scales. Alternatively, instead of Z-scores, you can also consider running factor or principal compenent analysis and derive standardized factor scores of different combinations of scale scores.

Dear Pawan,

if you want to follow the official ritual (which you partially should do to legitimize your work) you have to apply the translation-back translation-procedure by Brislin (1970). That is, you translate the original measure (say it is in English) into your language; then a further person not involved in the translation (who knows both languages well) back-translates the measure into English. If the procedure worked, the orginal measure should be similar to the back-translated one.

However, the procedure has some problems. First, there is no measure of "similarity" and not at all any criterion/boundary which degree of similarity speaks for measurement equivalence.

Second, "linguistic equivalence" does not imply measurement equivalence. You can have a perfect translation but due to cultural differences in the meaning of words, you measure something completely different. Hence, more important than linguistic equivalence is knowledge of the functional/psychological equivalence. This requires a translator who know the subject of research and especially the theoretical meaning of the construct of interest.

Beyond the translation process, I would further recomment intensive cognitive probing that consists of qualitative interviews mit subjects from the target population to assure that your translated measures works as intended. And if you want to do it perfectly, do a formal measurement equvalence/invariance test. This works by conducting a multigroup CFA or SEM in which one group is a English-speaking sample (which had the original measure) and the second is your new sample (perhaps you can get the data from the first study by the author of that study so you do not have to do a study by yourself).

There are tons of papers how to do it. Such a modeling approach would allow to test if both versions of the measure are comparably valid (i.e., measure the same construct).

If you allow some self-advertising, here is a paper in which we described all the details. You can find the more popular references in the paper:

Steinmetz, H., Schmidt, P., Tina-Booh, A., Wieczorek, S., & Schwartz, S. H. (2009). Testing measurement invariance using multigroup CFA: Differences between educational groups in human values measurement. Quality and Quantity, 43(4), 599-616. doi:10.1007/s11135-007-9143-x

Hope that helps

Holger

Item response theory.

Hope this helps,

Matt

Jimmy Y. Zhong

Thank you for your response

But, if I have to follow the whole process from reliability testing to deriving standardized factor loading, then what is the differences between developing the scale and adapting the scale??

Holger Steinmetz

Thank you for your comprehensive and interesting explanation.

I want to ask you about preparing questionnaire. What do you suggest me to do, when I have to adapt items from different studies having different rating scale? What are the important things for which I have to be careful?

Matthew Kerry

Thank you sir for your response.

I am reading about item response theory. I will contact you if I face any related problem.

Hi Pawan,

do you refer to different rating formats (e.g. 5-point scale)? If you want to do a invariance test (with CFA or as Matthew Kerry suggested with IRT), you should hold everything equal to the original items. Changing the format leads to metric non-invariance.

HTH

Holger

Dear Kumar

I recomed you Guillemin, Bombardier & Beaton. Cross cultural adaptation of health relates quality of life measures. Journal of Clinical Epidemiology

Pawan Kumar Sharma

Based on the responses so far, I think it is best that you give more details about what you intend to do by "adapting" a scale.

In practical terms, when you adapt a questionnaire/survey or any other scholarly work, you are redeveloping it to make it better, so there is no difference between adapting and (re)developing in the behavioral sense.

What I mentioned earlier on referred to adapting a pre-existing scale specifically in quantitative terms - that is, retaining the rating format but changing the wording of the items. You can also do the reverse, and compare the validity of the item responses based on IRT-related analyses (use IRTPRO or R for this purpose).

If you are focused about translating the scale to your native language (from English, presumably), you may consider changing the rating format to some format that is simpler or more culturally acceptable. You are still adapting a scale, conceptually speaking, since the original scale does not belong to you. Practically speaking, however, you are redeveloping and reassessing it - and so you need to be thorough.

You can talk with experts on your topic. The authors, that my partners are recommending you, are interesting. Also, if you have specific variables, you cand find some scale to apply in your research. Then, it is necessary to do a pilot study to check the scale.

Jimmy Y. Zhong Holger Steinmetz Matthew Kerry Abdul Hernandez Diego André Portillo Tinoco

Thank your for your reply.

First, items should fit the lingual, cultural and environmental aspects relevant to the target participants of the study. Second, validity and reliablity sould be established for the new study population.

This reference can give a good practical introduction to the field of testing

Shultz, K. S., Whitney, D. J., & Zickar, M. J. (2013). Measurement theory in action: Case studies and exercises. Routledge.

Dear Pawan,

Please check our instrument development work available on the following link:

Article Development of an instrument to measure Faculty’s informatio...

I hope it is helpful to you.

"In practical terms, when you adapt a questionnaire/survey or any other scholarly work, you are redeveloping it to make it better, so there is no difference between adapting and (re)developing in the behavioral sense."

This is not necessarily accurate and there are important distinctions. To start, it is important, in this discussion, to stay with "questionnaire/survey" and not exapand widely to "any other scholarly work. Thus, " Adapting" can mean making specific changes to an existing scale/measurment tool /questionnaire /survey in order to use it with a different population/group (than the one it was developed for) or otherwise make sure if will, "fit the lingual, cultural and environmental aspects relevant to the target participants of the study," as Fadia M. Nasser has stated. Quite often, I adapted exisitn measurement or questionnaires, simly from English to Spanish (not easily). This is different from and is not redeveloping to make it better.

If you are using individual items from several different studies and questionnaires, rather that ONLY ONE questionnaire, than you are, in fact developing a new questionnaire. The issue becomes, the "borrowing" every item or items from other questionnaires. I recommend that you compose several of your own items/questions, and possibly, use several others from one, two or three questionnaires. (is it possible to rephrase some itmes?) The reason is that, if a questionnire is published, it may well be copyrighted, and you need to contact the authors/publishers for permission to use their material.

Develop your own scale! What you're proposing is likely fraught with methodological challenges and ultimately compromises. Good measurement is the foundation of all research. Combining apples and oranges does not usually end in reliable and valid measures.

Dear Kenneth,

please note that there is a large literature on how to avoid exactly that (comparing apples and oranges). If he develops a new scale there is no way to connect his research with the research already conducted in other cultures wrt. to the construct in question. You are right, that this runs the danger of comparing apples and oranges, but as I said this can be avoided:

Spini, D. (2003). Measurement equivalence of 10 value types from the Schwartz value survey across 21 countries. Journal of Cross-Cultural Psychology, 34(1), 3-23.

Schaffer, B. S., & Riordan, C. M. (2003). A Review of cross-cultural methodologies for organizational research: A best-practices approach. Organizational Research Methods, 6, 169-215. doi:10.1177/1094428103251542

Stark, S., Chernyshenko, O. S., & Drasgow, F. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91(6), 1292-1306.

Steenkamp, J.-B. E. M., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25, 78-90.

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-69.

Best,

Holger

Holger Steinmetz, as I understand measurement invariance, it means that measurement issues should not influence responses to a question among different subgroups within the larger sample. This issue strikes as unlikely to depend on whether items are measured with 5-point versus 7-point response categories. Instead, it is more likely to be due to question wording.

But perhaps "metric non-invariance" is something different?

Otherwise, in my survey research training, I was taught that there was a strong preference for using the same Likert response format throughout the questionnaire, in order to make things as straight forward as possible for the respondent.

David,

I do not quite understand what you mean with "measurement issues should not influence responses". If you mean response styles, these could probably go into the indicators' error terms (unless two or more indicators share the response style, which is a problem for the measurement structure.

The key issues of invariance testing (for 90% of applications) is:

a) is the measurement structure the same? Depending on the degree and kind of violation, you may measure completely different things.

b) are the factor loadings the same? This is important for two reasons: 1) Loadings assign the metric to the latent variable; without equal metric there is no way to compare anything quantitatively. 2) Loadings are the presumed causal effect of the latent on the indicator, and thus a means to evaluate the validity and meaning of the latent. Unequal loadings may indicate that - although the structure is comparable, you "have a different fish in your net". This is metric invariance.

There is, however, a debate, how to approach these comparisons, wrt to whether to apply significance testing of a strict "no-difference"-assumption or rather a "degree of difference"-perspective. Since later substantial difference tests (e.g., moderator effects) are tested with NHST, I would tend to apply the same procedures to its prior tests of the basic assumptions.

Cieciuch, J., Davidov, E., Schmidt, P., Algesheimer, R., & Schwartz, S. H. (2014). Comparing results of an exact versus an approximate (Bayesian) measurement invariance test: A cross-country illustration with a scale to measure 19 human values. Frontiers in Psychology, 5(AUG), 1-10. doi:10.3389/fpsyg.2014.00982

With regard to you opinion of using the same metric throughout the survey, I agree while at the same time knowing that this enhances the danger of response styles to influence questions belonging to different constructs.

A much more important means would be to dramatically reduce the length of the questionnaire to incorporate golden standard indicators tested with regard to understanding and validity prior to the main survey,

Willis, G. B. (2005). Cognitive interviewing: A tool for Improving questionnaire design. Thousand Oaks, CA: Sage.

Groves, R. M. (1996). How do we know what we think they think is really what they think? In (pp. 389-402). New York: Russel Sage.

This, however, is in conflict with psychologists / sociologists deepest truism that you need many indicators to measure something. As most of psychologists do their research with students, and ever with "normal people" and almost never conduct presting, this i no surprise. I recommend to ever go through a questionnaire together with a respondent, you will suddenly experience, how good this works :)

Here is a (in my estimation) fruitful counterattack:

Hayduk, L. A., & Littvay, L. (2012). Should researchers use single indicators, best indicators, or multiple indicators. BMC Medical Research Methodology, 12(159), 1-17. doi:10.1186/1471-2288-12-159

Best,

Holger

I think that you would need to change the questonnaires to adapt to your prograom of research and then conduct the pilot study to test ICC or Crobach alpha for the revised instument's reliability. For validity, it is better for you to conduct factor analysis (CFA) using the revised instument; however, if you would just change (or delete) several questionnairs, you do not need to do the CFA. Just please mention that the original instrument has been verified to have a satisfactory validity (AAA, year) in the method section, and in the discussion, the exact validity test (CFA) about the revised instrument is warrnated in the next study. Hope this can be of help to you.

Cronbach's alpha for reliability of the items of course.

The analysis part depends on how you hypnotized the facts.

There is a book that might help you in order to theoretical understanding:

Contemporary Human Resource Management by Adrian Wilkinson 5/E chapter 4.

Holger Steinmetz, as far as I can tell, your version of "invariance" is indeed about invariance across subgroups within the overall sample, and I don't think that is really the key element in this question.

I also doubt the value of single item indicators, because there is no way to assess their reliability.

Where we do agree, however, is on the importance of cognitive interviewing as a form of pretesting. Since I don't think this topic has come up in this thread, it is important to emphasize its value.

Hi David,

you are absolutely right. Shame on me. I was triggered by the former response to my initial posting. Oups.

Best,

Holger

There are many guidelines available for translation and adaptation of a questionnaire from its original language (usually English) to a target language (your country national language). Examples include: (1)Beaton, D. E., Bombardier, C., Guillemin, F. & Ferraz, M. B. (2000). Guidelines for the Process of Cross-Cultural Adaptation of Self-Report Measures. Spine, 25(24), 3186-3191. (2) Guillemin, F., Bombardier, C. & Beaton, D. (1993). Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol, 46(12), 1417-1432. Both are the most cited papers for translation and adaptation of scales to another language. Generally, for adaptation, you need to carry out forward translations (at least two), synthesis of forward translations into one common translations by an expert committee (consist of translators and expert of the fields), back translation of the common translation to original language (English), expert committee review to finalize the translation, pre-testing of the translated version of scale, another expert committee meeting to finalize the scale, and lastly validation of the scale of its reliability and validity.