The point of the factor analysis, as I was taught, is to reduce the total size of your scale, and help with content validity (i.e. are you really measuring what you think you're measuring in the scale). In this case, your scale is already created, and how you choose to defend validity is up to you. I was taught the old methods of context, construct, etc validity, but last time I brought up factor analysis for a scale as a referee, I was dismissed as "old school" and a paper that I thought was negligent was published in a highly rated journal.

I can't recall off-hand if they mention the evidence behind factor analysis, but the Norman book is my go-to and the Bible of scales in my field:

Health Measurement Scales: A practical guide to their development and use. Streiner and Norman. http://www.amazon.com/Health-Measurement-Scales-practical-development/dp/0199231885 

so, my short answer would be: it depends on your referee.

Dear Dr. Philips, thank you very much for your anwer.

Ömer, if a questionnaire has already been validated then there is usually a copyright on this questionnaire that should be regarded. One cannot just delete some items or re-formulate items, but one has to use the original questionnaire. If you find out that the questionnaire’s psychometric properties are not satisfactory in your sample, you could develop a new questionnaire based on the items of the well-known questionnaire. But then it is necessary to contact the copyright holder to get permission for doing so, which may be either the author or the publisher.

Therefore you cannot and you should not change Maslach’s Burnout inventory.

Regards, Karin

Dear Mr. Şahin, thank you for your answer. 

Dear Dr. Schermelleh-Engel than you for your answer.

Dear Mr. Ömer Akgün Tekin, thank you four your question.  

In your studies where the factor analysis did not hold up, were the data were collected in Turkey or another non-English speaking country? It is not unusual for this to happen when scales are translated or used in countries that are culturally dissimilar to where the scale was developed. The article below discusses such issues, and gives sources for more information:

Spector, P. E., Liu, C., & Sanchez, J. I. (2015). Methodological and substantive issues in conducting multinational and cross-cultural research. Annual Review of Organizational Psychology and Organizational Behavior, 2(1), null. doi:doi:10.1146/annurev-orgpsych-032414-111310

In my opinion, after content analysis you should try confirmatory factor analysis.

Dear Dr. Spector thank you for your interest and answer. For example: the scale is translated to Turkish and translate - retranslate procedure, validity procedure and reliability procedure completed succesfully by an author and he published this work. He says that, the Scale consists of 40 items and 5 factors when we test it in Turkish culture. After that I used this scale in a Turkish sample. In this point do I have to use exploratory factor analysis again? (If I do not have to I need a citation for this information.) In this condition I always do exploratory factor analysis both in Turkish and Non-Turkish scales. But  I usually see that the factors and items do not divide like theoretical framework. For example. I used this scale (the author says that it consists of 40 items and 5 factors) in Turkish sample. I did exploratory factor analysis and I saw that the scale divided into 11 factors and some items are gathered under meaningless factors. What should I do ? In this condition I always delete some items and try to find meaningful factors by repeating factor analysis and sometimes I lose too many items from the scale. Am I on the right way ?

Thank you for your interest again.

Dear Dr. Özmen thank you for your support.

Dear Dr. Pandey thank you for your answer.

The answer I would have given when an active academic is still the same: If you have a theory- driven basis for handling a set of factors, spell it out and use it. Exploratory FA is an a-theoretical statistical exercise. Assuming what you describe is the practice of some journals, it's unfortunate. The question I'd leave you with is this -- how theoretically articulated is this Inventory?

One other piece of ammunition -- if this is a well-known inventory, cited in many previous works, (and it's consistent with your theoretical position) , those two facts alone strongly justify your use of it.

A different piece of research would explore the more interesting, broadly methodological question of twhether the widespread (hypothetically -- I'm not at all familiar with the inventory or this line of research) use of the inventory raises questions about the effect of its use on research findings -- a kind of meta-analysis of its utilization.

Ömer, let's review some points. (1) You have a theoretically supported questionnaire or test, (2) with psychometric validations for the original and translated versions, and (3) you want to apply it in another study.

In that case, exploratory factor analysis (EFA) should be ruled out since, as it has been said above, you are not developing a test or a theory, you are applying it to some particular use. What could be asked, and has some sense, is a confirmatory factor analysis (CFA).

CFA allows for evaluation of a priori defined structures, assessing the global goodness-of-fit, and particular areas of constraint in the model. This CFA lets you say "the answers/scores to the questionnaire/test behaves reasonably close to what we expected". You could use some guide from Brown (2015).

EFA is not useful because it doesn't help to build significant knowledge over previous research. Probably, your data is best explained by other factor structures, but the question is: could it be reasonably explained by the previously validated structure? CFA is not necessary if you can show your sample is independent (socio-demographically equal) to the original validation sample. But if it's not, or you can't demonstrate it, is better to show through CFA that your sample scores/answers behaves similarly to the original validation sample scores/answers, and therefore could be interpreted with the same theoretical framework, empirically sustained.

What you need to demonstrate is not EFA results, but that there is empirical evidence that the questionnaire/test with it's associated factor structure is valid for your sample. CFA is better for that, and maybe you don't need it if you can argue that it has been done in a representative sample of your sample's population.

Hope it helps! And if you need help with the CFA, you can write me a personal message.

Brown, T. A. (2015). Confirmatory factor analysis for applied research. London: the Guilford Press.

Dear Dr. Althauser thank you for your answer.

Not an expert in this but will you use the scale in its original language or translate it?

Translated scales need to be validated.