Dear @Zeeshan, I do share the opinion of  @Subhash! Results will be better, in general, if more data are available. But, it realy depends on type of your research and methodology! Do not neglect the quality of data! Better Data: Better Research!

The unit of analysis of your study (Organisational or individual) will determine the number of responses to collect from one organisation

Seriously? This question would be determined by the purpose of the study in context - e.g. what is collected data intended to be used for and in what context and so on. The purpose of the study together with the actual questions themselves in the context of the research question. So the numbers are completely irrelevant by themselves - it is what they are assigned to mean. The questions asked *and* the relevance of the answers must be related to the purpose of the research. So what does it matter if you have hundreds of irrelevant responses from the 'wrong' individuals? And what does it matter if you have hundreds of 'wrong' questions answered by the correct individuals, and so on. This issue is so intertwined with your research purpose (and research philosophy) that it is not meaningful to answer outside of the context of your research context. As you ask this question I would assume you are not (directly) replicating some other previously made research for comparative reason (or you are not aware of previous research in your exact research question).

More the responses better will be results in behavioural studies. If you collect two to five responses, you may increase the number of organisations. If you collect more than five responses from an organisation, you may reduce the number of target organisations. It depends on type and nature of study and situation.

Dear @Zeeshan, I do share the opinion of  @Subhash! Results will be better, in general, if more data are available. But, it realy depends on type of your research and methodology! Do not neglect the quality of data! Better Data: Better Research!

Hello, Zeeshan -

I am not sure if you phrased your question correctly. You refer to your questions as "variables"; however, a question is actually referred to as an "item". For example, a questionnaire instrument that gathers data on a variable such as "job satisfaction"  might have 10, 20, 50 or even 100 questions/items that pertain to that variable.

In another example, a variable such as "Personality Traits" might be composed of five factors such as Extraversion, Agreeableness, Conscientiousness, Emotional Stability, and Intellect. Each of those five factors are sub-scales of Personality Traits and each might be composed of 10 to 20 questions/items that might be measured on a 5-point Likert scale (1 = Totally disagree", 5 = "Totally agree"); thus, you can end up with 50 to 100 questions/items. So you see, the questions are not the variables, they only seek to collect data on that variable.

When all is said and done, depending on your study and its design, you might actually have less than 10 variables. In the example of the personality traits, you will have 5 variables. Add that to however many dependent variables and/or other mediating or moderating variables that you are using in your study and there you have your total amount of variables.So my point is, make certain that you are not confusing number of questions with number of variables.

Now, as far as sample size, G*Power3 (link provided below) is a free online program that allows you to calculate sample size. It comes with a guide that you can download. Alternately, you can apply heuristics (rule of thumb) from the literature. For quantitative studies, some suggest 10 per variable, others suggest 15, 20, or even 40. Certain types of multiple regression require a minimum of 300 (e.g. Aguinis et al, 2010).  Sample size is important because of such things as bias and Type 1 errors, etc., so consult the literature for advice concerning your particular study design.

Best Regards, and good luck with your thesis.

Ruth

Link:

http://www.gpower.hhu.de/

Hello, Zeeshan,

I am a bit confounded by your perception of questions and variables. Will each question represent a discrete variable ? I think closer reexamination of your research design may reveal bigger issues than your N (no pun Intended)

Mr. Subhash and Ljubomir thank you for your answers! And to all of you.

Mr. Peter and Ms. Ruth thank you too, however, I wish I could share with you more details about my thesis that would avoid any misunderstandings. Unfortunately, I couldn't prior to my study is done and my request was only regarding how many multiple responses (my unit of analysis would be Managers, Asst Manager, and Supervisors working in SMEs) can be collected from one organization for better quality of research. And regarding irrelevant responses I am not gonna get irrelevant responses for sure but it was not the intention of question.

I am quite sure that one of the first things a reviewer of a publication would ask for is related to *validity* - this would be directly related to the arguments underpinning *why is this relevant* and in other words *purpose*. So the number of questions, responses, people and number of organizations etc. will not *by itself* make *any* research relevant. Without a stated and justified purpose the quantity of any data is by definition irrelevant. So the answer to the question made will be *totally* dependent on context of the research agenda and purpose. THIS however was *not* described in the question - so any *specific* advice given (answer to the question) is in practice contextually unjustifiable by definition and so not dependable. The dilemma with this question is that the question cannot be answered with a specific answer that is contextually relevant to the researcher who asked the question. To pretend otherwise would be academically unjustified and unprofessional and from my point of view highly irresponsible as it would be misleading the person asking the question.

Ideally, if the unit of analysis is the organisation, you will need to have one respondent per organisation-you will need to identify that one person who is best positioned to provide answers to your questions in the organisation. As your unit of analysis would be employees holding supervisory positions as stated above, the best approach is to identify the various levels of supervisory positions and select a respondent per position. The more positions you identify in this case, the larger your sample size and the better your results will be. However, the practical challenge is how to identify the positions in organisation you are not part of.

From my point of view: There is no universally valid relationship between number of respondents and number of organizations. Even if the unit of analysis is *the organization*. It is quite possible to have respondents who provide answers for more than one organization. It is possible to have more than one respondent for any one organization. Further more it is possible to have no respondent for any one organization - as it is possible to access data about activities without involving respondents. Also it is possible to collect data through observation without any respondents. It is possible to have respondents from external sources and organizations. And so on.... Why -  "ideally the unit of organization NEED (?) to have one respondent per organization"  - is as a *generically* valid and *universally true* (?) proposition completely baffling to me. Why is anything chosen as a unit of analysis? The answer to such questions will also influence the method application and boundaries of inquiry including selection of relevant variables to use and which metrics to apply. The practical challenge in this research is to reflect over the purpose behind the research and the arguments for its relevance. Until then explicit advice about *ideal* contextually relevant inquiry will be meaningless and so without practical value. They will certainly not be ideal without contextual knowledge. If there is a purpose - it needs to be explored as part of the application of method and problem space for inquiry. If there is no purpose then the method of inquiry will matter little before hand - as such the research is not necessarily meaningless - but the meaning and relevance of it will have to be created through the retrospective analysis of the data and its situational validity and the boundary setting of the research question.

I agree with Ruth and Peter'. To add, when we make sample size calculation, our basis is our population's description and size. We therefore ask this question, is "organization" our population to base our sample size calculation? Or is it the number of responses? You have to clearly identify your variables and your primary source of data, and your unit of analysis.

I have a somewhat different take on this as you are really doing a two stage multilevel design where level 2 is the organization and level 1 are the multiple responses which can be analysed in a multivariate multilevel model.

There are a number of resources to help you design the study to maximize statistical power given the resources - in the following Optimal Design software you can put different costs for  obtaining different orgainizations and responses - the latter may be quite cheap compared to the former:

Optimal Design for Longitudinal and Multilevel Research Documentation for the “Optimal Design” Software

http://www.stat.cmu.edu/~brian/463/week14/od/od-manual-20111016-v300.pdf

http://sitemaker.umich.edu/group-based/optimal_design_software

This site explains some of the ideas behind the multivariate multilevel model

http://www.bristol.ac.uk/cmm/software/mlwin/mlwin-resources.html#multipleresp

The approach can be very general so that you can have a matrix sampling design in which some questions are asked of every organization  while more detailed questions are asked of  random subsets (but 45 is rather low for this!). The entire dataset is then analysed as a single multivariate model with multiple responses and the full responses help estimate the subsamples in a nice example of borrowing strength.

FInally I cannot tell from your initial question whether you have a three level model in which 5 types of response are nested within n individuals in m orgaizations - it depends on your use of the word 'responses' - does it mean multiple dependent variables (that is what I have taken it to be) or simply multiple respondents being asked the same question. If the latter, it is a two-level non-multivariate multilevel model.

If all this sound rather complicated, please feel free to ignore it!   

In addition to considering the number of observations you need, I suggest paying particular attention to the specificity of the questions you seek to have answered.  If your questions do not adequately specify the information you seek, it will not matter how many observations you collect.

The question concerning the number of answers does stand on its own. You must consider the larger context of the design of your study. Issues like units of study, quantitative vs. qualitative etc have to be addressed. Do not mix up the research question with the questions in your questionnaire. Try to imagine what is link between questions you ask in the field and the question you want to answer as a researcher. To you work with variables, indicators? How to you combine different answers?

Dear Zeeshan

I shall respond you very interesting and relevant question from three diverse perspectives:

1) The significant portion for data collection has been addressed by yourself while  giving sample adequacy norm (Hair et al.).

2) The answers given by different RG friends are also relevant will add to the quality of work.

3) My perspective is that you have mentioned "Human Resource Management" as your research area.  Now within this broad framework the exact title of research will matter much. It depends upon the size of universe for the study. As the main purpose in sampling design is to ensure that the sample is representative of the universe. For example, if your study is a 'Case Study' then entire sample is to be taken from the same organization. Definitely you need to adopt 'multilayer sampling approach  i.e. if you plan to take 10 respondents from one organization, you may fix a ratio (2 respondents from top level, 3 respondents from middle level and 5 respondents from operative level (this is an hypothetical example). But the only requirement is that the sample has to be adequate and representative as well.

The unit of analysis and significance of the objectives are the deciders.