Morphological analysis could be helpful to you.

You can analyse quantitative studies with very little participants. There are many options for handling your non normal data. Some tests, including the one sample Z test, T test and ANOVA assume normality.

However, if you have a small sample, you may want to run a non parametric test. A non parametric test is one that doesn’t assume the data fits a specific distribution type. Non parametric tests include the Wilcoxon signed rank test, the Mann-Whitney U Test and the Kruskal-Wallis test.

A sample size of 11 is inadequate for correlation because of very low statistical power and imprecision in the estimate of the correlation between your two variables in the population you are studying. I see two possibilities. First, 11 and 14 are enough subjects to estimate and compare means, perhaps with a t-test. Does it make sense to compare leaders and followers on any of your variables? Second, you could treat this as a qualitative study and content analyze the open-ended responses. Perhaps there are some useful insights in those answers.

Sekaran (20013) wrote:

"Roscoe (1975) proposes the following rules of thumb for determining sample size:

1. Sample sizes larger than 30 and less than 500 are appropriate for most research.

2. Where samples are to be broken into sub-samples; (male/females, juniors/seniors, etc.), a minimum sample size of 30 for each category is necessary.

3. In multivariate research (including multiple regression analyses), the sample size should be several times (preferably 10 times or more) as large as the number of variables in the study.

4. For simple experimental research with tight experimental controls (matched pairs, etc.), successful research is possible with samples as small as 10 to 20 in size."

Reference

Sekaran, U., 2003. Research methods for business: A skill building approach. John Wiley & Sons.

Thank you all for your advice. Paul E. Spector I think I understand what you mean, although these two groups are actually meant to be combined into 1 group (leaving only 14 subjects in the end). There are two groups because one are leaders and the other are employees (so 2 responses per company, which are then averaged into 1 'respondent' per company), which were meant to act as a way of verifying whether or not both respondents agree on the leadership style in their company.

I have done some research* and found that perhaps the best options in this case (low amount of participants, not a normal distribution, independent measures, ranked categorical data) would be to use the following tests (also mentioned by Mahfuz Judeh);

• Mann-Whitney U test or Kruskal-Wallis test (to test whether two or more groups are different)
• Binomial test (to test whether proportions in groups really are different)
• Kendall's Tau test (to examine for partial effects of a third variable when examining a correlation between two other variables)

I hope this will work out, as I have never used them before so still not sure if they are the right ones for this kind of data.

(*) Todd D. Little (2013) The Oxford Handbook of Quantitative Methods Volume 2, statistical analysis. Oxford Library of Psychology, Oxford University Press.

Dear Lorenz Kempeneers ,

I agree with Dr. Spector on the fact that your sample is too small for doing correlational analysis. However, that doesn’t mean that you can’t work with your data and provide insightful information. Since your survey included 60 questions, maybe you could describe and analyze the responses to each item (or some of them) and then compare them with the information from the open-ended questions. In doing so, you may find some inconsistencies between the participants’ responses to the standardized items and the open-ended questions that may be worth to be discussed. In addition, you could use non-parametric tests (e.g., Mann-Whitney U test, Kruskal-Wallis test) to explore potential differences between two or more independent groups. If you decide to work with categorical variables (e.g., ordinal, nominal) instead of quantitative variables, another alternative is creating a contingency table and then conducting a Chi2 test or a Fisher's exact test. I hope this helps! Best of luck with your research!

When you are working with sparse data you may be able to derive conclusions if you introduce uncertainty.

A suggestion: You can use Qualitative Comparative Analysis/fuzzy-set Qualitative Comparative Analysis (QCA/fsQCA), which can handle small sample size (eg, Ragin, 2008; and many other related resources on the internet, including fsQCA package).

Ragin, C. (2008), Redesigning Social Inquiry: Fuzzy Sets and Beyond, University of Chicago Press, Chicago IL.

Mine, first take in consideration the possibility to treat missing data with modern methods and then make the analysis suggested. Graham book "Missing Data" or the paper in Annual Review are good. Graham, John W. "Missing data analysis: Making it work in the real world." Annual review of psychology 60 (2009): 549-576. Also, Rubin's work.

If your considered it, SPSS or MPlus could be useful. I used SPSS, I know by reading that it could be possible with MPlus also.

Tries to use the "hermeneutic single case efficacy design" applied to small group.

See this article that I have in my profile.

Non-parametric analyses are recommended with a sample size of 11 or 14, respectively. Regression analysis can be used as there are no non-parametric equivalent for some of those tests and you have a lot of items. When sample size is less than 30 we use non-parametric analyses in Finland as a rule because we then consider the sample to be non-normal (In Sweden, they define "ordinal" in Likert scales differently and use non-parametric test for hundreds of part takers). The results between parametric and non-parametric analyses do not differ much in the end. I am sure you find significances also. You can also follow up your non-parametric resukts with trend analysis. If you use Kruskal Wallis ANOVA for unrelated groups (different subjects) you can follow up with Jonkheere trend. If you test same subjects for different questionnaires you use Friedmann ANOVA and follow up with Page's L-test. If you want an easy book how to calculate this (even by hand) you can read J. Greene and M. D'Olivieira (last edition): Learning to use statistical tests in psychology, Open University Press. Look at some of my examples:

Only 10 children survived congenital coagulation disorders in my research: School Absences and School Achievements in Children with Congenital Coagulation Disorders

S. Béatrice M. Kvist Pages 73-76 | Published online: 07 Jul 2006

Abstract

Kvist, S. B. M. 1988. School Absences and School Achievements in Children with Congenital Coagulation Disorders. Scandinavian Journal of Educational Research 32, 73‐76. Ten children suffering from congenital coagulation disorders (CCD) ‐haemophilia A and B and von Willebrand's disease – were compared with normal, healthy schoolmates with reference to scholastic achievement and school absences. The results indicated that in spite of a greater number of absences, the CCD children succeeded very satisfactorily in school. Problems related to need for attention, exclusion from certain school activities, and self‐provoked school absences were discussed.