For small samples, usually t-test is used to compare the means. However, Logistic Model can also be useful if the assumption of normality is not valid.

In addition to what was said above, you can try the Wald test (logistic regression), or the likelihood ratio test. It all depends on the nature of the data under scrutiny.

Again, I agree that the sample size is too small to invoke any meaningful statistical test results.

Small samples are generally criticized for their small not being able to take the benefits of central limit theorem, i.e. in order to use distribution-based test, the sample must be at least 30 (generally). This does not mean that small sample cannot be tested. Small sample is a fact of life in the real world: blood sample, semen sample, DNA, cell sample, etc.

Comparing sample 1 and sample 2 may be accomplished by the F-test:

F = S12 / S22

... where S2 = variance. The main sample has n1 = 12 or df = 11 and sample 2 n2 = 6 with degree of freedom df = 5. From the F-table, the critical value for F(11,5) is 3.20. H0: F < 3.20 means no significant difference. HA: F > 3.20 means there is a significant difference between the two samples.

http://www.itl.nist.gov/div898/handbook/eda/section3/eda359.htm

https://explorable.com/f-test

Dear investigators,

Thank you all so much for your answers, I was going to use Fisher's exact test and likelihoo ratios with 95 %CI as well. I''ll also try to perform Logistic Regression. Samples are surprisingly normally distributed (by shapiro-wilk test and skewness&kurtosis test), but I really do not like to beleive these results because of small sample size.

I am afraid that roc-curve is not suitable to assess the sensitivity and specificity of the diagnostic test and I'd prefere to use LR+ and LR- instead. Will it be justified?

Later: I'll try one-way ANOVA as well

Later: Not bad choice for groups with samples lower than 30. Degrees of freedom (x,y) for my groups are 1 and 16, so F should be higher than 4.4940 (for alpha

Jakhongir,

if your data is of categorical type, then a good test to perform would be the Fisher's exact test. If your data is of numerical type and under the assumption that you have normally distributed variables regarding the small sample, you could use T-test based on the F-test's result of variance.

non parametric Man-Whitney

Jakhongir -

If you are using hypothesis testing, you should note that an isolated p-value is not informative.  I suggest that you might want to do an Internet search on "effect size." (See the first link below for an example article I found by doing such a search.)  You could consider what might apply to your particular application.

Because a p-value is a function of sample size, a so-called 'significance' level should be set accordingly. It is very bad, but unfortunately very common practice, to set that level at 0.05, regardless. 

A power analysis or other sensitivity analysis should accompany a p-value. (See the second link, which leads to a letter I have in The American Statistician.) 

Note that one might consider a properly constructed hypothesis test to be analogous to a confidence interval, when one can consider a point estimate to be bounded by a null hypothesis and a simple alternative hypothesis. 

Best wishes - Jim

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3444174/

https://www.researchgate.net/publication/262971440_Practical_Interpretation_of_Hypothesis_Tests_-_letter_to_the_editor_-_TAS

Article Practical Interpretation of Hypothesis Tests - letter to the...

Jakhongir -

If you want to compare the means of two continuous distributions - say for a given variable/question  - might you want to find a confidence interval about the difference in the estimated means?  If the point estimate of the difference in means has a reasonable confidence interval around it which includes zero, then you cannot say you have demonstrated a substantial difference.  Whether or not zero is included within the confidence interval will depend partly upon the actual,  unknown difference in means, and partly upon how much clean data you collected.  With so little data, you will only be able to show a low confidence for a difference that may be substantial, but at any rate, a confidence interval about such a difference in means will be practical to evaluate, more practically interpretable, and analogous to presenting both a p-value and type II

 error analysis together.

Cheers - Jim 

First of all, did you check normality? If the data non normal distributed then You use Man Whitney test which is the same of t test for two independent samples for a non normal data.

By the way, you can use other distributions besides normality in confidence intervals, but the usual examples assume normality or t.  You can also fall back on the Chebyshev inequality. 

Dear Dr. Knaub,

Thank you for your comments and advices. I've calculated effect sizes for my variables, using Cohen's procedure i.e. (Mean of Group1-Mean of Group2) / Average SD of whole study population. Did I do everything right? Is small effect size better than large or not?

Thank you in advance

The lower the effect size, the less you can tell the sample means apart. 

If you can use confidence intervals, they would be most informative. 

Also, if you can provide graphics and descriptive statistics, they can be very helpful. 

My guess is that unless there are actually extreme differences between samples, you may need to consider this to be a pilot study, and plan more data collection in the future.

Best wishes on your project. 

Thank you so much!

Dear Jakhongir,

Your test choice will mainly depend on the type and distribution of variable you want to compare within the two groups. However, I would agree with the suggestion to consider your small sample as a pilot study that will help you design a future well powered study.

Dear researchers,

Thank you all so much! You've really helped me. I appreciate your contribution. Well, now it becomes more clear for me. I decided to use followings:

- Interclass correlation coefficients; 

- Responsivness (sensitivity, specificity, likelihood ratios, diagnostic odds ratio);

- F-test to compare two groups, because it is good enough for small sample sizes;

- Cohen’s d and correlation coefficient (r) proposed by Rosenthal and Rosnow for estimstion of effect size;

- Univariate model ANOVA for calculation of power of the diagnostic tests.

- Other tests when required.

Wish me good luck! I hope to share soon this manuscript with you and discuss.

Try this

Descriptive Inequality (Difference) and Inferential Tests of Inequality

Two- sample (bivariate) data

These tests are employed mostly in researches that attempt to find out if an intervention or a method that was introduced has enhanced or improved the performance of a target population when in fact the intervention was applied only to a sample of that population or if an organismic variable with dichotomous measure such as gender is a factor that causes difference in performance.

Tests of difference between two arithmetic means, and tests of significance of relationship, are the most popular tools used in researches apt for parametric tests. For large samples (30 or more), Z test is employed; for small samples (29 or less), t-test for dependent (or paired) samples and t-test for independent samples are used.

For non-parametric tests, Mann-Whitney U test is employed for two independent ordinal distributions (equivalent of t-test for independent samples), and Wicoxon T Test for two dependent ordinal distributions (equivalent of t-test for dependent samples) is used..

Descriptive statistics Measure of the variables and examples Inferential statistical tool to determine if the inequality in the sample means also exists in the population means

Inequality of the two arithmetic means, one mean is greater than the other, or one mean is less than the other where the samples are independent

Ratio

Years (as in age)

Pesos (as in income)

Meters (as in distance)

Points (as in score)

Kilograms (as in weight) Two- sample Z test for large independent samples (30 per group or more) whose feature being studied is in ratio measure

Two- sample t-test for small independent samples (29 per group or less) whose feature being studied is in ratio measure

(Parametric)

Inequality of the two arithmetic means, one mean is greater than the other, or one mean is less than the other where the samples are paired (dependent)

Ratio

Years (as in age)

Pesos (as in income)

Meters (as in distance)

Points (as in score)

Kilograms (as in weight) Two- sample Z test for large dependent samples (30 pairs or more) whose feature being studied is in ratio measure

Two- sample t-test for small dependent samples (29 pairs or less) whose feature being studied is in ratio measure

(Parametric)

Inequality of two proportions Nominal

Two- sample Z test for a large sample (30 or more) whose feature being studied is in nominal measure

Two- sample t-test for a small sample (29 or less) whose feature being studied is in nominal measure

(Parametric_

Inequality of the two sets of ranks, one set is greater than the other, or one set is less than the other Ordinal Mann-Whitney U test

(Non-parametric)

Inequality of the two sets of ranks, one set is greater than the other, or one set is less than the other Ordinal Wicoxon T test

(Non-parametric)

Thank you so much, Eddie!

Extrapolating from my background in patch clamping and calcium imaging, the first thing that I did was something simple but effective and that would be do a simple scatter plot of my data. You can use all the statistical methods you would like but showing the raw data as a scatter plot is really an effective method in presenting your data along with the choice of statistical analysis. It will give whomever you are presenting to a nice presentation of your data and will be easy to explain if only 1 point caused a deviation of your mean.

Dear Dr. Lester,

Bingo! Thank you so much. I've already done. Another Illustration in my draft manuscript is roc curve. These two charts are of most representatives for such kind of studies IMHO.

Thank you again.

check this https://reneshbedre.github.io/blog/ttest.html