24 cases and 80 variables? My best advice it to tell you that you should not waste your time in analysing this data. Thereby I assume that you do data mining, that is, the data is not obtained from a planned experiment with a specific question in mind. Whatever you will find, no matter how "significant" will likely be a spurious result. If you anyway want to go for it, I would think of a logistic regression model.

Hi,

Kindly be clear; do you have a binary dependent variable or a binary classification task (e.g., yes/no)? You have to know your data well in order to apply the suitable learning scheme.

Generally, you can start with linear classifiers. If that doesn't work well, try kernel methods and neural networks.

Some considerations:

- You could try to increase the number of your instances and explore their distribution in each class as well as their effect on the overall training process.

- You may reduce the dimensionality of your data.

- Build your predictive model under the evaluation of, e.g., k-fold cross-validation.

HTH.

Samer, PhD.

24 cases and 80 variables? My best advice it to tell you that you should not waste your time in analysing this data. Thereby I assume that you do data mining, that is, the data is not obtained from a planned experiment with a specific question in mind. Whatever you will find, no matter how "significant" will likely be a spurious result. If you anyway want to go for it, I would think of a logistic regression model.

Hi Diletta,

I think it depends on the purpose of your research. If you just want to find a way to describe the relationship between your independent and dependent variables of your data, it should be totally fine to try different statistical models and choose the most significant one.

However, if your purpose is to build a predictive model, then you'd better to get an independent set of data to validate your model.

Best,

Wen

That is a very small training data set. It looks like you need something like function approximation, however with such a small data set I am not sure you will get meaningful, significant, results. You could try techniques such as cross-validations or use synthetic training data.

You may try PLS or analogous methods that extend ordinary or generalized linear model for « more variables than observations » cases. Of course, with such a small sample size, interpretation — and especially avoiding overinterpretation — will be much harder than running the statistical method in a software...

While I tend to agree with Emmanuel if asked on general terms, there are several other options. Unfortunately, it is futile to discuss these in absence of any knowledge of the data types, research stage (exploratory or predictive) or research interests involved. Some of these options could be embedding (ie. understand the data as lying on a manifold in an 80-dimensional vector space and embedding this manifold in a lower, more appropriate (at less than 24) dimensional vector space), other options can exploit specific structures of the questions to be asked in order to approximate a suitable model and other again could pertain to feature reduction or engineering until the need of the research purpose is fulfilled.

Hi Ajit Roy,

I will recommend your answer because of two main reasons:

1- reading all the answers, and then

2- bringing *new* reasonable contribution to the list of answers.

Thanks for your contribution.

Cheers,

Samer

Decision tree methodology is a commonly used data mining method for establishing classification systems based on multiple covariates or for developing prediction algorithms for a target variable. This method classifies a population into branch-like segments that construct an inverted tree with a root node, internal nodes, and leaf nodes. The algorithm is non-parametric and can efficiently deal with large, complicated datasets without imposing a complicated parametric structure. When the sample size is large enough, study data can be divided into training and validation datasets. Using the training dataset to build a decision tree model and a validation dataset to decide on the appropriate tree size needed to achieve the optimal final model. This paper introduces frequently used algorithms used to develop decision trees (including CART, C4.5, CHAID, and QUEST) and describes the SPSS and SAS programs that can be used to visualize tree structure.

If you want to explore the relationship among variables for smaller sample set, less than 1000 you can go for fisher's test but if you want to model the relationship then go for binary logistic regression or classification tree.

For modeling part only that much sample won't be enough to get a strong model such as if you want to model using ensemble learners boosting, bagging or random forest. Even Neural Network needs high sample for making a good predictive model.

I will suggest to read "Hand book of bio statistic" a book written by JOHN H. MCDONALD, who has thoroughly discussed the relationship exploration techniques for small as well as large sample data-sets (categorical) using Chi-squared test of independence (for large sample) as well as fisher's test (for small sample

With such a data set, you may find statistically significant results which are also total nonsense. You would do well to read Wainer's article "The Most Dangerous Equation" to see the effects small sample sizes could have on your conclusions. Sometimes your sample is biased, and you simply can't do anything about it. So I would proceed with extreme caution.

I would first reduce the number of features through Principal Component Analysis or something similar (as in Kazimir Menzel's answer). Regardless of the fitting algorithm, I would also do a stratified leave-one-out/ 5+fold cross validation to make sure that I am not overfitting my model. How balanced is the data? If you have something like 20 in class A and only 4 in class B, there is very little hope of getting a meaningful result

As to the algorithm itself, with such a small data set fancy things like neural nets won't work. It is almost too small for regression if you have testing and training sets, I think. Bayesian methods work well on small data, but I would try a bunch of different algorithms, and compare their cross-validated scores (ROC AUC or confusion matrices) and see which one works - the no free lunch theorem holds.

Statistical analysis/ data mining technique depends on your output. Small sample size is not a problem.

You're welcome, Dr. Ajit.

Cheers,

Samer

You can try non-linear regression model if other basic assumptions are fulfilling. As your dependable variable is binary, so as other independent variables as you mentioned, I think logistic regression model will be appropriate as suggested by Dr. Wilhelm and Dr. Roy.

thanks for all