You may want to try tree-based approaches. You can simply start with Classification and Regression Trees (CART), however there are certain problems associated with some of the tree-based approaches like your predictions are restricted with the range of your input data. This is not the case for all the tree learners.

I would suggest gradient boosted trees (especially stochastic gradient boosted trees -> http://astro.temple.edu/~msobel/courses_files/StochasticBoosting(gradient).pdf). You can handle categorical predictors with tree-based learners. Missing values are allowed. As a side product, a feature importance value is evaluated for each feature. They are good at handling interaction effects and so on.

Be careful about the number of levels of your categorical predictors. Some tree-based approaches are biased towards the features with larger number of levels when building the trees.

You may want to try tree-based approaches. You can simply start with Classification and Regression Trees (CART), however there are certain problems associated with some of the tree-based approaches like your predictions are restricted with the range of your input data. This is not the case for all the tree learners.

I would suggest gradient boosted trees (especially stochastic gradient boosted trees -> http://astro.temple.edu/~msobel/courses_files/StochasticBoosting(gradient).pdf). You can handle categorical predictors with tree-based learners. Missing values are allowed. As a side product, a feature importance value is evaluated for each feature. They are good at handling interaction effects and so on.

Be careful about the number of levels of your categorical predictors. Some tree-based approaches are biased towards the features with larger number of levels when building the trees.

I think decision tree approach will be reasonable. You can use demo version of See5 from www.rulequest.com

Border pairs method can analyze data before learning proces to find out how difficult they are. This method always found a solution and is resistant against the overfitting and is working well with small number of learning patterns. Paper with description is on my RG page.

If you can choose which training data to learn, try active learning techniques to get the most out of your

If existing techniques to learn from your small dataset dose not work, you can use semi supervised approaches. First over sample your data and then use semi supervised approaches.

My experience is that there is not much you can do without a feature selection (dimensionality reduction) phase. Data are too sparse and any result would be too noisy... it however depends on the detailed nature of the data and of the problem.

You need to use a very simple model, such as the best univariate model, or a mixture of univariate models.

You can try the MODL algorithm, which regroup modalities of categorical attributes (and discretize continuous attributes) using a MDL criteria, and then mix the obtained best univariate predictors using a selective naive bayes.

You can try the software here:

www.khiops.com

Search for one-sample problems and small sample problems.. Its a topic of interest to many people like me :) .. More than dimensionality reduction, probably the question is how robust the set of features you are able to obtain from a given training examplers are to natural inter-sample variability.

i think MADE algorithm using rough set will do

For training, decision trees may be the best solution, but all other proposed algorithms could equally be applied.

What you really need is to test all these techniques(or any other), and estimate the generalization error. For validating different algorithms and estimating the generalization error, consider the following procedure:

A) generating bootstrap samples, that is one of the best approaches when only a few samples is available for training.

B) estimating the error using some validation technique in a loop and repeat many times, as follows:

for i=1 to 1000 (the higher, the best)

shuffle dataset

divide dataset into training and test sets (90-10% for example)

train your model with some technique with training set

determine mean error in test set

accumulate error

end

error = error/1000

C) Apply this technique to any technique, to estimate its generalization error.

D) Select the best technique, depending on this value.

I had a similar issue several years ago, where my dataset consisted of 40 samples (each individual sample was very expensive to come by, which is why there were so few of them). I used the bootstrapping technique to create larger pseudo-replicate sets from this original dataset with resampling, and then used these new datasets to train neural networks. From this technique alone, my error rates plumetted from over 100% down to ranges between 10% and 20%.