I have n classes plus one unknown. Unknown is not included in the training set as the way is not explored as yet by me.
I trained the network for n classes. The confusion matrix is very good.
Now if an unknown class object comes in for prediction, the neural network predicts it as any of the n classes. Which is clearly misclassified. The confidence also comes near by 0.998, which makes it difficult to filter out. Otherwise known class object of n trained classes is classified at same confidence.
I tried keeping a class which does not include any feature set of n classes i.e. sort of negative sampled class as unknown class. As output, the confusion matrix goes very bad. Bad enough not to go further with that. This I am still going through.
How to determine unknown class using neural network? That it doesn't fall in any known class classification.
I am using MATLAB ANN toolbox for now and FANN too. FANN works great.
If it is unexplored und unknown, I would avoid to use first a neural network, and rather use a more simple method. Then once this 'unknown' can be somehow defined, it can be included in the training.
A one-class SVM is a good choice for detection of outliers. But as your network seems to be sure the unknown pattern belongs to one of your known classes, have you checked how similar it is? If you compare the similarity between patterns with different known labels, and the similarity between the "unknown" patterns and the labeled ones, you might get an idea how similar they are.
Make sure you have the efficiencies under control, you may not want some entries of a known class to fall into the unknown class und thus causing false-positves...
for any un known class ANN never rejects it but assigned it to the own classes. So u can also train ur ANN with random unknown class data. SO when any unknown data came ANN will predict it as unknown class.
I think you should use an unsupervised learning algorithm (see Kohonen). Another approach might be based on a Bayes classifier.
One approach is to have one binary output node for each class. After the network has been trained you can use some threshold value for membership to that class, i.e. a sample with the output (0.1, 0.98, 0.2) most likely belongs to the second class. This way any sample that does not belong to the other classes can me marked as unknown. For example if the output of the network for a sample is (0.15, 0.21, 0.4) it would not belong to any of the other classes and would be "unknown" or "not-classified".
Pay attention with the Bayes methods, so that your result do not be biased! There is no objective priors
You might use reconstructive neural nets like restricted Bolztman Machines or some other variants. The basic idea is to model the data generatively and measure the model quality with reconstruction error of that instance form the model. If you are able to model the data well than the outlier instances (in you case unknowns) might be observed by high construction error.
agreed with Dr Marko that there are ANN that can reject unknown patterns not trained with. But an easier approach to solve your problem is either to use Unsupervised type ANN or define some features that are more closely resembles the unknown patterns and train the supervised ANN on those
Yes, you can use the free Som toolbox for Matlab for unsupervised classification based on Kohonen Maps
My experience is: Instead to classify n-class, a more effective strategy is to use hierarchical binary trees. The strategy works this way (having n-classes):
Level-1: classify everything belonging to class-n and NOT class-n.
Level-2: After to classify the entries from the class-n, the hierarchy of second level is applied. Classification to entries from the class-(n-1) and NOT class-(n-1).
...
Level-j: Classification to entries from the class-j and NOT class-j.
Do this up to j = N.
For the classifier itself, the SVM looks superior than multi-layer perceptron or RBF-NN.
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification