Hello everyone!
Another artificial neural network (ANN) question. I am using an ANN for classification. In order to determine the importance of each hidden node for the output node I am basing my decision on the values of the connection weights.
I am reading some papers and I found that different authors accepted different treshold values to determine how significant each node is to each output (e.g., 0.3, 0.4, 0.5...). What are you thoughts on this? Is there a comonly accepted minimum value for significance? Any important reference that suggests one? Thanks!
.
as long as you only consider the contributions of the hidden nodes (assuming only one layer of hidden nodes) to the output, you could adapt the deviance and likelihood test (traditionally used for logistic regression, for instance), with all the caveats which apply !
http://en.wikipedia.org/wiki/Logistic_regression
.
I think the values depends on your training of data, hidden nodes, number of iterations and type test..
I agree with Alamgir Khan. The threshold value depends upon your requirements and the model you developed for a particular type of data. Since ANN is model based and there is no standard unique models are available, it is required to adopt some own criteria to get the required output. The hidden layer is not same for all and it is according to your precption.
An excellent question Rita.
I suggest to use the Self-Organizing maps ANN. This unsupervised ANN structure does not have hidden layer so you do not have to worry about setting the weights for the connections all are self updated. When a pattern is presented to a self-organizing map neural network, their neurons compete among themselves in order to determine the one that has the best response to the input pattern. The neuron that generates the least Euclidian distance between the input and weights vector is the chosen winner, and it has its weights adjusted to respond better to the input pattern.
Hi Rita,
My 2 cents. You can try to look at knowledge extraction from trained ANN. Google for "rules extraction neural network".
Unfortunately AFAIK there are no available libraries that can perform such analysis (I'm working on this), but I think you can get general idea of how people are dissecting ANN's to explain classification.
Best regards,
Andrey
Let the output of your hidden node activation matrix be H and your target variables coded in one-of-C (1 for class c, 0 for the rest) or one-versus-all (1 for class c, -1 for the rest) be T. Extreme learning machines paradigm tells you that your optimal projection is based on Moore-Penrose generalized inverse: Beta=pinv(H)*T;
Here Beta is a discriminative projection. We have shown that discriminative projections provide very valuable information for feature selection (even better than the projection itself), in this case neuron selection. You can compute the norm of row vectors in Beta and rank them to get the saliency weights of each hidden node. Hope it helps.
Well, an interesting question. However, a little bit tricky because the weights depend on the topolocical structure, the learning data, how you train, etc. This is exactelly one of the drawbacks of ANNs. You got weights but there is (more or less) no information about the meaning. My recommendation is instead of applying classical ANNs take a look into Kernel-ANN because they are better explainable. Or use an completely different approach for classification, like the family of SVM, etc. Kernel-Methods or the like.
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification