Hello everyone. I made a convolutional neural network in php with backpropagation and gradient descent. I then downloaded a training set containing dog images and trained the neural network so that all the errors were close to 10 ^ -6 for each image in the set. To save computing power I calculated the SVD of the grayscale image and trained the neural network on the diagonal (the sigma matrix), normalizing all values to the largest eigenvalue. The result was not very good, because when I tried to refer a cat it was recognized as a dog. Do you think it is impossible to train a neural network in this way? Am I bound to pass all the pixels of the grayscale image of the training set one by one? In this case do you recommend to dimensionless with respect to the maximum value (255)? Thank you so much in advance for your contribution!
I can't tell you which is better, the sigma matrix or the raw image pixels, because it all depends on your specific problem; you need to test that yourself. However, I can see that you may have a data scale problem when using the diagonal normalized to the largest eigenvalue, i.e., your data is normalized between [0,1], probably with a considerable number of high values close to 1. The popular neural network libraries are usually adapted to deal with images and their corresponding usual values. Therefore, the synapses/filters are randomly initialized in the range [-0.1, 0.1], or maybe something smaller, closer to 0. Applying this to your data may be causing problems during gradient descent, such as vanish and/or exploding gradient, leading neurons not to train properly.
In this sense, it is highly recommended that you do a z-score normalization on your data right before training. This consists of subtracting the feature-wise average (zero mean) and dividing by the variance (unit variance). After doing that, you may succeed in training the network and answer your primary question: is the sigma matrix better than all the pixels? If it contains sufficient information, you can expect similar results. If you get worse results, you may evaluate if the trade-off between accuracy/computing power is worth it.
Very strange. Why do you use a slow language like php to build a neural network at all? And why do you think SVD decomposition work! Image identification is subtle and you are going to loose lot of information at the stage of SVD decomposition and normalization. It is like blind fold neural network.
I strongly recommend you to use python and keras, tensorflow or pytorch and get the job done easily. Is this an assignment? Otherwise don't waste time using php and SVD.
Generally, I would not recommend a language like PHP for something for training neural network.
When you are already getting not good results then this could imply that your method (SVD) is not suitable for representing the input data well to teach the model. As answered previously, you are losing lot of information which may result is losing the ability of the model to distinguish between classes
Thank you very much. I had to abandon the SVD path because, as you suggested, there was too great a loss of information. Using the individual (normalized) pixels as input worked. I am using php because I wanted to try to write the neural network (including backpropagation) and upload the results to a mysql database. Thanks again for your invaluable help!
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training