Dear Lila,
Under the plots section you can see a button called performance. You have to click on that to get the performance plot. Anyhow the algorithm has converged at a performance of 0.0595 which you can see under the progress section. It is not a satisfactory. You may have to train your ANN more.
Regards
Yes. I agree with Lila. You might still need to train your network. The problem seems to be the validation checks. Increase them you are probably using the default which is about 50. If need be increase it to 100 (the italized parameter below named - net.trainparam.max_fail). I also advise my students to choose the training aprameters by themselves rather than use the default settings used to build the algorithm. Here is a short subroutine that you can use to adjust the parameters on your code.
net.trainParam.goal=0; % error goal
net.trainParam.epochs=500; % maximum iterations
net.trainParam.show=25; % showing intervals
net.trainParam.max_fail=50; % Change this to effect the validation
net.trainParam.lr=0.01;
net.trainParam.mc=0.9;
Test and validation performance seem to be okay. However training performance is very bad. This is because the training stopped when maximum validation performance achieved. Overall, an mse 0.0595 means rmse of 0.244. This means the model error is more than 20% which is hardly acceptable. Avoid data division and conduct training without supplying any validation or test data to MATLAB
Dear Lila,
I have tried to clean up you code. Check your inbox for details.
For other experts, I have changed the training algorithm to BFGS Quasi-Newton (trainbfg). Here are the rest of the parameters and attached are the training results.
net.trainParam.max_fail=50; % Change this to effect the validation
net.trainParam.lr=0.01;%learning rate
net.trainParam.mc=0.9;
net.trainParam.show = 25;% showing intervals
net.trainParam.epochs = 1000;% maximum iterations
net.trainParam.goal = 1e-5;% error goal
I hope it shall help you. Please do not hesitate to contact me in case of any problem.
Dear sir in every successive training it gives different result without changing any parameters why ? how can i control ?
You are not facing the problem properly.
First of all, you must analyze if your data are suitably modeled with a neural network or any other classification method.
Second: You must have training, validation and test set of data with statistically significant amount of data. It seems to me that in this case, your test set has only two points????
A correlation of the order of 0.9 in the test set is in general a good performance, but what we expect from a reliable model is that correlation of the training set is greater than that from validation set, and this greater than the test set.
Dear patricio,
yes my data set is limited I have only 40 Tall building data. It is not so large amount for ANN but i can not get data more than this number . can i success from ANN , I want to do K- fold cross validation . my methodology can able to get result please suggest me .
Lila:
It will be interesting to know to what problem your data belong. What are the input and output variables? Indeed it will be convenient to have more data...
Yes sir,
I have 40 Tall building input data from architectural drawing of Tall building.and also design result from analysis. my data are not from any other experiment and measurement. Limited data is my problem but if performance from my network is as shown in below fig please suggest me it is good result for my available data set??
click on the performance rather. It is on the other picture in the ones I uploaded earlier. It is actually the MSE or SSE depending on what you choose. In the one I sent you earlier, the performance was 0.00098115
hi, depend what do you use your NN, for example:
- Classification: confusion matrix.
- Inference: RMSE
- Prediction: MAPE or SMAPE
this metrics are commonly used with test set.
Hello. A few preliminary questions:
- Can you share with us a short and sweet description of the problem you are trying to solve with the network?
- What are the characteristics of your problem that justify to using two hidden layers?
- If there in the Matlab package a feature to pull a ROC? If so, could you post the result?
the probable reason why the solution varies every time is that your network initialization function doesn't work.
Dear Nicolus
In my program i initialize the initial weight using following command.
net=feedforwardnet([HN1 HN2]);
net.inputs{1}.size=20;
net.layers{1}.transferfcn='logsig';
net.layers{2}.transferfcn='logsig';
net.layers{3}.transferfcn='purelin';
net=configure(net,ptr,ttr);
y1=net(ptr);
% plot(ptr,ttr,'o',ptr,y1,'x');
%%%%Initialize weight %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
net=init(net);
it is region i can not find .
Initialize both the net and the weights and biases. Try this few lines. I have tried before.
net.initFcn='initlay';
net.layers{1}.initFcn='initnw';
net.layers{2}.initFcn='initnw';
net.layers{3}.initFcn='initnw';
net=init(net);% initialize the net (weights and biases initialized)
Also check my thesis:
https://www.doria.fi/bitstream/handle/10024/98414/Rotich%20%28M.%20Sc%20Thesis%29.pdf?sequence=2
If it helps, remember to cite it and of course vote all our comments up on this site.
Best regards,
NR
I have used ANN for Fault-Detection and Isolation (FDIA) and normally checking the performance is by comparing the ANN output to the actual data set other than the set used in training the ANN.Your problem is in the same domain.
Additionally, the ANN normally is part of a closed loop, so if the inclusion of the ANN in the loop induce unreasonable delay, that would be an indication to re-design to come up with more robust ANN structure.
- Badges
- Science topic