Currently iam implementing back propagation algorithm,I stuck in updating weight vector after each iteration.
I followed the following approach to implement in Octave
Steps
1) Calculated Cost and gradient value i.e J and Capital delta for all data set
2)Calculated error as (Target_Value) - (Hypothesis value in 1st iteration)
2) for weight vector update i used conditional statements like
if(error>0)
theta1=theta1+grad1;
theta2=theta2+grad2;
% Unroll parameters
nn_params = [theta1(:) ; theta2(:)];
[J,grad] = nnCostFunction_NAL(nn_params,input_units,num_labels,hidden_layer_size,X,y);
grad1 = reshape(grad(1:24),4,6);
grad2 = reshape(grad(25:39),3,5);
elseif(error < zeroMat)
theta1=theta1-grad1;
theta2=theta2-grad2;
% Unroll parameters
nn_params = [theta1(:) ; theta2(:)];
[J,grad] = nnCostFunction_NAL(nn_params,input_units,num_labels,hidden_layer_size,X,y);
grad1 = reshape(grad(1:24),4,6);
grad2 = reshape(grad(25:39),3,5);
else
break
Initially i tried with 5 iteration to see weather cost function will converge?
but iam not getting update of weight vector each time the error term should change but it is not happening
Can any one let me know the bug or anything other than this approach
- Badges
- Science topic
- Similar topics
- Vectorization