Dear experts,
In this situation:
we have 60 numerical features (60 neurons in input layer).
The variation of samples are something like:
in 10 inputs: samples vary in the range of: [0, 1E-4]
in 20 inputs: samples vary in the range of: [3, 10]
in 30 inputs: samples vary in range of: [20000, 1e6]
1. How can we prepare such data set to train a Neural Network?
2. What cost function do you suggest for measuring errors and training the Neural Network?
You can start with standardizing the feature set which will convert your data to a zero mean and 1 standard deviation.
Would you please explain more? because normalization doesn't work in such data. for example if we consider 1E6 as 1, then some of our data becomes very small like 1E-10. also Logarithm seems not good.
Thank you Eduardo but it doesn't work in such case. let me clarify with some real data:
Consider:
min(x)=-1e-4
max(x)=1e+4
Rescaling: x' = (x- 1e-4)/(1e4-1e-4)
examples:
x1=0 ----> x1'=1e-8
x2=1000 ----> x2'=0.1
This rescaling only compress the data into the range [0,1] but still the difference between the data order exists. consider a single layer neural network in which the inputs and the outputs are required to be the same. if we consider MSE (Mean Squared Error) as loss function for training such NN, the NN would estimate x1 and x2 with a very different accuracy because the errors in estimating very small values such x2, plays no role in our loss function. after training the NN, you see that:
X1: output of NN for x1 as input
Consider that at end our NN has 0.001 error in estimating values and x1=1e-6:
X1-x1=0.001 ------> X1= 1e-6+0.001 it means that the NN is estimating x1 with a value of X1 which is about 1000 times greater than x1!
What is the solution in this situation? shall be solve the problem in normalization phase or in calculating loss function (or both)?
- Badges
- Science topic