Hi
Is the non-normally distributed data appropriate in MLP-based regression?
I know that normal distrubution of variables is one of the assumption of the linear multivariate regression (LMR) modelling. But what about sigmoidal multilayer perceptron?
I have 5 inputs and 1 output. Only one input and output are normal, while the rest is skewed (some even highly).
Let's say that MLP based on this data gives good results (R2, RMSE, MAE, etc.), far better than LMR. Also the MLP's residuals are not autocorrelated, normally distributed and homoscedastic. So, is that proper to tell that MLP can cope with highly skewed inputs?
Can the better performance of MLP be explained only by the non-linear model structure? Whether it is also a matter of ability of operation on the skewed data?
Thanks for answer.
Hi,
You could try this ms: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4106049/
but if you'D like to compare MLP and LMR first normalised your data as usual to remove the skewness
What matters in MLP is the range of inputs to the non-linear transformation function. Common choice of sigmoidal function is actually non-linear in a limited range of inputs. With inputs larger that +5 of smaller than -5, they will behave like a threshold function giving values very close to +1 or 0 (which is bad because backpropagarion cannot train these weights). With inputs between -0.1 and 0.1, they will behave almost like a linear function.
For an MLP to have a non-linear behavior, you must make sure that some portion of projected inputs have absolute value between 0.1 and 5. Or just look at the hidden layer outputs - if you have the majority of +1 and 0, there is a problem with normalization.
>I know that normal distrubution of variables is one of the assumption of the linear multivariate regression (LMR) modelling.
Regression and other linear models estimated via ordinary least squares do not assume that the predictor and/or response variables are normally distributed. They assume only that the error term is normally distributed, and are quite robust to even this assumption. The attached paper discusses this common misconception.
Article Assumptions of Multiple Regression: Correcting Two Misconceptions
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling