The selection of input variables is critical in order to find the optimal function in ANNs. Studies have been pointing numerous algorithms for input variable selection (IVS). They are generally classified in three different groups: (1) dimension reduction, (2) variable selection, and (3) filter. Each group holds several algorithms with specific assumptions and limitations.
If a researcher decides to use ANN, he might be happy to know...
1) Which approach is the most recommended to select ANN input variables?
2) What are the advantages and drawbacks of your choice in regard to other strategies?
3) Is the algorithm implemented in any statistical package (R or other free ones are more approachable)?
In order to increase the predictive efficiency of ANN, one can use all the three above mentioned approaches.
1. Dimension reduction or Normalization or Standardization is carried out to ensure that the variables receive equal attention (Maier and Dandy, 2000).
2. Variable selection: In my experience one can opt for Auto- Partial auto- cross- correlation function to determine the Input variables and their lag time which will certainly affects the models output. (Sudheer et al. (2002). Other than correlation function, Principle component analysis (PCA) can also be a good option for variable selection and can be carried out in SPSS, Matlab and many other. But I will prefer SPSS because of its user friendly GUI.
3. Filter: Correlation function and PCA will also help you to filter the parameters or the input variables and will minimize the uncertainty of the model prediction.
Conclusion:
In my personal experience in ANN models, I would certainly suggest employing all the methods to ensure minimum model uncertainty.
Maier, H. R., and Dandy, G. C. (2000). “Neural networks for the prediction and forecasting of water resources variables: A review of modelling issues and applications.” Environmental Modelling & Software, 15(1), 101–124.
Sudheer, K. P., Gosain, A. K., & Ramasastri, K. S. (2002). A data‐driven algorithm for constructing artificial neural network rainfall‐runoff models. Hydrological Processes, 16(6), 1325-1330.
Kumar, D., Pandey, A., Sharma, N., & Flügel, W. A. (2014). Modeling Suspended Sediment Using Artificial Neural Networks and TRMM-3B42 Version 7 Rainfall Dataset. Journal of Hydrologic Engineering.
Thank you for everyone's contribution and discussions. My question was actually inspired in the paper shared by Mahboobeh above, which in my opinion is a good overview but too broad to come up with a conclusion. But I appreciate that she shared this here.
I ended up using random forest to measure the importance as a relative increment in the mean squared error when the variable is removed from the model. Although, the mathematical basis of this modelling approach is different from the ANNs, I found it offers insights into the importance of the explanatory variables, allowing us to rank them and make decisions regarding which variables we should include in our models.
I used a correlation matrix as well to minimize the redundancy and established threshold values for maximum correlation and minimum importance. That can be combined to any sort of model as an exploratory analysis made beforehand.
RE: PCA: it is great for dimension reduction. But if you have a bunch of poor inputs, you will still have poor inputs after PCA. I believe it is also possible that you may have a higher frequency good input that would not show up in the lower frequency PCA terms.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling