Can a trained neural network replace optimization algorithms?
I am curious if that is possible maybe we can train neural networks once and use it instead of solving an optimization problem in every operation window of a system.
Edit
I am dealing with a resource allocation problem that I solve using an evolutionary optimization approach (every 5 minutes). It takes quite some time to reach a solution. I am sure there may better algorithms to reduce this time and complexity.
But, here I am curious if we train a neural network somehow to replace my resource allocation optimization algorithm? The purpose is to reduce complexity.
If it is possible, can you please refer to some literature or some specific technique.
Dear Dr. Saifullah Khalid and the other RG colleagues,
In my last published research, I have pointed out to a similar issue (ANN & Sea Lion as an Optimization Algorithm. For more information, please refer to section 3.4 of the following paper:
Nidhal Kamel Taha El-Omari, “Sea Lion Optimization Algorithm for Solving the Maximum Flow Problem”, International Journal of Computer Science and Network Security (IJCSNS), e-ISSN: 1738-7906, DOI: 10.22937/IJCSNS.2020.20.08.5, 20(8):30-68, 2020.
In the structure of an artificial neural network, there are cost functions. the aim of the learning process is the optimization of these functions learning or optimization algorithms. and it is this optimization that makes the strength of the network. Thus, neural network contains an optimization algorithm among other components including the connection weight, ......
When you ask if you can replace an optimization algorithm with a neural network? I think it depends on the type of data being processed, the type of neural network, and the configuration of the model in which the optimization algorithm is implemented. for example, i guess it can be possible if the neural network is a MultiLayer Perceptron.
Dear Saifullah Khalid ,
Look the link, maybe useful.
Preprint Optimization Algorithm Inspired Deep Neural Network Structure Design
Regards,
Shafagat
The Neural Networks performs the same tasks as Optimization Algorithms do. In other words, the aim of data analysis is nothing but optimization. Optimization is nothing but to obtain maximum accurate yields/productivity/outputs/results/objectives/answer... with minimum efforts & inputs.
Dear Saifullah
Normally optimization and neural network are two different concepts. In my view if you will optimize the parameters of the network, the performance of the network model become high. But if you want to use the network model with out optimization technique, then the better choice is SVM with regression analysis.
Dear Oraléou Sangué Djandja Shafagat Mahmudova Muhammad Ali Radha MOHAN Pattanayak
Thank you all for your insightful responses. Please let me clarify my question once again.
I am dealing with a resource allocation problem that I solve using an evolutionary optimization approach (every 5 minutes). It takes quite some time to reach a solution. I am sure there may better algorithms to reduce this time and complexity.
But, here I am curious if we train a neural network somehow to replace my resource allocation optimization algorithm? The purpose is to reduce complexity.
If it is possible, can you please refer to some literature or some specific technique.
An optimization algorithm is considered as a machine learning or artifial intelligent algorithm that take data as input make some processing and get the result
The neural networks make this process faster by using multiple layers after taking the input, they still add additional data to the previous ones utill get the best result
So yes we can replace the optimization algorithm by using the neural networks to get more better results with less of time than the optimization algorithm takes
Reema Ahmad Thanks, but how we go about it. is it just like regarding the output of the optimization algorithm as the target labels and training the neural network or some other more systematic approach also exist?
@Saifullah Khalid
You are welcome
I give this answer based on a comparison between the traditional optimization algorithms for example Support vector machine and the neural network for example Recurrent neural network
The traditional one go throught several steps like get the input data then make both feature extraction and selection then the classification step to get the desired output
While in neural network will move through input then its process containing three layers (i.e. input , hidden layers for processing, output) instead of the three steps that mentioned before to get the desired output
This will reduce the time and get an accurate results based on how many hidden layers the neural network has compared with the traditional optimization algorithm
better you should use hybrid of nature inspired optimization algorithms and ANN models. In this way you can find optimal parameters of ANN models.
@Rana thank you for the response but can you please give little more details or some literature link for such a case.
dear you can review my papers,i published papers using ann optimize using natural algorithms
I suggest you if use or replace the architecture of NN by CNNs it's very hard Optimization, that wa can change the architecture by adding the Dropout Functions, it' used for regulation for 2D with MaxPooling Functions for extraction of maps of feature, that give you the better results of accuray and lower Loss if implemented in GPU that is very better in time.
I hope that be Claire for you.@ Saifullah Khalid
visit the following link
https://towardsdatascience.com/optimizers-for-training-neural-network-59450d71caf6
Dear Dr. Saifullah Khalid and the other RG colleagues,
In my last published research, I have pointed out to a similar issue (ANN & Sea Lion as an Optimization Algorithm. For more information, please refer to section 3.4 of the following paper:
Nidhal Kamel Taha El-Omari, “Sea Lion Optimization Algorithm for Solving the Maximum Flow Problem”, International Journal of Computer Science and Network Security (IJCSNS), e-ISSN: 1738-7906, DOI: 10.22937/IJCSNS.2020.20.08.5, 20(8):30-68, 2020.
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training