Dear Saeed, I have familiarity with all you said, just give me some time to make a small research. Just to be sure.

AI- Artificial Intelligence; ANN- artificial Neural Networks; SVM - support vector machine; GA- genetic algorithms; PSO- particle swarm optimization. and Finally, since I am a Java programmer, and also Matlab,  

When you say "objective functions", you are not talking about Object Oriented programming, rather, the function you optimize, right? If you are using Matlab toolbox, they already have those routines implemented, if I am not mistaken, if you please to make your own, it is a long road, but not impossible. If all that I said make sense to you, I will try to take a minute to make a short research and come back to you later. Best, Jorge Pires.

Thanks Jorge.

Yea, I mean the fitness function that should be fed into GA toolbox to be used for optimization. Actually, the problem is that once I am building the ANN model through its toolbox in Matlab, I need to call this model for GA as its objective function but unfortunately, in GA toolbox, the objective function place is just to write it and I do not know how I can feed the built ANN in that field of GA toolbox..

The last time  I have used the Matlab toolbox, you could choose the optimizer, e.g. GAs. Are using the interface? You can find a lot on Matlab Tutorial (http://www.mathworks.it/help/pdf_doc/nnet/nnet_ug.pdf), try and let me know if you are not familiar already. I did not have time yet, but I will double check for you asap, and come back to you. Best, 

http://www.mathworks.it/help/pdf_doc/nnet/nnet_ug.pdf

Saeed,

Unfortunately to answer your question, it is going to take more time then I have thought, I just cannot remember how to do it, I know it is possible. For now, I want to suggest you the book MÜLLER, B. ; REINHARDT, J. ; STRICKLAND, M. T. Neural Networks: an Introduction. The physics of Neural Networks. Spring. Second edition updated and corrected. 1995.

and the professor: https://www.researchgate.net/profile/Agnaldo_Reis.

Both of them can be useful for you. If I succeed before you figure it our on your own or somebody else answer the question, I will be glad to come back. Best,

Saeed,

Unfortunately I do not have yet your answer. Let's hope meanwhile someone will show up to rescue us. So here goes the scores.

You know how to use the Matlab toolbox (i.e. nntool), you know how to use the global optimization toolbox of matlab (i,e, http://in.mathworks.com/help/pdf_doc/gads/gads_tb.pdf), we are talking about supervised learning, thus you have have one set of values for input and desired output. You have designed a neural network (e.g. Support Vector Machine and Artificial Neural Network), and you wish to optimize then, learning from a experimental data, using computational intelligence (e.g. Swarm Particle Optimization, Genetic Algorithms). You have a neural network, but the optimizer just accept written commands.  I hope you follow me. 

You have two ways to solve the problem, you either use the matlab optimizer, which is what you want to do, or make your own code, which what I would do (**). If you decide for the former, you must master the toolbox for global optimization, if not, learn it. See that matlab treats every function as object, Java style, OOP. Maybe it is possible to pass the neural network as a common function, a black box model (*); I know you can access the weights like in java, see the notation in the Tutorial I have sent in the previous post. You just need to give discrete values for the neural network objective function, I assume you are using LMS; in case you apply the toolbox directly, they are more well-elaborated models. Of course it does not seem efficient, it is better to make your own optimizer. 

In summary, in spite of the fact I see the problem is solvable, I have not done that before. The last I have used a neural network, I have designed my own. I will keep thinking and try to come up with a plausible answer. 

All the best,

Jorge Pires,

(**) see https://www.researchgate.net/publication/280877647_PharmacokineticPharmacodynamic_Modeling_Evolutionary_Algorithms_and_Optimal_Control_Theory_a_numerical-based_approach?ev=prf_pub

(*) See that I have problem with this term, see my paper https://www.researchgate.net/publication/280877055_On_the_Design_and_Development_of_a_Quantitative-_Qualitative_Methodology_for_Modeling_Physical-_Biological_Systems_Partial_and_Blind_Models?ev=prf_pub

Research Pharmacokinetic/Pharmacodynamic Modeling, Evolutionary Algor...

Research On the Design and Development of a Quantitative- Qualitative...

Thanks for your efforts and answers Jorge. I know how to use ANN toolbox and how to apply optimization toolbox but the problem is how we can link the ANN model developed by toolbox to use in optimization toolbox; for GA, Simulated Annealing and ...

I got it. I unfortunately I have never done that before, besides I believe is not impossible. If life were like computer program, mine would be ease. Unfortunately I cannot promise you anything, but I will keep thinking and I suggest you do the same. I have tried to invite a professor to join the discussion, but he is busy with a paper. This question would be easier if were working with this now, which is not the case, now I am working with biomathematics, differential equations and so on. Best

Fortunately I found the answer:

After training the ANN and building the model through ANN toolbox of Matlab, in the output generation dialogue box, we can generate ANN codes by clicking on Matlab Code Generation. 

Then, the generated Mat file should be saved in a file called ANN Fitness function. In this stage, through Global Optimization Toolbox, GA can be selected. Once GA is selected, it needs to call the objective function in the relevant tab. Now, it is the time to call the ANN model as the fitness function for GA through the following syntax:

@ANN Fitness function

* Just considering that for GA, the datasets should be in horizontal vectors.

Saeed, Good to know. However, I believe there is a more effective way. It means that if you make a small change on the neural network, you need to redo the process, handicraft .

"After training the ANN..." are you training it twice? it seems senseless. I can understand the you can treat the neural networks as a function, black-box like.

"ANN Fitness function", so they already have a way, I am supposing you cannot choose another name, such as "My ANN Fitness function". 

However, nice to know, problema solved. All the best, 

Hi Saeed &Jorg 

Iam very interested in this discussion as I  have same problem .

my question  to  Saeed  if I must use this ANN code or function as constrain with other  nonlinear constrain how can  I do that . 

thanks 

After training the ANN and building the model through ANN toolbox of Matlab, in the output generation dialogue box, you need to generate ANN codes by clicking on Matlab Code Generation. 

Then, the generated Mat file should be saved in a file called e.g.; ANN Fitness function. In this stage, through Global Optimization Toolbox, any optimization algorithm can be selected. Once selected, it needs to call the objective function in the relevant tab. Now, it is the time to call the ANN model as the fitness function through the following syntax:

@ANN Fitness function

Muqdad,

I know you have asked Saeed, however let me add what I think. By using the strategy of Saeed, if I understood properly, you must either define the restrictions using the toolbox for global optimization, given this option exists, or use an external strategy, such as weights.  That is how I would solve the problem. As far as I understood, Saeed is just using the neural toolbox for building an object, the objective function. In that case, you can additionally design your own optimizer. 

Best regards, Jorge Pires,

I solved the problem by writing mathematical equation representing the neural network. The weights of the trained network can be obtained by command getx. Please refer to this article for the same.

Article Artificial Intelligence versus Statistical Modeling and Opti...

Osama, honestly, I understood you have already faced the problem, as that you wrote a paper, I guess much before this question.  You say "writing mathematical equation representing the neural network", so you have not used Matlab toolbox, if so, what that has to do with the main question? By making an approach on your own, such as writing your own program, it is hard-work and valuable, but not the question.

As I have mentioned, I know books about global optimization such as GA for neural networks, it is something already quite explored, and I still believe that Matlab has embedded global optimization into the neural network toolbox, it just a matter of finding how to use it. 

Best regards,

Jorge Pires,

 Hi every one 

 I tried to  invoke the matlab code as constraints function but unfortunately i can not got logical and close solution. although when I test this code separately i got exact result, but when i used as constraint function it was not.  as I have compared the result with simple and exact solution optimization process(which must give same results ) . does any one have more suggestions 

Make a function in MATLAB which takes in an array of input values and returns the neural network predicted values.All these require simple commands which you will easily find in any matlab doc on net.Further,this function handle(@ )is tobe used in optimization algorithm GUI in matlab.Goodluck .

After calling the ANN model as the fitness function for GA, how to determine the variables?

For example how can we make GA choose between 3 training algorithm which are trainlm, trainbr, trainscg and also tries 1-10 number of hidden neuron?

I prepared a complete tutorial about optimizing artificial neural networks using genetic algorithm with Python implementation. It is titled "Artificial Neural Networks Optimization using Genetic Algorithm with Python" It is available here:

https://www.linkedin.com/pulse/artificial-neural-networks-optimization-using-genetic-ahmed-gad

https://www.kdnuggets.com/2019/03/artificial-neural-networks-optimization-genetic-algorithm-python.html

https://towardsdatascience.com/feature-reduction-using-genetic-algorithm-with-python-403a5f4ef0c1

You can read more and more in my 2018 book that covers GA in one of its chapters. The book is cited as “Ahmed Fawzy Gad ‘Practical Computer Vision Applications Using Deep Learning with CNNs’. Dec. 2018, Apress, 978–1–4842–4167–7” which is available here at Springer https://www.springer.com/us/book/9781484241660

Also here at Amazon: https://www.amazon.com/Practical-Computer-Vision-Applications-Learning/dp/1484241665/