Dear Firas

I would like to advise you on these three papers that can give you a wide perspective about the use and application of Deep Learning (DL) and Deep Reinforcement Learning (DRL) in dynamic systems and control.

DL and DRL learning are very related in the area of control and dynamic systems, given DRL has been used by some researchers as an adaptive control new perspective. Additionally, DL is part of the motor of DRL

[1]- This first paper is going to show a DL and Reinforcement Learning (RL) historical review, finally you will find an application perspective of these two topics to control and dynamic systems (General introduction)

[2]- This is going to model the control problem as a Markov Decision Process and it is going to face the problem focusing on Dynamic Programming (DP) and Artificial Neural Networks(ANN). This is going to be a very good starting point for your question answer.

[3]- It is a very good review from Benjamin Recht. It is going to give you a control systems review from the LQR perspective passing through DP, RL, and finally DRL. At this point, you are going to have clear the relation at DL and DRL and how these ideas are applied to dynamic systems and Control.

[1] Lee, Jay H., Joohyun Shin, and Matthew J. Realff. "Machine learning: Overview of the recent progresses and implications for the process systems engineering field." Computers & Chemical Engineering 114 (2018): 111-121.

[2] Lee, Jay H., and Jong Min Lee. "Approximate dynamic programming based approach to process control and scheduling." Computers & chemical engineering 30.10-12 (2006): 1603-1618.

[3] Recht, Benjamin. "A tour of reinforcement learning: The view from continuous control." Annual Review of Control, Robotics, and Autonomous Systems 2 (2019): 253-279.

In slow process control like temperature control, deep learning tchniques can be implemented. If a fast response is expected from a controller, then still deep learning techniques are impractical. However, shallow neural networks, fuzzy logic, neuro-fuzzy system-based model-free intelligent controllers can be used where fast response is required. Hope, it helps Firas Abdulrazzaq Raheem

Dear Firas,

I'm agree with colleague who preceded me and i add that the contribution of Deep Learning can also be found in Mobile Robotics, network traffic control systems...

For more details and information about this subject, i suggest you to see links and attached file in topic.

Deep-learning in Mobile Robotics - from Perception to Control Systems

https://www.researchgate.net/.../311805526_Deep-learning_in_Mo...

Deep Learning in Mobile and Wireless Networking: A Survey - arXiv

https://arxiv.org/pdf/1803.04311

State-of-the-art in artificial neural network applications: A survey - NCBI

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6260436/

Advancing Systems and Control Research in the Era of ML and AI

https://faculty.sites.uci.edu/.../files/.../Control_ML_AI_Final.pdf

Applications of Artificial Intelligence in Transport: An Overview - MDPI

https://www.mdpi.com/2071-1050/11/1/189/pdf

Notes on Artificial Intelligence (AI), Machine Learning (ML) and Deep ...

https://towardsdatascience.com/notes-on-artificial-intelligence-ai-m...

Applications of Reinforcement Learning in Real World

https://towardsdatascience.com/applications-of-reinforcement-learn...

Best regards

Hi Firas,

Just adding some nice recent works on deep learning based controller:

1. Preprint Online Deep Learning for Improved Trajectory Tracking of Unm...

2. Article A Novel Approach to Feedback Control with Deep Reinforcement Learning

3. https://pdfs.semanticscholar.org/4ca6/6a098611dcfd367f8ce86b193e37e85db119.pdf

Thank you.. Abdul Hady

I need to design controller using deep learning network

any one have any iadea about this subject or work on it please?

In my opinion, there are two basic directions.

One usage is the model approximation. One could use machine learning or deep learning to approximate the nonlinear residual low/high - order model, which is hard to obtain with a parametric method.

Then, this approximated model could be used in the controller, such as, MPC, feedback linearization, back-stepping and feedforward, etc.

The more accurate the model, the better the effect.

However, computation complexity is the main problem exists in MPC.

Ref could be found in "Gong, et al. (2020) Payload-agnostic Decoupling and Hybrid Vibration Isolation Control for a Maglev Platform with Redundant Actuation. Arxiv" and " Kabzan J, Hewing L, Liniger A, et al. (2019) Learning-Based Model Predictive Control for Autonomous Racing. IEEE Robotics and Automation Letters 4(4): 3363–3370. "

Both us Machine Learning to approximate the system model and improve system performance.

The other usage is optimization.

Most control problems could be interpreted as optimization problems.

Given the system's current states, the desired goal and the corresponding cost function, how to manipulate your system to achieve the goal while minimizing the cost, i.e., how to set/obtain your control policy?

There are a lot of tools you can use, such as iLQR, Q-learning, SARSA, DNN etc.

By the way, given the system's current states and input, you need to predict the system's states in the next step, i.e., you need to know to the system dynamics, which also could be addressed with the first usage (model approximation).

Thus, maybe you could see two deep learning scheme in one system, one is used to approximate system dynamics, the other is used to establish the control policy.

Ref could be found in " Abbeel, P., Coates, A., Quigley, M., & Ng, A. Y. (2007). An application of reinforcement learning to aerobatic helicopter flight. In Advances in neural information processing systems (pp. 1-8). " and " Tassa, Y., Mansard, N., & Todorov, E. (2014, May). Control-limited differential dynamic programming. In 2014 IEEE International Conference on Robotics and Automation (ICRA) (pp. 1168-1175). IEEE. "

Dear Firas

I would like to advise you on these three papers that can give you a wide perspective about the use and application of Deep Learning (DL) and Deep Reinforcement Learning (DRL) in dynamic systems and control.

DL and DRL learning are very related in the area of control and dynamic systems, given DRL has been used by some researchers as an adaptive control new perspective. Additionally, DL is part of the motor of DRL

[1]- This first paper is going to show a DL and Reinforcement Learning (RL) historical review, finally you will find an application perspective of these two topics to control and dynamic systems (General introduction)

[2]- This is going to model the control problem as a Markov Decision Process and it is going to face the problem focusing on Dynamic Programming (DP) and Artificial Neural Networks(ANN). This is going to be a very good starting point for your question answer.

[3]- It is a very good review from Benjamin Recht. It is going to give you a control systems review from the LQR perspective passing through DP, RL, and finally DRL. At this point, you are going to have clear the relation at DL and DRL and how these ideas are applied to dynamic systems and Control.

[1] Lee, Jay H., Joohyun Shin, and Matthew J. Realff. "Machine learning: Overview of the recent progresses and implications for the process systems engineering field." Computers & Chemical Engineering 114 (2018): 111-121.

[2] Lee, Jay H., and Jong Min Lee. "Approximate dynamic programming based approach to process control and scheduling." Computers & chemical engineering 30.10-12 (2006): 1603-1618.

[3] Recht, Benjamin. "A tour of reinforcement learning: The view from continuous control." Annual Review of Control, Robotics, and Autonomous Systems 2 (2019): 253-279.

Hi,

Please take a look at recent works on deep learning based controller.

https://www.springer.com/gp/book/9783030052751

Best regards