I am familiar with the traditional ways in which AI planning systems have been designed and built, but I am less familiar with the new and fashionable Deep Learning techniques. It is not immediately clear to me that the latter can be applied to plan generation and execution at all but I suspect that they can with a bit of ingenuity.
All help gratefully received and acknowledged!
JIm
Mizutani, H., Ueno, M., Arakawa, N., & Yamakawa, H. (2018). Whole brain connectomic architecture to develop general artificial intelligence. Procedia Computer Science, 123, 308-313.
Ivanov, A., & Zhilenkov, A. (2018, January). The prospects of use of deep learning neural networks in problems of dynamic images recognition. In Young Researchers in Electrical and Electronic Engineering (EIConRus), 2018 IEEE Conference of Russian (pp. 886-889). IEEE.
Moravčík, M., Schmid, M., Burch, N., Lisý, V., Morrill, D., Bard, N., ... & Bowling, M. (2017). Deepstack: Expert-level artificial intelligence in heads-up no-limit poker. Science, 356(6337), 508-513.
Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017). Neuroscience-inspired artificial intelligence. Neuron, 95(2), 245-258.
Reinforcement Learning (RL) is a convenient tool for determining goals and executing plans, where the agent learns by interacting with the environment (taking actions) and then adjusting its behavior according to the received rewards; in this analogy "goals" can be expressed as rewards and the agent adjusts its "plans" (policies) in attempts to maximising the received rewards.
Involving Deep Learning (DL) with Reinforcement Learning (RL) results in what is called Deep Reinforcement Learning (DRL). where Neural Networks can replace different parts of the RL system; for example the Neural Networks can be used to help an agent estimate the expected reward for possible actions before taking them. Let's say the agent has 3 possible actions under a certain state; the agent will feed the state and the action to the neural networks (NNs) and the NNs in return estimate the expected reward for taking that action in that situation. The agent this way has an estimate the expected reward for each possible action and decides which action it will take accordingly.
In another configuration; NNs can be used to directly select the action that the agent should take.
Deep Learning is considered as an "approximate solution method", where the NNs try to approximate a function that represent the expected reward(in one configuration) by experiencing large amount of trials and errors until being able to provide good results and generalise (this is the difficult part). therefore I agree with you that it's "not 100% genuine" though many of recent work in this field show excellent results.
Introduction to Reinforcement Learning by by Andrew Barto and Richard S. Sutton helps in explaining above concepts and ideas much better than I poorly did
http://incompleteideas.net/book/bookdraft2017nov5.pdf
When I started learning RL, I tried to write a summary here
Research A Concise Introduction to Reinforcement Learning
I will be grateful if you think that I can work on this part with you.
Probably James means something more sophisticated than "reactive" reinforcement-based "pseudo-planning". One of the Keras' developers and a Google researcher Francois Chollet recently stated that deep learning is simply more powerful pattern recognition when compared to the previous statistical and machine learning methods. “The most important problem for A.I today is abstraction and reasoning, current supervised perception and reinforcement learning algorithms require lots of data, are terrible at planning, and are only doing straightforward pattern recognition.” I agree with him - it's usually wise not to expect much from DLs especially if you don't have enough data or simulations are impossible because they are costly/dangerous or you don't have enough feedback for reinforcement.
Mizutani, H., Ueno, M., Arakawa, N., & Yamakawa, H. (2018). Whole brain connectomic architecture to develop general artificial intelligence. Procedia Computer Science, 123, 308-313.
Ivanov, A., & Zhilenkov, A. (2018, January). The prospects of use of deep learning neural networks in problems of dynamic images recognition. In Young Researchers in Electrical and Electronic Engineering (EIConRus), 2018 IEEE Conference of Russian (pp. 886-889). IEEE.
Moravčík, M., Schmid, M., Burch, N., Lisý, V., Morrill, D., Bard, N., ... & Bowling, M. (2017). Deepstack: Expert-level artificial intelligence in heads-up no-limit poker. Science, 356(6337), 508-513.
Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017). Neuroscience-inspired artificial intelligence. Neuron, 95(2), 245-258.
Many thanks for these very helpful answers!
Perhaps I should have made it clearer that by planning I mean eg what we all do around breakfast time when we construct in our minds a schedule/plan of what we are going to do during the day to come eg give my lecture in the morning, then fly to Berlin to have lunch with Margot, then go to the races in the afternoon and gamble, then have dinner with Claude and his friends.. Then we spend the remainder of the day carrying out our plan in detail, revising it, getting stuck in a traffic jam in London, realizing we don't have enough time and/or money and so on.. (Or consider a general and his/her staff planning an attack.)
How exactly does DL fit in at the programming level?
I speculate that if you divide plan creation, execution, revision etc into their many constituent steps you can apply DL to each individual step -- but hugely inefficient!
So I tend to agree with @Rafal Rzepka (and Francois Chollet), but..
Thanks again
Jim
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Artificial Intelligence