hello,
I'm new to deep learning, and i'm currently trying to write a program to play a game of bubble shooter. I thought a good approach would be using Q-deep-learning. I'd be happy if someone could advice me about - how to choose the states in this case? and how many neural layers is a good starting point? any reference to forums that discuss those topics?
Thank you!
Use the ATARI games as your orientation.
States could be the pixels. The combination of the last 4 screen outputs is better.
Actions and rewards are clear to you?
Overall, I don't find this application optimal for getting started. First try to solve a smaller physical problem (with a smaller state space).
There is a great package to get into reinforcement learning by OpenAI: https://gym.openai.com/
I'm sure there are many tutorials to start with!
Roman Liessner first of all, thank you for your advice. i'd like to hear what you think about actions and rewards. I thought about using epsilon greedy algorithm to choose between some random action or to choose an action based on prediction. I thought to use a reward when hitting the right color (+7) and when hitting wrong color (-5) and when loosing (-10) or something around those values.
do you have any suggestions about the architecture? I thought about using regular sequential model with dense layers (2 hidden). from what I've read, there are so many options, and they all say it's trial and error, so I'm trying to consult as many people as I can in order to minimize the set of models and type of layers and overall architecture I use. I hope it's ok that I'm asking so many questions, I'm just trying to understand on what should I focus for this task. Thank you again for your help!
Philipp Nolte thank you very much! i'll check it out right now.
Concerning the architecture, I recommend that you first take a look at the solutions for the ATARI games.
The RL Actions are probably the input the user uses. The Reward could be calculated from the displayed score.
Hello Elinor Ginzburg,
Like you said in your previous reply to Mr. Roman Liessner, it is a trial and error process most of the times. The best way to start would be have the least number of hidden layers (maybe 1 should be a good starting point). The problem with higher number of layers is they are difficult to interpret and consume a lot of training time (with my experience).
Like answered by Mr. Liessner, the states could be pixels. The reward definition process is a tedious process. You can only talk about this when you have some basic training done and observe how the agent is behaving. If winning is the ultimate goal, then I would advise setting a penalty of -100 or something when it loses (just to tell the agent what is important). Then you can observe the behaviour and proceed further.
But be careful not to give a reward only at the end. Then the learning process becomes very complicated.
Have a look at the topic Reward shaping.
Guru Bhargava Khandavalli and Roman Liessner , thank you very much for your help, i'll continue reading and learning.
Dear Elenor,
I have a further comment on the State Space. As explained before, the pixels are usually used today and this works in numerous examples.
A classical approach would be to derive features from the pixels themselves. For example you could count how many bubbles are still present, how many with a corresponding color or how many in the left part of the screen and how many on the right etc..
With a good choice of features you simplify the Deep RL learning problem. However, another algorithm is needed that calculates your state representation from the pixels.
All the best for you.
Roman Liessner Thank you once again, it's a great advice and i'll try using it.
- Badges
- Science topic