I want to improve the performance of the agent. I think the critical point of improving the agent is to normalize the observation and action. However, I don't know how to normalize them when the range of them are infinite. If anyone knows how to solve the problem or any tricks to improve the agents, you are very welcome to release any idea! Thank you so much!
They can’t be infinite, because that’s ambiguous. So they should be mapped to finite values.
If you receive an infinite observation, I would guess that something went wrong within the learning process or on the simulation model part (environment), e.g., a bug or the learning process diverges. In this respect, I would recommend to go troubleshooting here and first investigate the reason why you get an inf return value.
Where does +-inf come from? Maybe the creator of the data source misused the IEEE representation to express a special situation. So you could double or triple your input neurons, set the original value to 0 and indicate that the value was + or minus infinity by setting the additional input neurons to 1.
Regards,
Joachim
The values of observation come from the environment. Please check your environment model or limit such value at the observation input of agent.
please check your network input values, whryer they are normalized or not. Normalizing input values can sometimes help to limit the output response. Actions cant be inf.
This is my fault. I asked a wrong question. I mean "if the range of observations and actions are ±inf, how can I normalize them?", rather than the values of observations and actions are ±inf. @Syed Muhammad Talha Zaidi @Frank Cheng @Joachim Pimiskern @Oliver Wallscheid @Stam Nicolis
As an engineer I would doubt that there are no technical/natural limits regarding your observations/actions in your application. Any real-world system has its natural boundaries which can be used as a baseline for normalization. Maybe you can share some more details on your considered application for a mutual discussion on possible state/action limitations.
In any reinforcement learning environment, you most likely know the number of actions that the agent can take. It cannot be infinite. First of all, understand if the agent will choose each action(s) among a Discrete or Continuos set of value (check the gym and spaces library, Box, Discrete and MultiDiscrete action spaces).
On the other hand, I can imagine normalizing observations can be a little trickier.
One thing you could try to do is to normalize the observation based on the maximum observed value for each feature. That's what worked for me.
https://towardsdatascience.com/ultimate-guide-for-ai-game-creation-part-2-training-e252108dfbd1
- Badges
- Science topic