• OpenAI Gym. OpenAI Gym is a popular toolkit for developing and comparing reinforcement learning models.
• DeepMind Lab. DeepMind Lab is a 3D learning environment based on the Quake III first-person shooter video game, offering up navigation and puzzle-solving tasks for learning agents.
• Psychlab. Psychlab extends DeepMind Lab to support cognitive psychology experiments like searching an array of items for a specific target or detecting changes in an array of items. Human and AI agent performance on these tasks can then be compared.
• House 3D.  The primary task covered in the paper that introduced House3D was “concept-driven navigation,” e.g. training an agent to navigate to a room in a house given only a high-level descriptor like “dining room.”
• Unity Machine Learning Agents. is an open-source Unity plugin that enables games and simulations running on the platform to serve as environments for training intelligent agents.
• Ray. Developed by Ion Stoica and his team at the Berkeley RISELab, Ray is a framework for efficiently running Python code on clusters and large multi-core machines, specifically targeted at providing a low-latency distributed execution framework for reinforcement learning.

I agree with Yosra, both tools are something new which can be help in the development of such areas

Key Papers in Deep RL: https://spinningup.openai.com/en/latest/spinningup/keypapers.html#exploration