Hey Bashar,

Does your Q-function (model/network) take as input a single action/state pair or a sequence of actions/states? If you are working with input signals that are "sequential" (indexed by t, t+1, t+n), then maybe an LSTM (or Bi-LSTM) is better for taking the function of a Q-Table.

Sorry for not being able to help more, but that is all I can do with so little info. What kind of task are you trying to use Deep RL to solve?

Yes, adding and introducing new ways are more benificial and welcomed by data science and its community. However., the method must logical within the domain of data analysis good outcomes via adding some more power.

Let's take an example of the Bricks Breaker game. If you look at a single timestep snapshot of the game, you might not be able to describe if the circle was going up or down. But if you have information about, let's say, two to five previous timesteps, surely you can point which is its direction. LSTM carries past information, so if adding information about the past enriches the agent's observation, you should consider using it.