I am currently implementing the following two RL algorithms for 5G
(i) Power control to maximize throughput in a multi-gNB multi-UE scenario, and
(ii) Maximizing throughput subject to delay constraints in a single-gNB multi-UE scenario
I have made significant progress using an external Python program connected to NetSim's gNB scheduler. This setup facilitates the exchange of states and rewards between the scheduler and the Python program.
I am interested in exploring scenarios with time delays in state-reward exchanges, such as delayed state reception. How does RL adapt in such situations? I would appreciate any relevant research papers on this topic
The delay can either be:
a) A constant delay - Constant Delay MDP (CDMDP). The delay is deterministic and known.
b) A random delay - Random Delay MDP. The delay is stochastic, and the distribution of the random variable may be known.
What is your case? There are different methods to handle each.
Also, what is the RL algorithm you are using? It might be easier if you start with a tabular algorithm.
Would your python program operate like a RAN Intelligent controller (RIC)? What is the timing loop? Is it ~ 1sec or ~ 10milli sec?
Amit Choksi
Yes, this is similar to a near-real time RIC. The control loop would be 10msec or could even be at TTI time scale.
Sai R N
I can start with a constant delay MDP. As regards the RL algorithm, I can start with tabular Q-Learning if that is easier
- Badges
- Science topic
- Similar topics
- Geoscience
- Asia
- India