Here a quick but maybe not the best answer:

For example in the "classical" inventory system with (r,Q) policy and with 1 replenishment: The Inventory states are {0,1,2,...,R}

Here the system can recognize if an order was triggered: when the inventory state is less than a fixed parameter r.

For multiple replenishment orders we can add an additional parameter to the system:  Number of outstanding orders.

In  terms of the paper. we will have the new environment states

K:={(k1,k2) | k1 - number of outstanding orders, k2  number of items in the inventory}

The Matrix V can be defined in such a way that the system will try to replenish as long as k1 stays positive. After each replenishment the size of the inventory will be changed and the value k1 decremented.

Dear Professor Daduna:

          Thank you for your answer.

         Adding a new parameter k1 is usful to constitute a new environment states and  it is ceterain to has product solution for the system. However, the question lies that how to determine the replenishment rate and the number of the on  hand inventory after a replenishment.

           For a (r,Q) policy, more than one outstanding replenishment come from the decisions of the inventory managers. They often spit the order size "Q" into several order " Q1,Q2,...,Qn", where Q1+Q2+...+Qn=Q.  Even if we can assume that Q1=Q2=...=Qn=[Q/n], and the orders "Q1,Q2,...,Qn" are i.i.d of exponiential distribution, I find it is diffiult to implement the  (r,Q) policy and to write  the Matrix V.