Nonstationary Markov Decision Processes with Risk Probability Criteria

This paper considers a risk probability minimization problem for nonstationary discrete-time Markov decision processes, in which the transition probabilities and the reward functions depend on time. Different from the expected reward/cost criteria in the existing literature, the optimality performance here is to minimize the probability that the total rewards do not reach a given profit goal until the first passage time to some target set. Under mild reasonable conditions, we establish the corresponding optimality equations, verify that the sequence of the optimal risk functions is the unique solution to the optimality equations, and prove the existence of an optimal Markov policy.