The Existence of Optimal Control for Continuous-Time Markov Decision Processes in Random Environments

In this work, we investigate the optimal control problem for continuous-time Markov decision processes with the random impact of the environment. We provide conditions to show the existence of optimal controls under finite-horizon criteria. These results are established by introducing some restriction on the regularity of the optimal controls and by developing a new compactification method for continuous-time Markov decision processes, which is originally used to solve the optimal control problem for diffusion processes.