Stochastic Processes I

Course content

Markov chains: Transient and steady state behavior, First Passage times. Markov process: Transient and steady state behavior, First Passage times and phase type distributions. Queuing networks. Markov decision process (MDP): Continuous and discreet time models, MDP as linear programs, MDP under constraints, Maintenance applications. Reliability and maintenance problems: Definition, System configurations, Reliability improvements with cost considerations, Reliability bounds.