The outline below represents a tentative roadmap for the course. We may deviate from it depending on
interest and time.
Week 1-2. Course description and review of probability theory
Course description / probability space / random variables / expected value / independence / important
discrete and continuous distributions / conditional expectation / limit theorems
Week 3. Introduction to stochastic processes
Definition of stochastic processes / stationarity / ergodicity / examples and applications
Week 4-6. Poisson process
Definition of Poisson processes / interarrival and waiting time distributions / conditional distribution of arrival
times / nonhomogeneous Poisson process / compound Poisson random variables and processes / the
M/G/1 queue and business applications
Week 7-9. Renewal theory
Introduction and preliminaries / Wald’s equation and limit theorems / alternating renewal processes / delayed
renewal processes / renewal reward processes / queueing applications in business / regenerative processes
/ stationary point processes
Week 10-12. Discrete-time Markov chains
Introduction and examples / Chapman-Kolmogorov equations and classification of states / limit theorems /
applications of Markov chains / time-reversible Markov chains / applications in business
Week 13-14. Continuous-time Markov chains
Introduction to continuous-time Markov chains / birth and death processes / Kolmogorov differential
equations / limiting probabilities / time reversibility / applications to queueing theory
Week 15. Inventory models
Newsvendor problem / EOQ model / multi-period (s, S) model
Week 16. Markov decision processes
Introduction to MDP / policy iteration / value iteration / applications in business
Stochastic Processes, 2nd edition, by Sheldon M. Ross
教辅: Essentials of Stochastic Processes, by Rick Durrett
Foundations of Stochastic Inventory Theory, by Evan L. Porteus
Optimization of Business Processes: An Introduction to Applied Stochastic Modeling, by Ger Koole