# STOR 602: Probability and Stochastic Processes

Credits: 20

Tutors: Azadeh Khaleghi, Dave Worthington, Peter Jacko and Kevin Glazebrook.

Outline: The module will provide an introduction to probability, random variables, Markov processes, Poisson processes, Markov decision processes, and a range of key approaches for modelling in statistics and operational research. The emphasis in the module will be on the evaluation of complex stochastic properties via both analytical and computational methods. It will cover:

• Introduction to probability building from the axioms.
• Univariate random variables (discrete and continuous): standard distributions and their justifications, inter-relations, and properties.
• Multivariate random variables: marginals and copulas, their decomposition as a series of conditionals, dependence measures, and standard distributions.
• Simulation of random variables and approximation of their properties by Monte Carlo.
• Transformations of univariate and multivariate random variables.
• Limit theorems
• Poisson/Counting Processes.
• Branching processes: conditional probability and expectation, probability generating functions, links with queueing systems
• Queueing models: time dependent and steady-state behaviour. Approximation approaches.
• Stochastic Modelling and Optimisation: Markov chains, Markov reward chains, Markov Decision chains, modelling and computational methods

Objectives: Key models in statistics and operational research are based on probabilistic theory for random variables and stochastic processes. By providing this probabilistic theory and studying a wide range of stochastic models, this module provides the building blocks for modelling and understanding the properties of all stochastic behaviour studied in the MRes.

Learning Outcomes: On successful completion of this module students will be able to:

• recognise the contexts when certain random variables occur;
• simulate a range of random variables;
• derive a range of properties for random variables using analytic and simulation methods;
• transform random variables;
• demonstrate the effects of covariates and latent variables on the distribution of random variables;
• identify a range of stochastic models and derive a range of their behaviours including Poisson/Counting Processes, Branching processes, Markov processes, Queueing models, Optimisation of Markov decision chains.
• use R and Matlab to tackle a range of problems with stochastic processes which analytical methods cannot give tractable answers to.

Core texts:

• Grimmett, G  and Stirzaker, D. R. (2001). Probability and Random Processes (3rd edition) OUP Oxford.
• Ross, S. (2005). A First Course in Probability (Seven edition) Pearson Education.
• Morgan, B.J. T. (1984). Elements of Simulation. Chapman and Hall.
• Winston, W. L. (2004). Operations Research : Applications & Algorithms. Thomson/Brooks/Cole.
• Gross, D. and Harris, C. M. (1985) Fundamentals of Queueing Theory. Wiley.
• Gallager, R.G. (2013). Stochastic Processes: Theory for Applications. Cambridge University Press.

Assessment: Assessment will be through a combination of summer exam (80%) and coursework (20%). The coursework consists of weekly exercises during the first five weeks and of a computational task covering the other five weeks of material.

Contact hours: There will be a mixture of lectures, tutorials and computer workshops totalling approximately 40 hours contact. In addition, private study will make up the majority of the learning hours.