STAC62F - Probability and Stochastic Processes I
Announcements
Instructor
Professor Michael Evans
Office: IA4026
email: mevansthree.evans@utoronto.ca
Time and Place
Three hours of lectures per week with a video posted of each lecture.
The classes are: Monday 12-13 in IAB1100 and Wednesday 12-14 in SW319.
Website
http://www.utstat.utoronto.ca/mikevans/stac62/staC622025.html
Office Hours
The in-person office hours will be right after class in my office IA4026, Monday 13-15 and Wednesday 14-15.
Evaluation
There will be a midterm of 2 hours and a final exam of 3 hours worth 40% and 60%, respectively. Both the midterm and the final will be open book (paper only, no electronics).
Course Description
STAC62 is a theoretical course. It is concerned with the mathematics of probability theory. The
course material is difficult and somewhat abstract. You have to expect to work fairly hard to learn it
effectively. A good understanding of the topics covered is necessary for many applications like
mathematical finance, statistical computation, machine learning, statistical inference, etc.
The following topics will be covered.
1. Basic Probability
2. Random Variables and Stochastic Processes
3. Expectation
4. Convergence
5. Gaussian Processes
Lecture Notes and Texts
The course will be based on the class notes as posted here. You can print out the notes and follow along
in class or with the videos. The lectures will follow the notes fairly closely.
The notes will contain Exercises which you are required to do. Solutions to the Exercises will be
periodically posted typically a week after the relevant class. If you cannot do the Exercises, then you need to review the Lecture Notes until
you can, otherwise you have not understood the material. I will also post some additional Exercises from time to time.
If you do not spend time doing the Exercises you will very likely do poorly in the course.
It is the only way to learn this material.
The first four chapters and Chapter 11 of the online
textbook from STAB52 are also relevant to the
course. You are required to review this material. Some problems for the Exercises will be taken from this book.
The text Probability and Random Processes: by Grimmett and Stirzaker may also
prove to be helpful but it is generally above the level of this course.
The lecture slides will be posted below ahead of each class and correspond to the videos.
1. Basic Probability
2. Random Variables and Stochastic Processes
- Lecture 5
- random variables, inverse images, marginal probability model, random vectors
Using the Good Sets Principle
Solutions to Exercises
- Lecture 6
- cumulative distribution functions, discrete distributions, multinomial distribution
Solutions to Exercises
- Lecture 7 - absolutely continuous distributions, density functions, the standard multivariate normal distribution
Solutions to Exercises
- Lecture 8 - change of variable discrete case and marginal distributions
Solutions to Exercises
- Lecture 9
- change of variable absolutely continuous case case and marginal distributions
Solutions to Exercises
- Lecture 10
- definition of a stochastic process, Kolmogorov Consistency Theorem, Gaussian processes
See Solutions for Lecture 11 for Solutions to Exercises for Lecture 10.
- Lecture 11 - mutually statistically independent random variables
Solutions to Exercises
- Lecture 12 - conditional distributions, discrete and absolutely continuous cases, marginals and conditionals of the multivariate normal
Solutions 1 to Exercises
Solutions 2 to Exercises
3. Expectation
- Lecture 13
- simple functions, expectation of a simple function, positive and negative parts of a r.v., general definition of expectation of a r.v.
Solutions to Exercises, Lecture 13 Supplement.
- Lecture 14
- properties of E
- Lecture 15 - convergence with probability 1, monotone and dominated convergence theorems, computing expectations
Solutions to Exercises
- Lecture 16
- properties of mean vectors and variance matrices
Solutions to Exercises Part 1, Solutions to Exercises Part 2
- Lecture 17 - mean, autocovariance and autocorrelation functions for stochastic processes, random walks
Solutions to Exercises
- Lecture 18 - Markov inequality, Chebyshev inequality, Chernoff bounds, Cauchy-Schwartz inequality, best affine predictor
Solutions to Exercises
- Lecture 19
- Jensen's inequality, Kullback-Leibler distance
Solutions to Exercises
- Lecture 20
- conditional expectations, martingales
Solutions to Exercises,
Lecture 20 Supplement
- Lecture 21 - probability and moment generating functions, characteristic functions
Solutions to Exercises
4. Convergence
- Lecture 22
- convergence in distribution, weak law of large numbers, Central Limit Theorem
- Lecture 23 - convergence in probability, convergence in mean of order r, relationships among the modes of convergence, geometry of L^2
Solutions to Exercises
5. Gaussian Processes
- Lecture 24
- strictly stationary processes, discrete time, autoregressive of order 1 Gaussian process