| About Mastermath |
| Courses & exams |
| Program & schedule |
| Registration & other forms |
| Links |
| Locations |
| Contact |
|
Empirical Processes and Statistical Learning
| Credits | 8 credit points |
| Instructors | Vaart, A.W. van der (Vrije Universiteit) |
| avdvaart@math.leidenuniv.nl | |
| Aim | Learn the mathematical theory of empirical processes and its applications in statistics and machine learning |
| Description | The empirical measure of a set of random variables is the discrete random measure that puts a point mass of size one divided by the sample size at each of the random variables. The expectation of some function under this measure is just an average, and under appropriate integrability this average will satisfy a law of large numbers (LLN) and, after centering and scaling, a central limit theorem (CLT). Empirical process theory studies these objects for many functions jointly, and is concerned with the LLN or CLT uniformly in classes of functions, as well as inequalities that measure the size of suprema of these objects over classes of functions. The empirical distribution function and the classical empirical process on the line are very special examples, for which the uniform LLN and CLT were obtained by Glivenko-Cantelli in the 1930s and Donsker in the 1940/50s, respectively. The Kolmogorov-Smirnov statistic for goodness-of-fit and its approximation by the maximum of a Brownian bridge process is one important application of these classical results. The general theory of empirical processess is more recent, and is based on Vapnik-Cervonenkis combinatorial theory and Kolmogorov entropy. This theory has many applications in statistics. In this course we shall focus on its use to derive rates of estimation of nonparametric statistical procedures. In the terminology of computer science this is called statistical learning theory. For instance, one obtains a sample of instances (realizations of variables in some measurable space), each being classified as a 0 or 1, and one wants to build a procedure that can classify a future instance as a 0 or 1. Empirical risk minimization, support vector machines, and kernel learners are all methods to solve this problem, and can be studied using empirical process theory. |
| Organization | Lectures. |
| Examination | Written exam. |
| Literature | Chapters from Weak Convergence and Empirical Processes by Van der Vaart and Wellner (2nd edition). Lecture Notes. |
| Prerequisites | Measure-theoretic probability. A good basis in Analysis. |
| Remarks | Homepage lecturer. |