WS 2023/24: Mathematics of Machine Learning




The lecture “Mathematics of Machine Learning” serves as an introduction to analyzing common numerical problems appearing in modern machine learning applications. We will particularly focus on supervised learning and the dynamics of stochastic gradient descent. This involves the analysis of stochastic processes in discrete time, whose behavior is closely linked to deterministic, as well as stochastic, differential equations. We derive convergence rates for the classical Robbins-Monro algorithm and its Ruppert-Polyak smoothing and analyze the effect of adding inertia (momentum) to the dynamical system. Other possible topics include (stable) central limits theorems, Multilevel Monte Carlo and reinforcement learning. While the general techniques for the asymptotic analysis of stochastic processes are also introduced, proper basic knowledge of probability theory (including martingale theory) is required.