Statistical and Mathematical Methods for AI

Course contents

Theory module contents

1. Elements of Linear Algebra

• Matrices

• Solving systems of linear equations

• Vector Spaces

• Linear independence

• Basis and Rank

• Linear Mappings

• Affine Spaces

2. Analytic Geometry

3. Matrix Decompositions

4. Elements of Multivariate Analysis 

• Gradient, Jacobian, Hessian. Taylor theorem.

• Convex functions and sets.

5. Multivariate Optimization

• Linear least squares.

• Extrema of multivariate functions. Optimality conditions.

• Descent methods. Gradient type methods and Newton type methods.

• Regularization.

• Basis concepts of stochastic optimization.

6. Elements of probability and statistics.

• Probability and Bayes theorem.
• Random variables. Continuous and discrete distributions of random variables. Normal and Poisson distributions. 
• Independent and dependent variables. Covariance and correlation.
• Estimates: Maximum Likelihood and Maximum a Posteriori estimates.
• Cross entropy and Kullback-Leibler divergence.

Lab module contents

• Numerical Linear Algebra with python
  • python basics
  • Introduction to python for NLA
  • Visualization with matplotlib
  • Solving linear systems with python
• Basics of Machine Learning
  • A (very short) introduction to Machine Learning
  • Data Compression with Singular Value Decomposition (SVD)
  • Dimensionality Reduction with PCA
• Optimization

Lab module contents old

• Numerical Linear Algebra with python

• Lecture 1 : Introduction to numpy

• Lecture 2: Plotting with matplotlib

• Lecture 3: Introduction to the solution of linear systems using numpy

• Lecture 4: A (very short) introduction to Machine Learning

• Lecture 5: Data compression with SVD

• Lecture 6: Dimensionality Reduction with PCA

• Lecture 7: Optimization with Gradient Descent

• Lecture 8: Stochastic Gradient Descent

• Lecture 9: Linear and Polynomial Regression

• Lecture 10: MLE and MAP

• Lecture 11: Logistic regression (for Homework 3)

Homework assignments

Homework assignments

• Homework 1: Linear Algebra and Floating Point Arithmetic
• Homework 2: SVD and PCA for Machine Learning
• Homework 3: Optimization
• Homework 4: MLE and MAP

Exam

It is mandatory to complete the homework assigned in the Laboratory lessons to have the exam. The exam consists in a written test and a brief oral discussion about the homework assignments. The final score is the sum of:

• the score of the written test (maximum 22/30)
• the score of the oral exam on the assigned homework (maximum 10/30) If the final score is greater than 30, the laude is assigned.

Oral questions here.

Reading material

Reading

• M.P. Deisenroth, A.A. Faisal, C.S. Ong (2020) (main textbook)

Resources

• Course web page
• Tutor teaching material