Multivariate analysis

The objective of this course is to introduce the basic techniques for quantitative analysis of multivariate data, with a particular focus on techniques useful for signal processing and machine learning. The methods that will be of interest to us in this context are both statistical and algebraic in nature.

By the end of the course, students will be able to explain the principles of the techniques covered, distinguish between them, apply them in practice, and choose the one that is appropriate for a given application.

The initial part of the course includes a review of linear algebra (spectral decomposition, singular value decomposition) and probability and statistics (random vector, covariance matrix, empirical moments), as well as an introduction to the problem of low-rank approximation of a matrix. Next, we give a statistical formulation of the principal component analysis (PCA) technique, and then we study its properties and practical aspects related to its use. Finally, we introduce the independent component analysis (ICA) problem, and then describe some classical approaches (kurtosis maximization, negentropy maximization, mutual information minimization) and the basic algorithms that derive from them.

·        Linear algebra and matrix analysis (spectral decomposition, singular value decomposition, Frobenius and spectral norms of a matrix)

·        Probability and statistics (random vectors, their theoretical and empirical moments)