Catalog

Matrix Methods in Data Analysis, Signal Processing, and Machine Learning

Mathematics MIT CC BY-NC-SA 4.0 34 lectures

Linear algebra concepts are key for understanding and creating machine learning algorithms, especially as applied to deep learning and neural networks. This course reviews linear algebra with applications to probability and statistics and optimization–and above all a full explanation of deep learning.

Syllabus

  1. 1 Lecture 1: The Column Space of A Contains All Vectors Ax
  2. 2 Lecture 2: Multiplying and Factoring Matrices
  3. 3 Lecture 3: Orthonormal Columns in Q Give Q’Q = I
  4. 4 Lecture 4: Eigenvalues and Eigenvectors
  5. 5 Lecture 5: Positive Definite and Semidefinite Matrices
  6. 6 Lecture 6: Singular Value Decomposition (SVD)
  7. 7 Lecture 7: Eckart-Young: The Closest Rank k Matrix to A
  8. 8 Lecture 8: Norms of Vectors and Matrices
  9. 9 Lecture 9: Four Ways to Solve Least Squares Problems
  10. 10 Lecture 10: Survey of Difficulties with Ax = b
  11. 11 Lecture 11: Minimizing ‖x‖ Subject to Ax = b
  12. 12 Lecture 12: Computing Eigenvalues and Singular Values
  13. 13 Lecture 13: Randomized Matrix Multiplication
  14. 14 Lecture 14: Low Rank Changes in A and Its Inverse
  15. 15 Lecture 15: Matrices A(t) Depending on t, Derivative = dA/dt
  16. 16 Lecture 16: Derivatives of Inverse and Singular Values
  17. 17 Lecture 17: Rapidly Decreasing Singular Values
  18. 18 Lecture 18: Counting Parameters in SVD, LU, QR, Saddle Points
  19. 19 Lecture 19: Saddle Points Continued, Maxmin Principle
  20. 20 Lecture 20: Definitions and Inequalities
  21. 21 Lecture 21: Minimizing a Function Step by Step
  22. 22 Lecture 22: Gradient Descent: Downhill to a Minimum
  23. 23 Lecture 23: Accelerating Gradient Descent (Use Momentum)
  24. 24 Lecture 24: Linear Programming and Two-Person Games
  25. 25 Lecture 25: Stochastic Gradient Descent
  26. 26 Lecture 26: Structure of Neural Nets for Deep Learning
  27. 27 Lecture 27: Backpropagation: Find Partial Derivatives
  28. 28 Lecture 30: Completing a Rank-One Matrix, Circulants!
  29. 29 Lecture 31: Eigenvectors of Circulant Matrices: Fourier Matrix
  30. 30 Lecture 32: ImageNet is a Convolutional Neural Network (CNN), The Convolution Rule
  31. 31 Lecture 33: Neural Nets and the Learning Function
  32. 32 Lecture 34: Distance Matrices, Procrustes Problem
  33. 33 Lecture 35: Finding Clusters in Graphs
  34. 34 Lecture 36: Alan Edelman and Julia Language

Course materials