Catalog

Deep Learning

Electrical Engineering and Computer Science MIT CC BY-NC-SA 4.0 24 lectures

This course covers the fundamentals of deep learning, including both theory and applications. Topics include neural net architectures (MLPs, CNNs, RNNs, graph nets, transformers), geometry and invariances in deep learning, backpropagation and automatic differentiation, learning theory and generalization in high dimensions, and applications to computer vision, natural language processing, and robotics.

Syllabus

  1. 1 Lec 01. Introduction to Deep Learning
  2. 2 Lec 02. How to Train a Neural Net
  3. 3 Lec 03. Approximation Theory
  4. 4 Lec 04. Architectures: Grids
  5. 5 Lec 05. Architectures: Graphs
  6. 6 Lec 06. Generalization Theory
  7. 7 Lec 07. Scaling Rules for Optimization
  8. 8 Lec 08. Architectures: Transformers
  9. 9 Lec 09. Hacker's Guide to Deep Learning
  10. 10 Lec 10. Architectures: Memory
  11. 11 Lec 11. Representation Learning: Reconstruction-Based
  12. 12 Lec 12. Representation Learning: Similarity-Based
  13. 13 Lec 13. Representation Learning: Theory
  14. 14 Lec 14. Generative Models: Basics
  15. 15 Lec 15. Generative Models: Representation Learning Meets Generative Modeling
  16. 16 Lec 16. Generative Models: Conditional Models
  17. 17 Lec 17. Generalization: Out-of-Distribution (OOD)
  18. 18 Lec 18. Transfer Learning: Models
  19. 19 Lec 19. Transfer Learning: Data
  20. 20 Lec 20. Scaling Laws
  21. 21 Lec 21. Language Models
  22. 22 Lec 23. Metrized Deep Learning
  23. 23 Lec 24. Inference Methods for Deep Learning
  24. 24 PyTorch Tutorial

Course materials