Lecture Date Topics Slides, Videos Additional Materials Quiz
0
  • Course Logistics
  • Learning Objectives
  • Grading
  • Deadlines
Slides
YouTube
No Quiz
1 August 24
  • Introduction
Slides
YouTube
MediaServices
Quiz 1: Aug 28 – Aug 30
2 August 26
  • Neural Nets As Universal Approximators
Slides
YouTube
MediaServices
IIT CS6840: Size and Depth Complexity of Boolean Circuits
The Synthesis of Two-Terminal Switching Circuits
3 August 31
  • Training Part I
    • The Problem of Learning
    • Empirical Risk Minimization
Slides
YouTube
MediaServices
Quiz 2: Sep 4 – Sep 6
4 September 2
  • Training Part II
    • Gradient Descent
    • Training the Network
Slides
YouTube
MediaServices
5 September 9
  • Training Part III
    • Backpropagation
    • Calculus of Backpropagation
Slides
YouTube
MediaServices
Quiz 3: Sep 11 – Sep 13
6 September 11
  • Training Part IV
    • Convergence issues
    • Loss Surfaces
    • Momentum
Slides
YouTube
MediaServices
7 September 14
  • Training Part V
    • Optimization
    • Batch Size, SGD, Mini-batch, Second-order Methods
Slides
YouTube
MediaServices
Quiz 4: Sep 18 – Sep 20
8 September 16
  • Training Part VI
    • Optimizers and Regularizers
    • Choosing a Divergence (Loss) Function
    • Batch Normalization
    • Dropout
Slides
YouTube
MediaServices
9 September 21
  • Convolutional Neural Networks (CNNs) I
Slides
YouTube
MediaServices
Quiz 5: Sep 25 – Sep 27
10 September 23
  • CNNs II
Slides
YouTube
MediaServices
11 September 28
  • CNNs III
Slides
YouTube
MediaServices
Quiz 6: Oct 2 – Oct 4
12 September 30
  • CNNs IV
Slides
YouTube
MediaServices
13 October 5
  • Recurrent Neural Networks (RNNs) I
Slides
YouTube
MediaServices
Quiz 7: Oct 9 – Oct 11
14 October 7
  • RNNs II
Slides
YouTube
MediaServices
- October 12
  • No Class - Fall Break
No Quiz
- October 14
  • No Class - Fall Break
No Quiz
15 October 19
  • Sequence to Sequence Models
  • Connectionist Temporal Classification (CTC)
Quiz 8: Oct 23 – Oct 25
16 October 21
  • Connectionist Temporal Classification
    • Blanks
    • Beam Search
17 October 26
  • Language Models
  • Translation
Quiz 9: Oct 30 – Nov 1
18 October 28
  • Attention Models
  • Transformers
19 November 2
  • Transformers and Newer Architectures
Quiz 10: Nov 6 – Nov 8
20 November 4
  • Large Language Models
21 November 9
  • Representation and Autoencoders
Quiz 11: Nov 13 – Nov 15
22 November 11
  • Variational Auto Encoders
23 November 16
  • Normalizing Flows and Diffusion
Quiz 12: Nov 20 – Nov 22
24 November 18
  • Generative Adversarial Networks
25 November 23
  • Graph Neural Networks
Quiz 13: Nov 27 – Nov 29
26 November 25
  • Hopfield Networks
27 November 30
  • Boltzmann Machines
Quiz 14: Dec 4 – Dec 6
28 December 2
  • Homework 1-4 Debrief and Deployment