11-785 Deep Learning

11-785 Introduction to Deep Learning

Fall 2023
Class Streaming Link

In-Person Venue: Giant Eagle Auditorium, Baker Hall (A51)

Bulletin and Active Deadlines

Assignment	Deadline	Description	Links
HW4P1	Early Submission: 16th Nov, 11:59 PM Final Submission: 2nd Dec, 11:59 PM	Language Modeling	Autolab
HW4P2	Early Submission: 16th Nov, 11:59 PM Final Submission: 2nd Dec, 11:59 PM	Attention-based Speech Recognition	Kaggle Writeup Starter Notebook
Project Gallery
Here's an example of a successful project from Fall 2020. The team developed an AI Limmerick generator, and compiled a book from the AI Poet's creations.	Project Report, Project Video, Book (Amazon)
This piece is performed by the Chinese Music Institute at Peking University (PKU) together with PKU's Chinese orchestra. This is an adaptation of Beethoven: Serenade in D major, Op.25 - 1. Entrata (Allegro),for Chinese transverse flute (Dizi), clarinet and flute.

Note for Enrolled Students: Please sign up for Piazza if you haven't done so. :-)

The Course

“Deep Learning” systems, typified by deep neural networks, are increasingly taking over all the AI tasks, ranging from language understanding, speech and image recognition, to machine translation, planning, and even game playing and autonomous driving. As a result, expertise in deep learning is fast changing from an esoteric desirable to a mandatory prerequisite in many advanced academic settings, and a large advantage in the industrial job market.

In this course we will learn about the basics of deep neural networks, and their applications to various AI tasks. By the end of the course, it is expected that students will have significant familiarity with the subject, and be able to apply Deep Learning to a variety of tasks. They will also be positioned to understand much of the current literature on the topic and extend their knowledge through further study.

If you are only interested in the lectures, you can watch them on the YouTube channel.

Course description from student point of view

The course is well rounded in terms of concepts. It helps us understand the fundamentals of Deep Learning. The course starts off gradually with MLPs and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. We get a complete hands on with PyTorch which is very important to implement Deep Learning models. As a student, you will learn the tools required for building Deep Learning models. The homeworks usually have 2 components which is Autolab and Kaggle. The Kaggle components allow us to explore multiple architectures and understand how to fine-tune and continuously improve models. The task for all the homeworks were similar and it was interesting to learn how the same task can be solved using multiple Deep Learning approaches. Overall, at the end of this course you will be confident enough to build and tune Deep Learning models..

Prerequisites

We will be using Numpy and PyTorch in this class, so you will need to be able to program in python3.
You will need familiarity with basic calculus (differentiation, chain rule), linear algebra and basic probability.

Units

Courses 11-785 and 11-685 are equivalent 12-unit graduate courses, and have a final project and HW 5 respectively. Course 11-485 is the undergraduate version worth 9 units, the only difference being that there is no final project or HW 5.

Acknowledgments

Your Supporters

Instructors:

Bhiksha Raj : bhiksha@cs.cmu.edu
Rita Singh : rsingh@cs.cmu.edu

TAs:

Denis Musinguzi: dmusingu@andrew.cmu.edu
Dheeraj Pai: dmohanda@andrew.cmu.edu
Harini Subramanyan: harinisu@andrew.cmu.edu
Harshit Mehrotra: hmehrotr@andrew.cmu.edu
Harshith Arun Kumar: harunkum@andrew.cmu.edu
Jean De Dieu Nyandwi: jeandedi@andrew.cmu.edu
Jeel Shah: jdshah@andrew.cmu.edu
Kelsey J. Harvey: kelseyjh@andrew.cmu.edu
Liangze "Josh" Li: liangzel@andrew.cmu.edu
Meng Zhou: mengzhou@andrew.cmu.edu
Miya Sylvester: nsylvest@andrew.cmu.edu
Pavitra Kadiyala: pkadiyal@andrew.cmu.edu
Qin Wang: qinw@andrew.cmu.edu
R Raghav: rraghavr@andrew.cmu.edu
Rucha Manoj Kulkarni: rmkulkar@andrew.cmu.edu
Sarthak Bisht: sbisht@andrew.cmu.edu
Shikhar Agnihotri: sagnihot@andrew.cmu.edu
Sumesh Kalambettu Suresh: skalambe@andrew.cmu.edu
Jiaye "Tony" Zou: jiayez@andrew.cmu.edu
Vish: vishhvas@andrew.cmu.edu
Yohannes Haile: yohanneh@andrew.cmu.edu
Muhammed Danso: mdanso@andrew.cmu.edu
Schadrack Niyibizi: sniyibiz@andrew.cmu.edu
Emmanuel Ndayisaba: enn@andrew.cmu.edu

Pittsburgh Schedule (Eastern Time)

Lecture: Monday and Wednesday, 8:00 a.m. - 9:20 a.m. - Good times :)

Recitation: Friday, 8:00 a.m. - 9:20 a.m.

Event Calendar: The Google Calendar below ideally contains all events and deadlines for student's convenience (Deadlines for Assignments/HWs will be updates as and when they are released). Please feel free to add this calendar to your Google Calendar by clicking on the plus (+) button on the bottom right corner of the calendar below. Any adhoc changes to the schedule will be visible on the calendar first.

OH Calendar: The Google Calendar below contains the schedule for Office Hours. Please feel free to add this calendar to your Google Calendar by clicking on the plus (+) button on the bottom right corner of the calendar below. Any adhoc changes to the schedule, including extra OH, will be visible on the calendar first.

Office hours: This is the schedule for this semester. We will be using OHQueue(11-785) for both zoom and in-person Office hours. Please refer the OH calendar/ Piazza for updated information on OH hours.

Day	Time (Eastern Time)	TA	Zoom/In Person
Monday	7:00AM to 8:00AM	Muhammed Danso	Zoom
	11:00AM to 12:30PM	Kelsey J. Harvey	In-person
	1:00PM to 2:00PM	Miya Sylvester	Zoom
	4:30PM to 6:30PM	Harini & Rucha & Pavitra & Dheeraj	In-person (Wean 3110)
	9:30PM to 10:30PM	Meng Zhou	Zoom
Tuesday	7:00AM to 8:00AM	Schadrack Niyibizi	In-person (Kigali)
	8:00AM to 9:00AM	Raghav & Sumesh	In-person (GHC 5417)
	11:00AM to 12:00PM	Liangze "Josh" Li	Zoom
	2:30PM to 3:30PM	Jeel & Pavitra	In-person (GHC 5417)
	5:00PM to 6:00PM	Harshit Mehrotra	In-person (GHC 5417)
	6:00PM to 7:00PM	Harshith Arun Kumar	Zoom
Wednesday	7:00AM to 8:00AM	Muhammed Danso	In-person (Kigali)
	12:30PM to 1:30PM	Dheeraj Pai	In-person (TCS349)
	12:30PM to 1:30PM	Harini & Rucha	Zoom
	4:00PM to 5:00PM	Tony & Qin	In-person (Wean 3110)
	6:30 PM to 7:30 PM	Sarthak Bisht	In-person (GHC 5417)
Thursday	5:00AM to 6:00AM	Yohannes Haile	In-person (Kigali)
	7:00AM to 8:00AM	Schadrack Niyibizi	Zoom
	8:00AM to 9:00AM	Raghav & Sumesh	In-person (GHC 5417)
	10:00AM to 11:00AM	Emmanuel Ndayisaba	In-person (Kigali)
	11:00AM to 12:00PM	Liangze "Josh" Li.	Zoom
	5:00PM to 6:00PM	Jeel & Harshit	In-person (GHC 5417)
	6:00PM to 7:00PM	Harshith Arun Kumar	Zoom
Friday	9:00AM to 10:00AM	Denis Musinguzi	Zoom
	9:00AM to 10:00AM	Yohannes Haile	Zoom
	2:00PM to 3:00PM	Miya Sylvester	Zoom
	4:30PM to 5:30PM	Qin Wang	Zoom
Saturday	9:00AM to 10:00AM	Shikhar Agnihotri	In-person (Wean 3119)
	10:00AM to 11:00AM	Jiaye "Tony" Zou	Zoom
	12:00PM to 2:00PM	Kelsey J. Harvey	Zoom
	5:00PM to 6:00PM	Sarthak Bisht	In-person (GHC 5417)
Sunday	9:00AM to 10:00AM	Denis Musinguzi	In-person (GHC 5417)
	10:00AM to 11:00AM	Shikhar Agnihotri	Zoom
	5:00PM to 6.00PM	Emmanuel Ndayisaba	Zoom
	9:00PM to 10:00PM	Meng Zhou	Zoom

Homework Hackathon: During 'Homework Hackathons', students will be assisted with homework by the course staff. It is recommended to come as study groups.
Location: Rashid Auditorium, located on the fourth floor of the Hillman Center (the part of the Gates and Hillman Centers closest to Forbes Avenue)
Time: Every Saturday, around 2-5pm

Course Work

Policy
Breakdown
Score Assignment		Grading will be based on weekly quizzes (24%), homeworks (50%) and a course project (25%). Note that 1% of your grade is assigned to Attendance.
Quizzes
Quizzes		There will be weekly quizzes. We will retain your best 12 out of the remaining 14 quizzes. Quizzes will generally (but not always) be released on Friday and due 48 hours later. Quizzes are scored by the number of correct answers. Quizzes will be worth 24% of your overall score.
Assignments
Assignments		There will be five assignments in all, plus the Peer Review assignment during the last week of the semester. Assignments will include autolab components, where you must complete designated tasks, and a kaggle component where you compete with your colleagues. Autolab components are scored according to the number of correctly completed parts. We will post performance cutoffs for HIGH (90%), MEDIUM (70%), LOW (50%), and VERY LOW (30%) for Kaggle competitions. Scores will be interpolated linearly between these cutoffs. Assignments will have a “preliminary submission deadline”, an “on-time submission deadline” and a “late-submission deadline.” Early submission deadline: You are required to make at least one submission to Kaggle by this deadline. People who miss this deadline will automatically lose 10% of subsequent marks they may get on the homework. This is intended to encourage students to begin working on their assignments early. On-time deadline: People who submit by this deadline are eligible for up to five bonus points. These points will be computed by interpolation between the A cutoff and the highest performance obtained for the HW. The highest performance will get 105. Late deadline: People who submit after the on-time deadline can still submit until the late deadline. There is a 10% penalty applied to your final score, for submitting late. Slack days: Everyone gets up to 10 slack days, which they can distribute across all their homework P2s only. Once you use up your slack days you will fall into the late-submission category by default. Slack days are accumulated over all parts of all homeworks, except HW0, to which no slack applies. Kaggle scoring: We will use max(max(on-time score), max(slack-day score), .0.9max(late-submission score))* as your final score for the HW. If this happens to be a slack-days submission, slack days corresponding to the selected submission will be counted. Assignments carry 50% of your total score. HW0 is not graded (but is mandatory), while each of the subsequent four are worth 12.5%. A fifth HW, HW5, will be released later in the course and will have the same weight as a course project. Please see Project section below for more details. Bonus HWs will count towards the score of the correlating HWp1 assignment number. (For example, Bonus1 points go towards HW1p1.) The Peer Review assignment is required of all students, 11-485/685/785. The task is for all students to review and grade 4-6 of the videos. It is to be completed over the last weekend, after classes finish (but before finals week). We will tell you which projects you have been assigned to review; each review should take around 15~20 minutes. Here is what we expect you to do for each review: Watch the video carefully. As you watch the video, jot down some notes/concerns/questions that you might have. Reference the initial report to clear up any confusion. You must post at least one comment to the corresponding Piazza post of your reviewee. This comment must be a meaningful question or concern that demonstrates you have understood the material. Finally, fill out the project review form carefully.
Project
Project		All students taking a graduate version of the course are required to do a course project. The project is worth 25% of your grade. These points are distributed as follows: 20% - Midterm Report; 35% - Project Video; 5% - Responding to comments on Piazza; 40% - Project report. Note that a Project is mandatory for 11-785/18-786 students. In the event of a catastrophe (remember Spring 2020), the Project may be substituted with HW5. 11-685 Students may choose to do a Project instead of HW5. Either your Project OR HW5 will be graded. Important Information for project reports and video presentations (including midterm report rubric, final report rubric, video timeline, and video grading)
Attendance
Attendance		If you are in section A you are expected to attend in-person lectures. We will track attendance. If you are in any of the other (out-of-timezone) sections, you must watch lectures live on zoom. Real-time viewing is mandatory unless you are in inconvenient time zones. Others are required to obtain specific permission to watch the pre-recorded lectures (on MediaServices). If viewed on MediaServices, the lectures of each week must be viewed before 8AM of the Monday following the following week (Otherwise, it doesn’t count) At the end of the semester, we will select a random subset of 50% of the lectures and tabulate attendance If you have attended at least 70% of these (randomly chosen) lectures, you get the attendance point
Final grade
Final grade		The end-of-term grade is curved. Your overall grade will depend on your performance relative to your classmates.
Pass/Fail
Pass/Fail		Students registered for pass/fail must complete all quizzes, HWs and if they are in the graduate course, the project. A grade equivalent to B- is required to pass the course.
Auditing
Auditing		Auditors are not required to complete the course project, but must complete all quizzes and homeworks. We encourage doing a course project regardless.
		End Policy

Study groups

This semester we will be implementing study groups. It is highly recommended that you join a study group; Check piazza for further updates.

Piazza: Discussion Board

Piazza is what we use for discussions. You should be automatically signed up if you're enrolled at the start of the semester. If not, please sign up TBA. Also, please follow the Piazza Etiquette when you use the piazza.

AutoLab: Software Engineering

AutoLab is what we use to test your understand of low-level concepts, such as engineering your own libraries, implementing important algorithms, and developing optimization methods from scratch.

Kaggle: Data Science

Kaggle is where we test your understanding and ability to extend neural network architectures discussed in lecture. Similar to how AutoLab shows scores, Kaggle also shows scores, so don't feel intimidated -- we're here to help. We work on hot AI topics, like speech recognition, face recognition, and neural machine translation.

MediaServices/YouTube: Lecture and Reciation Recordings

CMU students who are not in the live lectures should watch the uploaded lectures at MediaServices in order to get attendance credit. Links to individual videos will be posted as they are uploaded.

Our YouTube Channel is where non-CMU folks can view all lecture and recitation recordings. Videos marked “Old“ are not current, so please be aware of the video title.

Books and Other Resources

The course will not follow a specific book, but will draw from a number of sources. We list relevant books at the end of this page. We will also put up links to relevant reading material for each class. Students are expected to familiarize themselves with the material before the class. The readings will sometimes be arcane and difficult to understand; if so, do not worry, we will present simpler explanations in class.

You can also find a nice catalog of models that are current in the literature here. We expect that you will be in a position to interpret, if not fully understand many of the architectures on the wiki and the catalog by the end of the course.

Academic Integrity

You are expected to comply with the University Policy on Academic Integrity and Plagiarism.

You are allowed to talk with and work with other students on homework assignments.
You can share ideas but not code. You should submit your own code.

Your course instructor reserves the right to determine an appropriate penalty based on the violation of academic dishonesty that occurs. Violations of the university policy can result in severe penalties including failing this course and possible expulsion from Carnegie Mellon University. If you have any questions about this policy and any work you are doing in the course, please feel free to contact your instructor for help.

Class Notes

A book containing class notes is being developed in tandem with this course; check it out.

Schedule of Lectures

You can watch the recorded lectures on MediaServices.

Lecture	Date	Topics	Slides and Video	Additional Materials	Quiz
0	-	Course Logistics Learning Objectives Grading Deadlines	Slides (*.pdf) Video (YT)		-
1	Monday, 28 Aug	Introduction	Slides (*.pdf) Video (YT)	The New Connectionism (1988) On Alan Turing's Anticipation of Connectionism McCullogh and Pitts paper Rosenblatt: The perceptron Bain: Mind and body Hebb: The Organization Of Behaviour	Quiz 1
2	Wednesday, 30 Aug	Neural Nets As Universal Approximators	Slides (*.pdf) Video (YT)	Shannon (1949) Boolean Circuits On the Bias-Variance Tradeoff	Quiz 1
3	Monday, 4 Sep	Training 1: Learning the Network (Mandatory to watch recording before Wednesday)	Slides (*.pdf) Video (YT)		Quiz 2
4	Wednesday, 6 Sep	Training 2: The problem of learning, Empirical Risk Minimization	Slides (*.pdf) Video (YT)	Widrow and Lehr (1992) Adaline and Madaline Convergence of perceptron algorithm Threshold Logic TC (Complexity) AC (Complexity)	Quiz 2
5	Monday, 11 Sep	Training 3: Empirical risk minimization and gradient descent Training the network: Setting up the problem Backpropagation Calculus of Backpropagation	Slides (*.pdf) Video (YT) Video, 5b (YT)	Werbos (1990) Rumelhart, Hinton and Williams (1986)	Quiz 3
6	Wednesday, 13 Sep	Training 4 Convergence issues Loss Surfaces Momentum	Slides (*.pdf) Video (YT)	Backprop fails to separate, where perceptrons succeed, Brady et al. (1989) Why Momentum Really Works	Quiz 3
7	Monday, 18 Sep	Training 5 Optimization Batch Size, SGD, Mini-batch, second-order methods	Slides (*.pdf) Video (YT)	Momentum, Polyak (1964) Nestorov (1983) Derivatives and Influences	Quiz 4
8	Wednesday, 20 Sep	Training 6 Optimizers and Regularizers Choosing a divergence (loss) function Batch normalization Dropout	Slides (*.pdf) Video, 8a (YT) Video, 8b (YT)	Derivatives and Influence Diagrams ADAGRAD, Duchi, Hazan and Singer (2011) Adam: A method for stochastic optimization, Kingma and Ba (2014)	Quiz 4
9	Monday, 25 Sep	CNN 1: Shift Invariance	Slides (*.pdf) Video (YT)		Quiz 5
10	Wednesday, 27 Sep	CNN 2	Slides (*.pdf) Video, 10a (YT) Video, 10b (YT)		Quiz 5
11	Monday, 2 Oct	CNN 3	Slides (*.pdf) Video (YT)		Quiz 6
12	Wednesday, 4 Oct	CNN 4 - Newer Architectures	Slides (*.pdf) Video (YT)		Quiz 6
13	Monday, 9 Oct	RNN 1	Slides (*.pdf) Video (YT)		Quiz 7
14	Wednesday, 11 Oct	RNN 2	Slides (*.pdf) Video (YT)	LSTM How to compute a derivative, extra help for HW3P1 (*.pptx)	Quiz 7
-	Monday, 16 Oct	No Class - Fall Break	-		Quiz 8
-	Wednesday, 18 Oct	No Class - Fall Break	-		Quiz 8
15	Monday, 23 Oct	Sequence to sequence Connectionist Temporal Classification (CTC)	Slides (*.pdf) Video, 15a (YT) Video, 15b (YT)		Quiz 9
16	Wednesday, 25 Oct	Connectionist Temporal Classification (CTC)	Slides (*.pdf) Video (YT)	Labelling Unsegmented Sequence Data with Recurrent Neural Networks	Quiz 9
17	Monday, 30 Oct	Language modeling Translation	Slides (*.pdf) Video (YT)		Quiz 10
18	Wednesday, 1 Nov	Attention Models Transformers	Slides (*.pdf) Video (YT)	Attention Is All You Need The Annotated Transformer - Attention is All You Need paper, but annotated and coded in pytorch!	Quiz 10
19	Monday, 6 Nov	Transformers Newer Transformer Architectures GNNs	Slides (*.pdf) Video (YT)	A comprehensive Survey on Graph Neural Networks	Quiz 11
20	Wednesday, 8 Nov	Learning Representations Autoencoders Losses	Slides (*.pdf) Video (YT)		Quiz 11
21	Monday, 13 Nov	Variational Auto Encoders	Slides (*.pdf) Video (pre-recorded) (YT)	Tutorial on VAEs (Doersch) Autoencoding variational Bayes (Kingma)	Quiz 12
22	Wednesday, 15 Nov	Variational Auto Encoders II	Slides (*.pdf) Video (pre-recorded) (YT)		Quiz 12
23	Monday, 20 Nov	Flow and Diffusion	Slides (*.pdf) Video (YT)		Quiz 13
-	Wednesday, 22 Nov	No Class - Thanksgiving	-		Quiz 13
24	Monday, 27 Nov	Generative Adversarial Networks 1	Slides (*.pdf) Video (YT)		Quiz 14
25	Wednesday, 29 Nov	Generative Adversarial Networks 2	Slides (*.pdf) Video (YT)		Quiz 14
26	Monday, 4 Dec	Hopfield Nets and Auto Associators	Slides (*.pdf) Video (YT)		No Quiz
27	Wednesday, 6 Dec	Guest Lecture: Pulkit Agrawal, MIT	Video (YT)		No Quiz

Schedule of Recitations

` `

Recitation	Date	Topics	Materials	Videos	Instructor
0A	Friday, 25th August	Python & OOP Fundamentals		Video (YT)	Sumesh & Raghav
0B	Friday, 25th August	OOP Fundamentals	Notebook (*.zip)	Video (YT)	Aishwarya & Denis
0C	Friday, 25th August	NumPy Fundamentals	Notebook + Exercise (*.zip)	Video (YT): 1, 2, 3	Harini & Meng
0D	Friday, 25th August	PyTorch Fundamentals	Notebook + Slides + Exercise (*.zip)	Video (YT)	Jean & Sumesh
0E	Friday, 25th August	Introduction to Google Colab	Colab Notebook	Video (YT)	Rucha & Tony
0F	Friday, 25th August	Google Cloud	VM Setup Script	Video (YT): 1, 2	Meng & Harini
0G	Friday, 25th August	AWS		Video (YT)	Josh & Yohannes
0H	Friday, 25th August	Kaggle		Video (YT)	Denis & Shikhar
0I	Friday, 25th August	Datasets	Colab Notebook	Video (YT) 1, 2	Tony & Muhammed
0J	Friday, 25th August	Dataloaders	Notebook (*.zip)	Video (YT)	Raghav & Qin
0K	Friday, 25th August	Data Preprocessing	Colab Notebook 1, 2, 3	Video (YT) 1, 2, 3	Qin & Tony
0L	Friday, 25th August	Debugging	Slides (*.zip)	Video (YT) 1, 2, 3, 4	Shikhar & Harshith
0M	Friday, 25th August	What to do when struggling		Video (YT)	Miya & Aishwarya
0N	Friday, 25th August	Cheating		Video (YT)	Dheeraj & Sarthak
0O	Friday, 25th August	Workflow of HWs	Notebook (*.zip)	Video (YT)	Jeel & Jean
0P	Friday, 25th August	WandB	Colab Notebook	Video (YT) 1, 2	Harshit & Jeel
0Q	Friday, 25th August	To write a report		Video (YT)	David (Spring 2022)
0R	Friday, 25th August	Flow of the project		Video (YT)	Pavitra & Rucha
0S	Friday, 25th August	Git		Video (YT)	Josh & Harshit
0T	Friday, 25th August	Losses	Notebook (*.zip)	Video (YT)	Dheeraj & Harshith
0U	Friday, 25th August	Block Processing	Colab Notebook	Video (YT)	Vish & Muhammed
1	Friday, 1st September	Your first MLP Code	Notebook	Video (YT)	Jiaye "Tony" Zou & Kelsey J. Harvey
HW1 Bootcamp	Thursday, 7th September	How to get started with HW1	Slides (*.zip)	Video (YT)	Denis Musinguzi, Harshit Mehrotra, Liangze "Josh" Li, Pavitra Kadiyala, Qin Wang, Yohannes, Schadrack
2	Friday, 8th September	Optimizing the Networks, Hyperparameter Tuning, Ensembles	Slides (*.pdf)	Video (YT)	Harshit Mehrotra & Liangze "Josh" Li
Hackathon	Saturday, 9th September	HW1			Dheeraj Pai, Liangze "Josh" Li, Pavitra Kadiyala
3	Friday, 15th September	Computing Derivatives & Autograd	Slides (*.pdf)	Video (YT)	Dheeraj Pai & Miya Sylvester
Hackathon	Saturday, 16th September	HW1			Dheeraj Pai, Pavitra Kadiyala, R Raghav
4	Friday, 22nd September	Hyperparameter Tuning Methods, Normalizations	Slides (*.pdf)	Video (YT)	Harini Subramanyan & Jeel Shah
Hackathon	Saturday, 23rd September	HW1			Harshit Mehrotra & Jeel Shah
HW2 Bootcamp	Thursday, 28th September	How to get started with HW2	Slides, P1 (.pdf) Slides, P2 (.pdf)	Video (YT)	Harini Subramanyan, Jeel Shah, Qin Wang, R Raghav, Rucha Manoj Kulkarni, Vish, Muhammad Danso, Schadrack Niyibizi
5	Friday, 29th September	CNN: Basics and Backprop	Slides(*.pptx)	Video (YT)	R Raghav & Schadrack Niyibizi
Hackathon	Saturday, 30th September	HW2			Miya Sylvester, Rucha Manoj Kulkarni, Schadrack Niyibizi
6	Friday, 6th October	CNNs: Classification, Verification	Slides (*.pptx)	Video (YT)	Sumesh Kalambettu Suresh & Emmanuel Ndayisaba
Hackathon	Friday, 7th October	HW2			Harini Subramanyan, Harshith Arun Kumar, Schadrack Niyibizi
7	Friday, 13th October	Paper Writing Workshop	Slides (*.pdf)	Video (YT)	Pavitra Kadiyala & Dheeraj Pai
Hackathon	Saturday, 14th October	HW2			Qin Wang, Harshith Kumar, R Raghav, Sumesh Kalambettu Suresh, Schadrack Niyibizi
Hackathon	Saturday, 21st October	HW2			Schadrack Niyibizi
HW3 Bootcamp	Thursday, 26th October	How to get started with HW3		Video (YT)	Harshit Mehrotra, Qin Wang, Rucha Manoj Kulkarni, Shikhar Agnihotri Sumesh Kalambettu Suresh, Yohannes, Jiaye "Tony" Zou
8	Friday, 27th October	RNN Basics	Slides (.pdf) Notebook (.zip)	Video (YT)	Meng Zhou & Shreyas
Hackathon	Saturday, 28th October	HW3			Miya Sylvester, Rucha Manoj Kulkarni, Schadrack Niyibizi
9	Friday, 3rd November	CTC, Beam Search	Notebook (.ipynb) Slides (.pptx)	Video (YT)	Qin Wang & Shikhar Agnihotri
Hackathon	Saturday, 4th November	HW3			Qin Wang, Schadrack Niyibizi
HW4 Bootcamp	Thursday, 9th November	How to get started with HW4	Slides (.pdf)	Video (YT)	Denis Musinguzi, Harini Subramanyan, Harshit Mehrotra, Liangze "Josh" Li, Meng Zhou, Shikhar Agnihotri, Vish, Muhammad Danso
10	Friday, 10th November	Attention, MT, LAS	Slides (.pdf)	Video (YT)	Rucha Manoj Kulkarni & Muhammed Danso
Hackathon	Saturday, 11th November	HW4			Harini Subramanyan, Liangze "Josh" Li, Miya Sylvester, Muhammad Danso, Schadrack Niyibizi
11	Friday, 17th November	Transformers	Transformer Notebook ViT Notebook	Video (YT)	Vish & Yohannes Haile
Hackathon	Saturday, 18th November	HW4			Harshit Mehrotra, Schadrack Niyibizi
12	Friday, 1st December	Graph Neural Networks	Colab Notebook	Video (YT)	Denis Musinguzi & Kelsey J. Harvey
Hackathon	Saturday, 2nd December	HW4	TBA	TBA	Schadrack Niyibizi
13	Friday, 8th December	Autoencoders and VAEs (GANs)	TBA	TBA	Harshith Arun Kumar & Emmanuel Ndayisaba

Assignments

∑ Ongoing, ∏ Upcoming

Assignment	Release Date (EST)	Due Date (EST)	Related Materials / Links
HW1P1	Wednesday, 6th Sept, 12:00 AM	Early Submission: 14th Sept, 11:59 PM	Autolab
HW1P1	Wednesday, 6th Sept, 12:00 AM	Final: 23rd Sept, 11:59 PM	Autolab
HW1P2	Wednesday, 6th Sept, 12:00 AM	Early Submission: 14th Sept, 11:59 PM	Kaggle, Writeup (pdf), Starter Notebook
HW1P2	Wednesday, 6th Sept, 12:00 AM	Final: 24th Sept, 11:59 PM	Kaggle, Writeup (pdf), Starter Notebook
HW1 Bonus	Wednesday, 6th Sept, 12:00 AM	Saturday, 14th Oct, 11:59 PM	Autolab
HW1 Autograd	Wednesday, 6th Sept, 12:00 AM	Saturday, 21st Oct, 11:59 PM	Autolab
HW2P1	Monday, 25th Sept, 12:00 AM	Early Submission: 5th Oct, 11:59 PM	Autolab
HW2P1	Monday, 25th Sept, 12:00 AM	Final: 22nd Oct, 11:59 PM	Autolab
HW2P2	Monday, 25th Sept, 12:00 AM	Early Submission: 5th Oct, 11:59 PM	Kaggle-Classification Kaggle-Verification Writeup (pdf) Starter Notebook
HW2P2	Monday, 25th Sept, 12:00 AM	Final: 22nd Oct, 11:59 PM
HW2 Bonus	Monday, 25th Sept, 12:00 AM	Saturday, 4th Nov, 12:00 AM	Autolab
HW2 Autograd	Monday, 25th Sept, 12:00 AM	Saturday, 11th Nov, 12:00 AM	Autolab
HW3P1	Monday, 23rd Oct, 12:00 AM	Early Submission: 31st Oct, 11:59 PM	Autolab, Writeup (pdf), Handout (.tar)
HW3P1	Monday, 23rd Oct, 12:00 AM	Final: 9th Nov, 11:59 PM	Autolab, Writeup (pdf), Handout (.tar)
HW3P2	Monday, 23rd Oct, 12:00 AM	Early Submission: 31st Oct, 11:59 PM	Kaggle, Writeup (pdf), Starter Notebook, MCQ
HW3P2	Monday, 23rd Oct, 12:00 AM	Final: 9th Nov, 11:59 PM	Kaggle, Writeup (pdf), Starter Notebook, MCQ
HW3 Bonus	TBA	TBA
HW3 Autograd	TBA	TBA
HW4P1	Monday, 6th Nov, 12:00 AM	Early Submission: 16th Nov, 11:59 PM	Autolab, Writeup (pdf), Handout (.tar)
HW4P1	Monday, 6th Nov, 12:00 AM	Final: 2nd Dec, 11:59 PM	Autolab, Writeup (pdf), Handout (.tar)
HW4P2	Monday, 6th Nov, 12:00 AM	Early Submission: 16th Nov, 11:59 PM	Kaggle, Writeup (pdf), Starter Notebook
HW4P2	Monday, 6th Nov, 12:00 AM	Final: 2nd Dec, 11:59 PM	Kaggle, Writeup (pdf), Starter Notebook
HW4 Bonus	TBA	TBA
HW4 Autograd	TBA	TBA