Deep Learning – SP24
Notes, links, and resources for the class
Description
This course concerns the latest techniques in deep learning and representation learning, focusing on
supervised and unsupervised deep learning, embedding methods, metric learning, convolutional nets
and recurrent nets, with applications to computer vision, natural language understanding, and speech
recognition. The prerequisites include DS-GA 1001 Intro to Data Science or a graduate-level machine
learning course.
Links
● Course public folder: bit.ly/NYUDLSP24.
● Spring 2020 class website here. Spring 2021 here. Fall 2022 here.
● Campuswire forum is available here. You’ve been enrolled with your NetID@nyu.edu email.
Contact the TA if this does not work for you.
● Notebooks available from these repos: DLFL22, DLSP21, DLSP20.
Google Calendar
This calendar contains events to lectures, practica, and office hours. Only available for NYU students
with an NYU Google account (click switch accounts if the link doesn’t work). 🔗 Link.
People
Instructors
Yann LeCun <yann@cs.nyu.edu>
Alfredo Canziani <canziani@nyu.edu>
Mengye Ren <mengye@cs.nyu.edu>
Teaching Assistant (TA)
1. Jiachen Zhu <jz3224@nyu.edu>
Graders
1. Simran Makariye <sdm8499@nyu.edu>
2. Sriharsha Gaddipati <sg7372@nyu.edu>
3. Yaswanth Orru <yko207@nyu.edu>
4. Chaitanya Agarwal <ca2719@nyu.edu>
Logistics
Accessing previous lectures
You can find the previous offering’s recording in these YouTube playlists SP20, SP21, and FL22.
Syllabus
TBD, but as a reference you can use the previous syllabus on the website.
Grading, undergraduates and masters (default)
30% Homework – theoretical component
30% Homework – practical component
30% Final competition
10% Class participation
5% Extra credit (not required)
Grading, doctorate candidates (REQUIRES APPROVAL)
60% Individual deep learning research
30% Homework – theoretical component
10% Class participation
Deadlines schedule
This may be subject to change. We will have homework every two weeks, and then after the last
homework and until the end of the semester, we will hold the final competition.
Release date Due date
Homework 1 Thu 25
Homework 2
Homework 3
Homework 4
Final Competition
Collaboration policy
In this course, collaboration is encouraged but all the work that you submit MUST BE YOUR OWN. That
is, the writing and code must be your own. To avoid plagiarising, you shouldn't be looking at someone
else's solution before or while you write down your own. If someone is caught by the graders, there will
not be any second chances given, and the student will be directly reported to the Academic Integrity
office. Consequences include failure in the course, suspension or expulsion from the University.
How to fail this course
Not submitting your assignment is a perfect strategy to fail this course. This includes submitting
someone else’s assignment instead of your own. Check below for late submission policy.
Compute resources
TBD.
Overleaf
NYU has free access to Overleaf for all faculty and students at https://www.overleaf.com/edu/nyu.
Office hours
● Jiachen Zhu: Thus 13:30 Room 350, 60 5th Ave
● Mengye Ren: Wed 3pm Room 508, 60 5th Ave (second half of the semester)
There might be changes to the schedule in some weeks, so keep an eye on the class calendar. If nobody
comes to the office hour within the first 5 minutes, unless you let us know you’re planning to come late
(email the TA), the office hours will be called off.
Office hours with Alfredo Canziani happen right after class. Office hours with Yann LeCun are on-demand
only and subject to availability. You should ask him questions during normal class hours.
You also find us on Campuswire. Your questions are likely to be others’ questions as well. So, it’s nice to
have them all in one place. Also, answering others’ questions will give you extra bonus points.
Communication
All communication should happen through Campuswire. WE WILL NOT RESPOND TO EMAILS unless we
tell you so. Throughout the class, please feel free to ask any questions or raise any concerns on
Campuswire, in public posts or in private chat with any of us.
Class components
Lectures and practicums
14 lectures and practicums. We may also host invited speakers.
Homework
60% of the grade
There will be three homework assignments due on Sunday of the following week (you have roughly 10
days to solve it). We discourage working during the weekend since we won’t be able to answer your
questions. More details will be added here as we progress through the semester. Every extra day (max 4
days) will cost 10% of the grade. If you have any special circumstances, please reach out to the TA at
least 1 day before the submission deadline.
Each homework will consist of two components – theory and practice. The theory will include some
questions about understanding the mathematics behind the material explained in class, and the
practical part will be about implementing the covered concepts. Each component contributes 30% of the
final grade, so the total contribution of the homework will be 60%. There will be no midterm.
Homework cannot be done the night before the deadline, and to encourage you to start early we will
not respond to campuswire questions on Sat or Sun.
Final competition
30% of the grade
The details of the competitions will be announced later. For the final competition, you will form teams
of about 3 people to work on a problem of our choice. Groups will then submit their solutions, which we
will test and rank by their performance. You will be graded depending on how high your team is in the
ranking.
Class participation
10% of the grade
We strongly encourage you to actively participate in the class and the discussions on Campuswire. We
all learn better if we actively discuss the material, and to encourage you to ask and answer questions
during lectures and on Campuswire, we count your activity as 10% of the grade.
To be precise, we award:
● 6% grade for in-class interactions, questions asked and responses.
○ In particular, we’ll use this form, reachable also at bit.ly/NYUDLSP24QA, to log
instructor-approved interactions.
● 4% grade for Campuswire: Up to 1% for each question, and up to 2% for responded questions,
with a total capped at 4%.
● Note: Not any question/response is awarded points. Questions should be thoughtful, responses
should be exhaustive. Negative points for misleading, incomplete, or plagiarized contributions.
Extra credit
Additional 5%
We are committed to keeping this course open and inclusive. We want you to contribute to making it
better so that anyone who has access to the internet can access high-quality course materials. Your
contributions can earn you up to 5% extra credit.
Possible contributions:
● Fixing typos or bugs in notebooks or notes
● Adding or improving the course material
● Helping with translation (we don’t award extra credit for this as it’s hard for us to verify your
work). Refer to GitHub wiki for some examples if you want to contribute.
When you find something you want to do, please put your plan into a spreadsheet here (add new link),
to make sure multiple people don’t do the same thing.
When sending pull requests to the course repo, please include [DLSP24] in the title of the pull request.
Doctorate Candidates grading
Doctoral candidates with previous knowledge of Deep Learning and neural network frameworks
(PyTorch, TensorFlow) can choose to substitute the practical part of the homework and the final
competition with their own research and join PhD track grading. PhD track students still must complete
the theoretical part of the homework.
In order to be considered for PhD track, you need to:
1. Drop by Alfredo’s office (510 @CDS) no later than Fri 26 Jan, and make sure you understand
what consequences choosing the PhD track has.
2. Send us a 2-page proposal (ref included) with abstract, intro, and baselines and 3 minutes
presentation video with up to 6 slides including the title describing your plan for this semester’s
research. Here’s a tutorial on preparing presentations. Research should ideally be related to
deep learning. It may be some sort of application of deep learning to what you’re doing.
3. If the proposal is approved, you won’t have to do the practical part of the homework and the
final competition. As a final deliverable, by the final competition due date PhD Track students
must submit a 4-page (ref included) summary of the work, a 3-minute presentation, and a
GitHub link.
Reading materials
Here we will post useful materials to read to better understand the material in class.
Generally useful stuff
● Mathematical background:
○ YouTube lectures – 3blue1brown Essence of Linear Algebra
○ Mathematics for Machine Learning book from Deisenroth (the first 6 chapters are quite
useful).
● Deep Learning
○ Deep Learning Book by Ian Goodfellow, Yoshua Bengio and Aaron Courville - General
overview of a lot of topics in the field.
● Machine Learning
○ https://www.goodreads.com/book/show/85020.Pattern_Classification
Per Lecture Materials
Lecture Reading Materials
Backpropagation Mathematics for Machine Learning Chapter 5
Optionally - Deep Learning Book Section 6.5
Energy models http://yann.lecun.com/exdb/publis/pdf/lecun-
06.pdf
Recordings
1. P01 Course introduction and inference
2. L01 Classification and backprop
3. P02 PyTorch 5-step training, classification and regression notebooks
4. L02 History, early models, backpropagation, and weight sharing
