Unicode Machine Learning Summer Course 2021 / Summer 2021

Course Description

The goal of this class is to train you in becoming a deep learning practitioner along with a working understanding of the mathematical intuition underlying the algorithms used in commonplace applications, particularly deep learning. This course, based almost entirely on the book ‘Dive into Deep Learning’, represents our attempt to make deep learning approachable, teaching you the concepts, the context, and the code. During the class, you will do data analyses and build deep neural networks with existing software, and write your own programs to implement and extend key techniques. You will also have to write reports about your analyses to provide non-technical readers an understanding of the conclusions you have drawn.

Course Staff

Sponsors

We’re thankful to Google AI Research India for their support in organizing the course.

Course Mechanics

A numeric grade will be assigned to each student based on an absolute grading metric. A (tentative) midterm exam 30%, class participation will be 20%, and the final project 50%. You will also be graded by your own project partners, and these results will be kept confidential.

While this grade is largely irrelevant to your learning experience, we believe it is important to hold you accountable to attend the recitations and we will be listing the top performers as well as best projects on our website as an indication of the effort the corresponding student(s)/team(s) invested into this course.

We will have a separate mention of the most helpful student(s) that contribute to answering others’ queries on CampusWire (course invites will be shared prior to the class–reach out via email if you have not heard back by the first lecture) as well as driving the discussion during the lectures.

Lectures

Lectures will be held once per week, last 2 hours, and include a 10-minute break. They will be primarily discussion-oriented, and it is strongly recommended that you watch the lectures available on the ‘Dive into Deep Learning’ website prior to each week’s lecture. This is simply due to a time-crunch since we are compressing a 12-week course into an 8-10 week duration. This will undoubtedly lead to the omission of several topics despite the ambitious course schedule, which is an idealized version of what we might cover. The pace of teaching will be adapted to match the average comfort level in the class tracked via interviews with students as well as anonymized in-class polls.

You are responsible for all material covered in lecture, whether or not it is in the textbook. If you are unable to attend a particular lecture, arrange to get notes from a classmate. If you have problems coming to lecture, please email the Course Staff unicode [dot] mlsc [at] gmail [dot] com. We will be recording the lectures but we cannot commit to releasing recordings within a week of each lecture (although we will attempt to do so). Therefore, it is highly recommended to attend lectures synchronously!

We will do our best to accommodate changes in the lecture schedule should a significant majority of the class feel the need to do so.

Prerequisites

Although there will be sessions introducing many machine learning concepts from the ground up, we will by and large assume that you have a basic familiarity with several types of math: calculus, matrix and vector algebra, and basic probability. You don’t need to be an expert in all these areas, but you will need to be conversant in each, and to understand:

• Basic probability and statistics - For example, we assume you know how to find the mean and variance of a set of data, and that you understand basic notions such as conditional probabilities and Bayes rule. During the class, you might be asked to calculate the probability of a data set with respect to a given probability distribution.

• Basic calculus - For example, we rely on you being able to calculate derivatives and partial derivatives. During the class we will sometimes calculate derivatives (gradients) of functions involving several variables.

• Linear algebra - For example, we assume you know how to multiply vectors and matrices, and that you understand matrix inversion.

Homework

We have decided not to keep weekly homeworks for the duration of this course. This is to allow the teaching assistants the time and bandwidth to focus on delivering quality recitations without being bogged down by grading duties.

Exams

There will be a take-home midterm exam (30% of your final grade). You will have one day to work on the midterm. There will also be a final project (50%), due at the end of the course. The logistics for the project are explained separately in a subsequent section. The exams may require you to use any material already covered in the readings, lectures or assignments. All exams will be cumulative.

Exams must also be submitted through the corresponding Google Form (TBA). Communicating your results to others is as important as getting good results in the first place. The exam will require you to write about some data analysis and what you learned from it; this writing is part of the assignment and will be graded. As always, raw computer output and Python code is not acceptable; your document must be humanly readable. You should submit a Jupyter Notebook file, integrating text, figures and Python code; submit both your output PDF file (*.pdf) AFTER having run all the cells, and the source (*.ipynb) that we can rerun WITHOUT installing additional packages than are recommended for the exam. If you need to use additional packages in Python please first post a question on CampusWire to confirm that you need the package for the exam, then ensure they are installable via the Python Package Index (pip install <package_name>) and finally include the requirements.txt file along with your exam. Microsoft Word files get an automatic grade of 0, with no feedback.

Grading

The purpose of this course is to help you learn machine learning. The purpose of the project and midterm is to help you learn by giving you structured opportunities for practice. The purpose of grading is primarily to give you feedback, distinguishing what you did well on from what you should work on improving.

If you believe that particular submission has been incorrectly graded, tell us as soon as possible. As a final word of advice, “what is the least amount of work I need to do in order to get the grade I want?” is a much worse way to approach higher education than “how can I learn the most from this class and from my teachers?”.

Solutions

Solutions for the exam will be available, after their due date. Do not share them with anyone, even after the course has ended. Please also refrain from posting your own solutions publicly (although feel free to post about your projects and the topics that you have learnt about in class, just not the specific questions and answers).

Interviews

To help the instructors get a better sense of how the class is going, every week (after the first week of classes), four students will be selected at random, and will meet with me for 10 - 15 minutes each, to explain their work and to answer questions about it. You may be selected on multiple weeks, if that’s how the random numbers come up. This is not a punishment, but a way to see whether the problem sets are really serving their goal of helping you learn the course material; being selected will not hurt your grade in any way (and might even help).

Recitations

We will have one recitation per week lasting one hour total that will introduce you to some of the topics that we did not have enough time to cover during the main lectures. These topics will be left to the discretion of the recitation leader(s) but will be announced in the week prior to the lecture. This will also allow you to step beyond the ‘Dive into Deep Learning’ textbook, grow familiar with practical problems in the field of machine learning, and start to explore research directions of interest to you.

Office Hours

Office Hours are typically unstructured sessions where the Course Staff will be available to help you with questions about the topics covered in the course, additional follow-ups from the Recitations or the Course Lectures. We will update our availability in the format provided below, so there will be multiple sessions a week where you can ask your questions in a less public setting. These sessions will not be recorded.

• Day
  Time
• Monday
  6:00 - 7:00
• Tuesday
  4:00 - 5:00
• Wednesday
  11:00 - 12: 00
• Thursday
  6:00 - 7:00

If you cannot make the regular office hours, or have concerns you’d rather discuss privately, please email me about making an appointment. This will not be offered on a regular basis, unless you have a strong reason to request my time.

Final Project

There will be a final project for this course that will determine 50% of your final grade. Ideally, you want to start working on the project early on and ask us for feedback on topic selection and research questions that may be a good fit for the class requirements. The goal of the project is investigating a specific application of machine learning (including deep learning).

Logistics

The project should be carried out in groups of three. If you cannot make a team yourself (feel free to use CampusWire/Discord to form teams), please write to us so we may connect you with other candidates looking to form a team.

Proposal

The proposal must be one page long and it is due in the third week of the course. It should include the following information (each point will be evaluated separately):

• A description of a specific question that you want to explore. The question can be experimental, i.e. whether a certain technique works for a particular application, or theoretical, whether a certain theoretical tool can be used to analyze a data-analysis method. Be as concrete as you can.

• Context for your topic, including relevant bibliographic references.

• An outline of what you plan to do and a precise justification of how it relates to the question that you are studying. For experimental projects, this includes a description of the dataset you plan to use.

We will provide feedback on your proposal and also allocate separate office hours sessions to discuss projects.

Project report

The project should be written in Latex and be no more than 5 pages (not including references). It should include the following sections, which will be evaluated separately:

• Introduction: What question have you been studying? Why should anyone care?

• State of the art: Provide context, with relevant bibliographic references.

• Methodology: How did you tackle the question? How did you modify existing methods? What datasets did you use? What theoretical tools did you apply? If you deviated from your original plan, explain why.

• Results: What results did you obtain? Do they make sense? Provide a thorough analysis. Negative results are completely fine (they can be very valuable!).

• Discussion: What did you find out? Do the results suggest any other interesting questions to explore?

Presentation

Projects will be presented in class. Presentations will last 5 minutes (strict limit) and have a maximum of 6 slides.

Possible topics

You may find some topics of interest among the list of projects on the Dive into Deep Learning website.

Textbook

The primary textbook for the course will be the textbook Dive Into Deep Learning whose PDF copy is available on the website. In general, we will reuse the slides, and follow the discussion in the past iterations of Alex Smola’s Spring 2019 Course at Berkeley.

Similar Courses

• Tamara Broderick’s Machine Learning Course at MIT
• Alfredo Canziani and Yann LeCun’s Deep Learning Course at NYU (although this has more recent, advanced topics)

Collaboration, Cheating and Plagiarism

Everything you turn in for a grade must be your own work, or a clearly acknowledged borrowing from an approved source; this includes all mathematical derivations, computer code and output, figures, and text. Any use of permitted sources must be clearly acknowledged in your work, with citations letting the reader verify your source. You are free to consult the textbook and recommended class texts, lecture slides and demos, any resources provided through the class website, books and papers in the library, or online resources, though again, all use of these sources must be acknowledged in your work.

In general, you are free to discuss homework with other students in the class, though not to share work; such conversations must be acknowledged in your assignments. You may not discuss the content of assignments with anyone other than current students or the instructors until after the assignments are due. (Exceptions may be made, with prior permission, for approved tutors.) You are, naturally, free to complain, in general terms, about any aspect of the course, to whomever you like.

During the take-home exams, you are not allowed to discuss the content of the exams with anyone other than the instructors; in particular, you may not discuss the content of the exam with other students in the course.

Any use of solutions provided for any assignment in this course in parallel iterations or from other related courses is strictly prohibited, both for homework and for exams. Do not copy the old solutions (in whole or in part), do not “consult” them, do not read them, do not ask your friend who took the course last year if they “happen to remember” or “can give you a hint”. Doing any of these things, or anything like these things, is cheating, it is easily detected cheating, and we will take strict action against those involved in such activities.

If you are unsure about what is or is not appropriate, please ask me before submitting anything; there will be no penalty for asking. If you do violate these policies but then think better of it, it is your responsibility to tell me as soon as possible to discuss how your mis-deeds might be rectified. Otherwise, violations of any sort will lead to severe, formal disciplinary action, including removal from the course and all future association with Unicode.

Accommodations for Students with Disabilities

For individualized academic adjustment based on a documented disability, contact the UMLSC Instructors unicode [dot] mlsc [at] gmail [dot] com

Python

Python is a free, open-source software package/programming language. You should have begun to learn it in the first/second year of engineering (if not before), and this class presumes that you have working knowledge of Python. Every assignment will require you to use it. No other form of computational work will be accepted. If you are not able to use Python, or do not have ready, reliable access to a computer on which you can do so, let us know at the start of the course.

At the top of this Colab Notebook are some separate resources for learning more about Numpy, Linear Algebra and some additional sample problems. Here is a solution for one of the problems, at the NYU AI School.

Other Iterations of the Class

Any use of solutions provided elsewhere is not only cheating, it is very easily detected cheating

A Note from the Course Instructor

It is quite insane for us to imagine successfully delivering a course based on cutting-edge research structured as a 10-14 week undertaking with a fraction of the resources and none of the financial support afforded by a large university. As such, we do not have any obligation to undertake such a massively time-consuming activity as teaching a full-fledged course would entail. Arguably, most of us might not have the expertise to do so either. The purpose of this labor is to take the first steps towards building a machine learning foundation and stimulate the growth of a healthy research community that is able to actively dissect the latest and greatest machine learning and deep learning advances from the literature. Most importantly, it is to foster the spirit of learning, exploration, and communication about computer science research among budding researchers that might not otherwise have access to such resources.

My course staff and I have some collective experience with community-building. It’s been 4 years since we started Unicode to teach students software development and with some luck and a lot of hard work from mentors and mentees, we are at a self-sustaining stage with almost no external input required to grow our community and achieve positive social impact. Our hope with this course is, equivalently, to create a lasting community of individuals, some of whom might continue to do meaningful research with Unicode Research, mentor others, and also serve as teaching assistants for future iterations.

Bearing all this in mind, I request you to be gentle in your communication and expectations of the teaching assistants and myself; we would be grateful if you are proactive with your questions, especially with pointing out if/where we may be incorrect with our explanations. Finally, do cut us some slack if exams are not graded promptly. As the instructor for a first-time course, I bear all the responsibility of conveying the correct information to you and any mistake on the part of the TAs is a product of my oversight. I commit to doing my level best to stimulate and monitor the class discussions in a safe and productive manner, but I request your proactive participation (and will duly reward it) throughout the course for your benefit as well as that of your classmates.

Code of Conduct

All participants—students, staff, and volunteers in our course—are required to act in accordance with this code of conduct during the event, on recordings made to be presented as part of the event, and on official communication channels, including social media. Course Staff will enforce this code, and we expect cooperation from all participants to help ensure a safe, productive, and welcoming environment for everybody.

The course commits itself to providing a workshop experience for all participants that is free from harassment, bullying, discrimination, intimidation, personal attacks, unnecessary disruption of talks or other events, retaliation, and behavior that interferes with another’s full participation or may create an inappropriate or toxic environment. This includes offensive comments related to gender, gender identity and expression, age, sexual orientation, medical condition, disability, physical appearance, race, ethnicity, religion (or lack thereof), or any other personal characteristics.

Participants asked by any member of the course staff to stop any such behavior are expected to comply immediately. If a participant engages in such behavior, the organizers, after careful review where all parties involved will have an opportunity to present their side, may take any action they deem appropriate, including: a formal or informal warning to the offender, expulsion from the workshop, barring from participation in future workshops, or reporting the incident to the offender’s local institution. A response of “just joking” will not be accepted; behavior can be harassing without an intent to offend. If action is taken, an appeals process will be made available.

If you have concerns related to your inclusion in the course, or observe someone else’s difficulties, or have any other concerns related to inclusion, please contact the course staff by email at unicode [dot] mlsc [at] gmail [dot] com. Reports made during the workshop will be responded to in less than 24 hours; those at other times, in less than two weeks. We are prepared and eager to help students feel safe for the duration of the course. Threats or acts of retaliation against individuals who report inappropriate conduct pursuant to this Policy or provide information in connection with a report by another individual will not be tolerated. If you believe that you have been retaliated against for such action, you may use the above procedures to report the pertinent facts. We gratefully accept feedback from everyone on policy and actions; please contact us at unicode [dot] mlsc [at] gmail [dot] com.

Sources

This page has been shamelessly copied over from Cosma Shalizi and Carlos Fernandez Granda’s course/project description pages. Our code of conduct has been adapted from that of the Learning Theory Alliance. The website theme is based on jekyll-course-website-template by kazemnejad.