| 8/15/2020 | Webpages for this course have been moved to this site. |
| 2019/09/30 | Course management has been moved to the Courseworks, and access has been provided to those students who are registered into the course. |
Common email e4040TAs@columbia.edu.
Office Hours (Mudd 1301); Arjun DCunha: see announcements in Courseworks; Aijia Gao: see announcements in Courseworks; Zhen Wang: see announcements in Courseworks; Chengrui Wu: see announcements in Courseworks; Huixiang Zhuang: see announcements in Courseworks; Deepak Ravishankar: see announcements in Courseworks;
Students will contribute to the course by:
HW(40%), (one) Exam(25%), Project(35%).
Late homeworks (assignments): A student is entitled to 4 late days without penalty. For all homeworks together, a student can divide those four days in any fashion needed. Examples: (i) Homework 3 is late 4 days, in which case no other homework can be late for any amount of time; (ii) Homework 1 is late 1 day, homework 2 is late 2 days, in which case the student still has one more late-day for future assignments. The unit of delay can not be divided into less than a full day (like hours). Requests for additional extensions will not be granted: if the budget of 4 days is blown, the student will be given 0 credit for homework(s) for which their submission is late.
Computational tools are essential for learning about, designing, and experimenting with deep learning models. At the highest level, deep learning developers use one of the deep learning frameworks to build and run models, which rely on a myriad of either generic or custom software libraries. Course E4040 will use a particular version of the TensorFlow framework. The underlying computing language is Python, and the course will rely on Python code/libraries and Jupyter Notebooks for developing and experimenting with code. The students will be asked to run the tools and deep learning models on their own (local) computers, as well as in the Google cloud. Versioning of tools is complex, and students need to follow the precise tool version instructions in order to have functional computing environments. The course relies on Conda and PIP for tool versioning.
Students will be asked to use their personal/local computers to run python code, jupyter notebooks, TensorFlow framework and deep learning models. This is possible both with computers which have graphics processing units (GPUs) and with computers which do not have GPUs, where the execution will be faster on computers with GPUs. The begining assignments in this course are runnable on single processor (non-GPU) computers. Later assignments and projects will require that students run them on machines with GPUs; in this case the assignments can be run on personal computers or on the Google Cloud GPU machines. We will provide coupons for Google cloud access. Clicking on this section leads to instructions how to install the required tools on personal computers.
There are many online Python tutorials. This section is linked to one such tutorial. For majority of this course, we request that students use python 3.6 (compatible with the prescribed version of Tensorflow 1.13) when working on programming assignments. In the shared Google cloud image, we only installed Python 3.6 version. Therefore, you should strive to use exactly the same version on your personal/local computers.
For more complex assignments, students will be able to run deep learning code on Google Compute Engine (GCE) virtual machine (VM) instances, with access to GPU resources. The instructors will provide coupons (codes) which will enable students to use Google cloud resources for free. This tutorial provides instructions how to setup a VM instance using the VM image created for the E4040 course.
TensorFlow is an open-source deep learning framework developed by Google. It is essential that you learn to program in TensorFlow for this course. Note that the 2019 version of the E4040 course uses version 1.13 of TensorFlow, whereas the most recent TensorFlow homepage may be refering to a more recent version (such as 2.0). For 2019 assignments, students have to use the course-prescribed versions of TensorFlow and Python. If students wish to experiment with other versions of tools, they are advised to create additional (conda) virtual environments to prevent conflicts with tool versions used for the assignments.
The GCP VM instances which we use are Linux-based. If you haven't used Linux system before, this tutorial offers hands-on instructions to basic Linux commands that you will be using throughout the course.
Your homework assignments will be based on Github classroom repositories ocassionally we use Bitbucket as well), which can be manipulated by Git commands or visual clients such as GitKraken. If you don't know what Git is, it is strongly recommended that you read the first 3 chapters of this website. Additional instructions will be provided for use of the assignment-related github directories.
Colab provides a free cloud service based on Jupyter Notebooks. It supports free GPUs with popular deep learning libraries with an execution time limit. It is a good platform for developing and testing the deep learning code, which can avoid the issues of tool installation. However, it has limitations. We do not recommend that you do the assignments on Colab since everything except Jupyter Notebooks would be automatically deleted every 12 hours. Moreover, we request that you first set up your own local computing environment and google cloud resources (GCE) since both are a necessary skill for a practicing deep learning professional.
Solutions to some common problems
ECBM E4040 Neural Networks and Deep Learning, 2019.
Columbia University