Contents

Course Details

Course Code COMP9444
Course Title Neural Networks and Deep Learning
Convenor Alan Blair
Admin TBA
Classes Tuesday 2-4pm and Wednesday 6-8pm (weeks 1-5, 7-10)
Consultations TBA
Units of Credit 6
Ed Website https://edstem.org/au/courses/7036/
WebCMS http://www.cse.unsw.edu.au/~cs9444/21T3/
Handbook Entry http://www.handbook.unsw.edu.au/postgraduate/courses/current/COMP9444.html

Course Summary

This course aims to introduce students to the main topics and methods in the field of neural networks and deep learning, ranging from traditional neural network models to the latest research and applications of deep learning.

Topics chosen from: perceptrons, feedforward neural networks, backpropagation, deep convolutional networks for image processing; geometric analysis of trained neural networks; recurrent networks, language processing, semantic analysis, long short term memory; Hopfield networks, restricted Boltzmann machines and autoencoders, generative adversarial networks; deep reinforcement learning; designing successful applications of neural networks; recent developments in neural networks and deep learning.

Student Learning Outcomes

After completing COMP9444, students should

  • understand aspects of the social, intellectual, and neurobiological context of neural networks and deep learning
  • have an understanding of a variety of NN and DL techniques, including the Planned Topics listed below
  • be able to analyse a problem for neural network solution in terms of these techniques
  • have an awareness of the computational theory underlying the various methods
  • have a working knowledge of one or more neural network simulation packages, and be able to use them to perform a range of computational tasks
  • have experience in programming neural network and deep learning applications
  • gain exposure to research techniques in neural networks, deep learning and cognitive science: some topics will be based on research papers and monographs, to which references will be given in the course notes

Textbook

The textbook for this course is:

Deep Learning
By Ian Goodfellow, Yoshua Bengio and Aaron Courville
MIT Press

http://www.deeplearningbook.org
https://mitpress.mit.edu/books/deep-learning

Assumed Knowledge

The course will assume knowledge of the following mathematical topics:

  • Linear Algebra (2.1-2.8)
  • Probability (3.1-3.14)
  • Calculus and Chain Rule (6.5.2)

Students should study the relevant sections of the textbook (shown in brackets) and, if necessary, try to revise these topics on their own during the first few weeks of the course.

Planned Topics

The planned topics for this course are:

Week Topic Textbook
wk1 Tue Neuroanatomy and Perceptrons (1.2, 9.10)
wk1 Wed Multi Layer Networks and Backpropagation (4.3)
wk2 Tue Probability, Generalisation and Overfitting (3.1-14, 5.1-6, 7.11-12)
wk2 Wed Cross Entropy, Softmax, Weight Decay, Momentum (6.1-5)
wk3 Tue PyTorch
wk3 Wed Hidden Unit Dynamics
(8.2-3)
wk4 Tue Convolutional Networks (7.9, 9.1-5)
wk4 Wed Image Processing (7.4, 8.4, 8.7.1)
wk5 Tue Recurrent Networks (10.2)
wk5 Wed Long Short Term Memory (10.7, 10.10)
wk6 (Flexibility Week)
wk7 Tue Word Vectors (12.4)
wk7 Wed Language Processing
(10.4)
wk8 Tue Reinforcement Learning
(12.5.1.1)
wk8 Wed Deep Reinforcement Learning
(18.1, 20.9)
wk9 Tue Hopfield Networks & Boltzmann Machines
(16.7, 17.4, 18.2, 20.1-4)
wk9 Wed Autoencoders
(14.1-5, 20.10.3)
wk10 Tue Generative Adversarial Networks
(20.10.4)
wk10 Wed Review


The relevant sections of the textbook are shown in brackets.
The textbook may be supplemented with additional materials for some topics.

Teaching Strategies

Due to social distancing restrictions, the course will be delivered online. Students are required to watch pre-recorded lecture videos before each session. The scheduled class time will take the form of an interactive video chat session, and will be used to briefly summarise the content, discuss recent developments, and to answer any questions that you may have about each topic.

We are planning to deliver all the course materials through the course Ed page . This includes course content (text, images and embedded videos) as well as exercises, quizzes, coding exercises (using jupyter notebooks), recordings of online sessions, assignments and on-line discussion.

WebCMS will be used only for group formation for Assignment 2.

The course materials, embedded videos and online sessions introduce you to the various concepts and methods, provide motivating examples to help you understand them, and demonstrate skills and processes. You should not expect to understand the material completely simply by watching the embedded videos. You should also:

  • review the lecture material before and after the scheduled class
  • discuss the material with fellow students if possible
  • read up on the topics covered in each lecture
  • complete relevant quizzes, exercises, coding exercises and assignments
  • consider exploring the topic on-line by writing and running your own programs
  • ask questions in an online consultation session, if you still don't understand the material

Assessment

The assessment for this course will be:

Assignment 1 30%
Assignment 2 30%
Final Exam 40%

The assignments will involve writing code in PyTorch. Please try to install PyTorch on your own laptop. These are the versions of modules currently installed on the CSE lab machines.

python3 3.7.3
torch 1.8.1
numpy 1.16.2
sklearn 0.20.2


Student Conduct

The Student Code of Conduct ( Information , Policy ) sets out what the University expects from students as members of the UNSW community. As well as the learning, teaching and research environment, the University aims to provide an environment that enables students to achieve their full potential and to provide an experience consistent with the University's values and guiding principles. A condition of enrolment is that students inform themselves of the University's rules and policies affecting them, and conduct themselves accordingly.

In particular, students have the responsibility to observe standards of equity and respect in dealing with every member of the University community. This applies to all activities on UNSW premises and all external activities related to study and research. This includes behaviour in person as well as behaviour on social media, for example Facebook groups set up for the purpose of discussing UNSW courses or course work. Behaviour that is considered in breach of the Student Code Policy as discriminatory, sexually inappropriate, bullying, harassing, invading another's privacy or causing any person to fear for their personal safety is serious misconduct and can lead to severe penalties, including suspension or exclusion from UNSW.

If you have any concerns, you may raise them with your lecturer, or approach the School Ethics Officer , Grievance Officer , or one of the student representatives.

Plagiarism is defined as using the words or ideas of others and presenting them as your own. UNSW and CSE treat plagiarism as academic misconduct, which means that it carries penalties as severe as being excluded from further study at UNSW. There are several on-line sources to help you understand what plagiarism is and how it is dealt with at UNSW:

Make sure that you read and understand these. Ignorance is not accepted as an excuse for plagiarism. In particular, you are also responsible that your assignment files are not accessible by anyone but you by setting the correct permissions in your CSE directory and code repository, if using. Note also that plagiarism includes paying or asking another person to do a piece of work for you and then submitting it as your own work.

UNSW has an ongoing commitment to fostering a culture of learning informed by academic integrity. All UNSW staff and students have a responsibility to adhere to this principle of academic integrity. Plagiarism undermines academic integrity and is not tolerated at UNSW. Plagiarism at UNSW is defined as using the words or ideas of others and passing them off as your own.

If you haven't done so yet, please take the time to read the full text of

The pages below describe the policies and procedures in more detail:

You should also read the following page which describes your rights and responsibilities in the CSE context:

Course Evaluation and Development

Every term, student feedback is requested in a survey using UNSW's myExperience online survey system where the feedback will be used to make improvements to the course. Students are also encouraged to provide informal feedback during the session, and to let course staff know of any problems as soon as they arise. Suggestions will be listened to openly, positively, constructively, and thankfully, and every reasonable effort will be made to address them.

Inspired by feedback from previous offerings, we have endeavoured to make the course content more accessible by expanding the dot points from the lecture slides into full paragraphs with images and clickable references, and editing the lecture recordings into short videos embedded below the corresponding text. We hope these changes will help to make this course a rewarding and enjoyable experience.

Resource created Tuesday 07 September 2021, 01:04:13 PM, last modified Tuesday 14 September 2021, 07:12:24 AM.


Back to top

COMP9444 21T3 (Neural Networks and Deep Learning) is powered by WebCMS3
CRICOS Provider No. 00098G