Week 
Content 
0 
Introduction and math revision
 Suggested readings 
 Other suggested video material 

1 
Feedforward neural network
 Other suggested readings 
 Other paper exploiting the inspiration from biological neural networks to develop new artificial neural networks:
 Other suggested video material 
 Videos from
Andrew Ng's Coursera course, on neural networks:
[1]
[2]
[3]
[4]
[5]
[6]
[7]

2 
Training neural networks
 Other suggested readings 
 Papers discussing tricks for training neural networks:
 Papers exploring optimization methods for training neural networks:
 General notes on optimization on large data sets (excellent summary of many methods):
 Other suggested video material 
 Videos from
Andrew Ng's Coursera course, on training
neural networks:
[1]
[2]
[3]
[4]
[5]
[6]
[7]

3 
Conditional random fields
 Other suggested readings 
 Other suggested video material 

4 
Training CRFs
 Other suggested readings 
 Papers discussing nonlinear conditional random fields:
 Precursor paper on conditional random fields:
 Papers on alternative training methods for conditional random fields:
 Paper describing different methods for taking into account the testtime error function during training:
 Other suggested video material 

5 
Restricted Boltzmann machine
 Other suggested readings 
 Other paper on other approaches for training models with intractable normalization constants:
 Papers on extensions of the restricted Boltzmann machine:
 Papers on more advanced sampling methods:
 Other suggested video material 

6 
Autoencoders
 Other suggested readings 
 Theoretical paper demonstrating the optimality results for the linear autoencoder:
 Papers on different extensions of the autoencoder:

7 
Deep learning
 Other suggested readings 
 Paper on deep belief networks (DBNs):
 Paper on deep autoencoders:
 Detailed paper on deep learning:
 Experimental evaluations of deep learning methods:
 Papers on alternative approaches for unsupervised pretraining of deep networks:
 Papers on dropout regularisation methods:
 Paper on another type of nonfeedfoward deep network:
 Other suggested video material 

8 
Sparse coding
 Other suggested readings
 Papers on other sparse representation models:
 Variants of sparse coding models:
 Online sparse coding algorithm:
 Method to accelerate inference in sparse coding model:
 Other suggested video material 

9 
Computer vision
 Other suggested readings 
 Experimental evaluation of good practices in using convolutional networks:
 Convolutional version of the restricted Boltzmann machine:
 Summary of the neurophysiology of the visual cortex:
 Analysis of random filters:
 Different applications to computer vision of neural networks:
 Other convolutional systems:
 Other suggested video material 

10 
Natural language processing
 Other suggested readings 
 Papers on language modeling with neural networks:
 Modeling documents with neural networks:
 Other papers on word tagging with neural networks:
 Other efficient training algorithms for text data:
 Papers on learning word vector representations:
 Papers on recursive neural network:

Parsing Natural Scenes and Natural Language with Recursive Neural Networks
by Richard Socher,
Jeffrey Pennington,
Eric Huang,
Andrew Ng and
Christopher Manning
[video]

SemiSupervised Recursive Autoencoders for Predicting Sentiment Distributions
by Richard Socher,
Jeffrey Pennington,
Eric Huang,
Andrew Ng and
Christopher Manning

Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection
by Richard Socher,
Eric Huang,
Jeffrey Pennington,
Andrew Ng and
Christopher Manning

Semantic Compositionality through Recursive MatrixVector Spaces
by Richard Socher,
Brody Huval,
Christopher Manning
and Andrew Ng
 Other suggested video material 
