Deep learning
Deep learning is part of a broader family of machine learning methods based on artificial neural networks with representation learning. Learning can be supervised, semi-supervised or unsupervised.
- Feed forward Neural networks.
- Gradient descent and the back propagation algorithm.
- Unit saturation
- aka the vanishing gradient problem, and ways to mitigate it.
- RelU Heuristics for avoiding bad local minima.
- Heuristics for faster training.
- Nestors accelerated gradient descent.
- Regularization. Dropout.
- Architectures
- Convolution / pooling layers
- LSTM
- GRU
- Encoder Decoder architectures
- Autoencoder (standard, sparse, denoising, contractive, etc), Variational Autoencoders
- Adversarial Generative Networks
- Autoencoder and DBM
- Attention and memory models, Dynamic memory networks
- Image segmentation
- Object detection
- Automatic image captioning
- Image generation with Generative adversarial networks
- Video to text with LSTM models.
- Attention models for computer vision tasks.
- Introduction to NLP and Vector Space Model of Semantics
- Word Vector Representations: Continuous Skip-Gram Model Continuous Bag-of- Words model (CBOW)
- Glove
- Evaluations and Applications in word similarity
- Analogy reasoning
- Named Entity Recognition, Opinion Mining using Recurrent Neural Networks
- Parsing and Sentiment Analysis using Recursive Neural Networks
- Sentence Classification using Convolutional Neural Networks
- Dialogue Generation with LSTMs
- Applications of Dynamic Memory Networks in NLP
- Recent Research in NLP using Deep Learning:
- Factoid Question Answering
- Similar question detection
- Dialogue topic tracking
- Neural Summarization
- Smart Reply