Sequence to sequence learning with neural networks Sutskever et al. NIPS, 2014 Yesterday we looked at paragraph vectors which extend the distributed word vectors approach to learn a distributed representation of a sentence, paragraph, or document. Today's paper tackles what must be one of the sternest tests of all when it comes to assessing how … Continue reading Sequence to sequence learning with neural networks
Tag: Deep Learning
The deep learning subset of machine learning.
GeePS: Scalable deep learning on distributed GPUs with a GPU-specialized parameter server
GeePS: Scalable deep learning on distributed GPUs with a GPU-specialized parameter server - Cui et al. 2016 (EuroSys 2016) We know that deep learning is well suited to GPUs since it has inherent parallelism. But so far this has mostly been limited to either a single GPU (e.g. using Caffe) or to specially built distributed … Continue reading GeePS: Scalable deep learning on distributed GPUs with a GPU-specialized parameter server
ImageNet Classification with Deep Convolutional Neural Networks
ImageNet Classification with Deep Convolutional Neural Networks - Krizhevsky et al. 2012 Like the large-vocabulary speech recognition paper we looked at yesterday, today's paper has also been described as a landmark paper in the history of deep learning. It's also a surprisingly easy read! The ImageNet dataset contains over 15 million labeled high-resolution images of … Continue reading ImageNet Classification with Deep Convolutional Neural Networks
Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition
Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition - Dahl et al. 2011 The title may be a bit of a mouthful, but this paper is often cited as a watershed moment for deep learning and speech recognition. It represents the first application of deep neural networks for large vocabulary speech recognition (LVSR), and … Continue reading Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition
Deep Learning in Neural Networks: An Overview
Deep Learning in Neural Networks: An Overview - Schmidhuber 2014 What a wonderful treasure trove this paper is! Schmidhuber provides all the background you need to gain an overview of deep learning (as of 2014) and how we got there through the preceding decades. Starting from recent DL results, I tried to trace back the … Continue reading Deep Learning in Neural Networks: An Overview
Distributed TensorFlow with MPI
Distributed TensorFlow with MPI - Vishnu et al. 2016 A short early release paper to close out the week this week, which looks at how to support machine learning and data mining (MLDM) with Google's TensorFlow in a distributed setting. The paper also contains some good background on TensorFlow itself as well as MPI - … Continue reading Distributed TensorFlow with MPI
Strategic Dialogue Management via Deep Reinforcement Learning
Strategic Dialogue Management via Deep Reinforcement Learning - Cuayahuitl et al. 2015 If computers learning to play Atari arcade games by themselves isn't really your thing, perhaps you're more into board games? How about a Deep Reinforcement Learning system that learns how to trade effectively in Settlers of Catan! Again, we're not talking about a … Continue reading Strategic Dialogue Management via Deep Reinforcement Learning
Memory Networks
Memory Networks Weston et al. 2015 As with the Neural Turing Machine that we look at yesterday, this paper looks at extending machine learning models with a memory component. The Neural Turing Machine work was developed at Google by the DeepMind team, today's paper on Memory Networks was developed by the Facebook AI Research group. … Continue reading Memory Networks
Neural Turing Machines
Neural Turing Machines - Graves et al. 2014 (Google DeepMind) A Neural Turing Machine is a Neural Network extended with a working memory, which as we'll see, gives it very impressive learning abilities. A Neural Turing Machine (NTM) architecture contains two basic components: a neural network controller and a memory bank. Like most neural networks, … Continue reading Neural Turing Machines
Graying the Black Box: Understanding DQNs
Graying the Black Box: Understanding DQNs - Zahavy et al. 2016 It's hard to escape the excitement around deep learning these days. Over the last couple of days we looked at some of the lessons learned by Google's machine learning systems teams, including the need to develop ways of getting insights into the predictions made … Continue reading Graying the Black Box: Understanding DQNs
the morning paper 