DeepSense: a unified deep learning framework for time-series mobile sensing data processing

May 17, 2017 ~ Adrian Colyer ~ 5 Comments

DeepSense: a unified deep learning framework for time-series mobile sensing data processing Yao et al., WWW'17 DeepSense is a deep learning framework that runs on mobile devices, and can be used for regression and classification tasks based on data coming from mobile sensors (e.g., motion sensors). An example of a classification task is heterogeneous human ... Continue Reading

Learning to act by predicting the future

May 12, 2017November 11, 2019 ~ Adrian Colyer ~ 5 Comments

Learning to act by predicting the future Dosovitskiy & Koltun, ICLR'17 The Direct Future Prediction (DFP) model won the 'Full Deathmatch' track of the Visual Doom AI Competion in 2016. The competition pits agents against each other, with their performance measured by how many 'frags' they get. (A frag is a kill, apparently - not ... Continue Reading

Understanding deep learning requires re-thinking generalization

May 11, 2017November 11, 2019 ~ Adrian Colyer ~ 33 Comments

Understanding deep learning requires re-thinking generalization Zhang et al., ICLR'17 This paper has a wonderful combination of properties: the results are easy to understand, somewhat surprising, and then leave you pondering over what it all might mean for a long while afterwards! The question the authors set out to answer was this: What is it ... Continue Reading

Neural architecture search with reinforcement learning

May 10, 2017November 11, 2019 ~ Adrian Colyer ~ 7 Comments

Neural architecture search with reinforcement learning Zoph & Le, ICLR'17 Earlier this year we looked at 'Large scale evolution of image classifiers' which used an evolutionary algorithm to guide a search for the best network architectures. In today's paper, Zoph & Le also demonstrate that learning network architectures (and also in their case recurrent cell ... Continue Reading

Semi-supervised knowledge transfer for deep learning from private training data

May 9, 2017November 11, 2019 ~ Adrian Colyer ~ 1 Comment

Semi-supervised knowledge transfer for deep learning from private training data Papernot et al., ICLR'17 How can you build deep learning models that are trained on sensitive data (e.g., concerning individuals), and be confident to deploy those models in the wild knowing that they won't leak any information about the individuals in the training set? As ... Continue Reading

DeepCoder: Learning to write programs

March 29, 2017November 11, 2019 ~ Adrian Colyer ~ 1 Comment

DeepCoder: Learning to write programs Balog et al., ICLR 2017 I'm mostly trying to wait until the ICLR conference itself before diving into the papers to be presented there, but this particular paper follows nicely on from yesterday, so I've decided to bring it forward. In 'Large scale evolution of image classifiers' we saw how ... Continue Reading

Large-scale evolution of image classifiers

March 28, 2017November 11, 2019 ~ Adrian Colyer ~ 6 Comments

Large-scale evolution of image classifiers Real et al., 2017 I'm sure you noticed the bewildering array of network architectures in use when we looked at some of the top convolution neural network papers of the last few years last week (Part 1, Part2, Part 3). With sufficient training data, these networks can achieve amazing feats, ... Continue Reading

A miscellany of fun deep learning papers

March 24, 2017November 11, 2019 ~ Adrian Colyer ~ 2 Comments

To round out the week, I thought I'd take a selection of fun papers from the 'More papers from 2016' section of top 100 awesome deep learning papers list. Colorful image colorization, Zhang et al., 2016 Texture networks: feed-forward synthesis of textures and stylized images Generative visual manipulation on the natural image manifold, Zhu et ... Continue Reading

Recurrent Neural Network models

March 23, 2017November 11, 2019 ~ Adrian Colyer ~ 2 Comments

Today we're pressing on with the top 100 awesome deep learning papers list, and the section on recurrent neural networks (RNNs). This contains only four papers (joy!), and even better we've covered two of them previously (Neural Turing Machines and Memory Networks, the links below are to the write-ups). That leaves up with only two ... Continue Reading

Convolution neural networks, Part 3

March 22, 2017November 11, 2019 ~ Adrian Colyer ~ 4 Comments

Today we're looking at the final four papers from the 'convolutional neural networks' section of the 'top 100 awesome deep learning papers' list. Deep residual learning for image recognition, He et al., 2016 Identity mappings in deep residual networks, He et al., 2016 Inception-v4, inception-resnet and the impact of residual connections or learning, Szegedy et ... Continue Reading

the morning paper

a random walk through Computer Science research, by Adrian Colyer
Made delightfully fast by strattic

Deep Learning