Why does deep and cheap learning work so well Lin & Tegmark 2016 Deep learning works remarkably well, and has helped dramatically improve the state-of-the-art in areas ranging from speech recognition, translation, and visual object recognition to drug discovery, genomics, and automatic game playing. However, it is still not fully understood why deep learning works … Continue reading Why does deep and cheap learning work so well?
Tag: Deep Learning
The deep learning subset of machine learning.
Texture networks: feed-forward synthesis of textures and stylized images
Texture Networks: Feed-forward synthesis of textures and stylized images Ulyanov et al., arXiv, March 2016 During the summer break I mostly stayed away from news feeds and twitter, which induces terrible FOMO (Fear Of Missing Out) to start with. What great research was published / discussed that I missed? Was there a major industry announcement … Continue reading Texture networks: feed-forward synthesis of textures and stylized images
Mastering the game of Go with deep neural networks and tree search
Mastering the Game of Go with Deep Neural Networks and Tree Search Silver, Huang et al., Nature vol 529, 2016 Pretty much everyone has heard about AlphaGo’s tremendous Go playing success beating the European champion by 5 games to 0. In all the excitement at the time, less was written about how AlphaGo actually worked … Continue reading Mastering the game of Go with deep neural networks and tree search
Deep neural networks for YouTube recommendations
Deep Neural Networks for YouTube Recommendations Covington et al, RecSys '16 The lovely people at InfoQ have been very kind to The Morning Paper, producing beautiful looking "Quarterly Editions." Today's paper choice was first highlighted to me by InfoQ's very own Charles Humble. In it, Google describe how they overhauled the YouTube recommendation system using … Continue reading Deep neural networks for YouTube recommendations
End-to-end learning of semantic role labeling using recurrent neural networks
End-to-end learning of semantic role labeling using recurrent neural networks Zhou & Xu International joint conference on Natural Language Processing, 2015 Collobert’s 2011 paper that we looked at yesterday represented a turning point in NLP in which they achieved state of the art performance on part-of-speech tagging (POS), chunking, and named entity recognition (NER) using … Continue reading End-to-end learning of semantic role labeling using recurrent neural networks
Building end-to-end dialogue systems using generative hierarchical neural network models
Building end-to-end dialogue systems using generative hierarchical neural network models Serban et al. AAAI 2016 After reading a few of these papers on generative non-goal driven dialogue systems, I’ve ended up both impressed at the early results and the direction they point in, as well as somewhat underwhelmed at the potential for this technology to … Continue reading Building end-to-end dialogue systems using generative hierarchical neural network models
Incorporating (a) copying mechanism in sequence to sequence learning
Incorporating copying mechanism in sequence to sequence learning Gu et al. 2016, with a side-helping of Neural machine translation by jointly learning to align and translate Bahdanau et al. ICLR 2015 Today’s paper shows how the sequence-to-sequence conversational model we looked at yesterday can be made to seem more natural by including a “copying mechanism” … Continue reading Incorporating (a) copying mechanism in sequence to sequence learning
A neural conversation model
A Neural Conversation Model Vinyals & Le, ICML 2015 What happens if you build a bot that is trained on conversational data, and only conversational data: no programmed understanding of the domain at all, just lots and lots of sample conversations…? Building on the sequence to sequence technique that we looked at previously, this is … Continue reading A neural conversation model
A survey of available corpora for building data-driven dialogue systems
A survey of available corpora for building data-driven dialogue systems Serban et al. 2015 Bear with me, it’s more interesting than it sounds :). Yes, this (46-page) paper does include a catalogue of data sets with dialogues from different domains, but it also includes a high level survey of techniques that are used in building … Continue reading A survey of available corpora for building data-driven dialogue systems
Semi-supervised sequence learning
Semi-supervised sequence learning - Dai & Le, NIPS 2015. The sequence to sequence learning approach we looked at yesterday has been used for machine translation, text parsing, image captioning, video analysis, and conversational modeling. In Semi-supervised sequence learning, Dai & Le use a clever twist on the sequence-to-sequence approach to enable it to be used … Continue reading Semi-supervised sequence learning
the morning paper 