Why does deep and cheap learning work so well Lin & Tegmark 2016 Deep learning works remarkably well, and has helped dramatically improve the state-of-the-art in areas ranging from speech recognition, translation, and visual object recognition to drug discovery, genomics, and automatic game playing. However, it is still not fully understood why deep learning works ... Continue Reading

Texture Networks: Feed-forward synthesis of textures and stylized images Ulyanov et al., arXiv, March 2016 During the summer break I mostly stayed away from news feeds and twitter, which induces terrible FOMO (Fear Of Missing Out) to start with. What great research was published / discussed that I missed? Was there a major industry announcement ... Continue Reading

End-to-end learning of semantic role labeling using recurrent neural networks Zhou & Xu International joint conference on Natural Language Processing, 2015 Collobert’s 2011 paper that we looked at yesterday represented a turning point in NLP in which they achieved state of the art performance on part-of-speech tagging (POS), chunking, and named entity recognition (NER) using ... Continue Reading

Building end-to-end dialogue systems using generative hierarchical neural network models Serban et al. AAAI 2016 After reading a few of these papers on generative non-goal driven dialogue systems, I’ve ended up both impressed at the early results and the direction they point in, as well as somewhat underwhelmed at the potential for this technology to ... Continue Reading

Incorporating copying mechanism in sequence to sequence learning Gu et al. 2016, with a side-helping of Neural machine translation by jointly learning to align and translate Bahdanau et al. ICLR 2015 Today’s paper shows how the sequence-to-sequence conversational model we looked at yesterday can be made to seem more natural by including a “copying mechanism” ... Continue Reading

A Neural Conversation Model Vinyals & Le, ICML 2015 What happens if you build a bot that is trained on conversational data, and only conversational data: no programmed understanding of the domain at all, just lots and lots of sample conversations…? Building on the sequence to sequence technique that we looked at previously, this is ... Continue Reading

Semi-supervised sequence learning - Dai & Le, NIPS 2015. The sequence to sequence learning approach we looked at yesterday has been used for machine translation, text parsing, image captioning, video analysis, and conversational modeling. In Semi-supervised sequence learning, Dai & Le use a clever twist on the sequence-to-sequence approach to enable it to be used ... Continue Reading