Optimisation and training techniques for deep learning

Today we're looking at the 'optimisation and training techniques' section from the 'top 100 awesome deep learning papers' list. Random search for hyper-parameter optimization, Bergstra & Bengio 2012 Improving neural networks by preventing co-adaptation of feature detectors, Hinton et al., 2012 Dropout: a simple way to prevent neural networks from overfitting, Srivastava et al., 2014 ... Continue Reading

When DNNs go wrong – adversarial examples and what we can learn from them

Yesterday we looked at a series of papers on DNN understanding, generalisation, and transfer learning. One additional way of understanding what's going on inside a network is to understand what can break it. Adversarial examples are deliberately constructed inputs which cause a network to produce the wrong outputs (e.g., misclassify an input image). We'll start ... Continue Reading

Learning to protect communications with adversarial neural cryptography

Learning to protect communications with adversarial neural cryptography Abadi & Anderson, arXiv 2016 This paper manages to be both tremendous fun and quite thought-provoking at the same time. If I tell you that the central cast contains Alice, Bob, and Eve, you can probably already guess that we're going to be talking about cryptography (that ... Continue Reading