Opening the black box of deep neural networks via information Schwartz-Viz & Tishby, ICRI-CI 2017 Yesterday we looked at the information theory of deep learning, today in part II we’ll be diving into experiments using that information theory to try and understand what is going on inside of DNNs. The experiments are done on a … Continue reading Opening the black box of deep neural networks via information – Part II

Opening the black box of deep neural networks via information Schwartz-Viz & Tishby, ICRI-CI 2017 In my view, this paper fully justifies all of the excitement surrounding it. We get three things here: (i) a theory we can use to reason about what happens during deep learning, (ii) a study of DNN learning during training … Continue reading Opening the black box of deep neural networks via information – part I

Matrix capsules with EM routing Anonymous ;), Submitted to ICLR’18 (Where we know anonymous to be some combination of Hinton et al.). This is the second of two papers on Hinton’s capsule theory that has been causing recent excitement. We looked at ‘Dynamic routing between capsules’ yesterday, which provides some essential background so if you’ve … Continue reading Matrix capsules with EM routing

Dynamic routing between capsules Sabour et al., NIPS’17 The Morning Paper isn’t trying to be a ‘breaking news’ site (there are plenty of those already!) — we covered a paper from 1950 last month for example! That said, when exciting research news breaks, of course I’m interested to read up on it. So The Morning … Continue reading Dynamic routing between capsules