Skip to content

An experiment with awesome deep learning papers

February 26, 2017

There have been several lists of deep learning papers doing the rounds. Recently Terry Taewoong Um’s list of the top 100 awesome and most cited deep learning papers caught my eye. Deep learning is an exciting area and it’s moving fast. I’d like to know what’s in those 100 papers (thankfully, we have at least looked at some of them before), and I suspect many of you would too. The problem is, at five papers per week it would take 20 weeks of The Morning Paper to cover the list! That’s too long, and even if it wasn’t, it wouldn’t match my goals for diversity of content. Ideally I’d press pause on the world for a while, go take a bunch of math classes, read through all of these papers, and then be in a strong position to understand the 2017 intake. That’s not possible either, so we need a faster plan…

This week on The Morning Paper therefore I’m trying something a little different: we’ll be working through some of the top 100 papers list in sections, covering multiple papers per day but with shorter reviews of each. Hopefully you’ll find it a good way to take on board a lot of research very quickly.

From my perspective the results are mixed – I certainly covered a lot of ground (26 deep learning papers read and reviewed over the week), but I think I discovered my current limit for paper reviews per week in the process! (It happened to coincide with a a busy work week as well). Another consequence is that the posts are longer than my usual target length, coming in at about 15 mins reading time according to my editor’s toolbar. I fear that may be too long, even in condensed review format. Let me know what you think.

One thing’s for sure, for the following week I’m going back to just one paper per day thank you very much! (And we’ll be looking at something other than deep learning too 😉 ).

7 Comments leave one →
  1. Kyle M. permalink
    February 26, 2017 5:53 pm

    “Ideally I’d press pause on the world for a while, go take a bunch of math classes, read through all of these papers, and then be in a strong position to understand the 2017 intake.”

    This is actually what I’m doing right now 🙂 I’ve been using Khan Academy for the math part, as a prerequisite for Andrew Ng’s course, but a lot of the papers are still very fascinating and informative without all the relevant math skills.

  2. February 26, 2017 8:32 pm

    Awesome Adrian!
    I follow this blog on a daily basis and I’m also a researcher working with DL. I’d be very please to see more DL-related content here.

    warm regards from Brazil

  3. katagorikal permalink
    February 27, 2017 7:06 am

    Sounds like you need some NLP, references graph theory and deep learning analysis of the deep learning papers? Feed them their own dog food.

  4. Paul B permalink
    February 27, 2017 4:11 pm

    Is there a guide to “math background needed for deep learning”? I guess stats and linear algebra to start…

Trackbacks

  1. Convolutional neural networks, Part 1 | the morning paper

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: