Other Challenges
Nuts and Bolts of Applying Deep Learning (Andrew Ng)
The Wall - Deep Learning
Deep neural nets are huge and bulky inefficient creatures that allow you to effectively solve a learning problem by getting huge amounts of data and a super computer. They currently trade efficiency for brute force almost every time.
The Expert