There are so many articles about learning Deep Learning but still I decided to write one more. The reason is I find many of those articles saying the same thing over and over again. The same set of online courses and the same set of books. I think there is a need for a new guide for learning DL for people who are already well-versed with traditional ML.
Deep Learning is as much science as it is art. It’s increasingly looking like the most promising candidate among a set of different techniques for solving Artificial Intelligence one day. I’ve met and spoken to a lot of people recently who believe doing deep learning is pretty easy, you only need an open source library like TensorFlow, Theano etc. and decent data at your disposal, and you are all set. Trust me, it’s not true.
Coming from a science background before venturing into the world of ML first as an engineer and then as a founder I think one should seriously dive very deep into a field to appreciate the low-level details before building models in a hackish way. This method is OK for a small personal project but not good if you want to be a good researcher in the field one day or if you have plans of building a great product for the real world.
We at Artifacia broadly classify all of AI into Visual Understanding and Language Understanding. This is not exactly the best approach in the world but it helps us organize and execute our projects pretty efficiently. Much of our work is applied in nature with a small part of it being basic and long term in nature such as Project Turing and Project Button. We expect to publish some of our ongoing work sometime next year.
Even though my co-founder and CTO Vivek primarily looks after technology and research at Artifacia, I continue to spend 20% of my time with the research team to be able to do the right kind of mapping between our technology and product, and between our product and business. Moreover, I like speaking to them and continue my learning of an area I believe will impact every industry similar in scale to the Internet and the Personal Computer before that.
The following is the list of essential read for anyone who really wants to learn the fundamentals of Deep Learning :
- A Few Useful Things to Know about Machine Learning by Pedro Domingos
- Deep Learning by Yann LeCun, Yoshua Bengio, and Geoffrey Hinton
- Representation Learning: A Review and New Perspectives by Yoshua Bengio, Aaron Courville, Pascal Vincent
- Convolutional Networks for Images, Speech and Time-Series by Yann LeCun and Yoshua Bengio
- Learning Deep Architectures for AI by Yoshua Bengio
- ImageNet Classification with Deep Convolutional Neural Networks by Alex Krizhevsky, Ilya Sutskever and Geoffery Hinton
- LSTM: A Search Space Odyssey by Klaus Greff, Rupesh K. Srivastava, Jan Koutník, Bas R. Steunebrink and Jürgen Schmidhuber
- Distributed Representations of Words and Phrases and Their Compositionality by Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado and Jeffrey Dean
- Recurrent Neural Network Based Language Model by Tomas Mikolov, Martin Karafiat, Lukas Burget, Jan Cernocky, Sanjeev Khudanpur
- Sequence to Sequence Learning with Neural Networks by Ilya Sutskever, Oriol Vinyals and Quoc V. Le
- Text Understanding from Scratch by Xiang Zhang and Yann LeCun
Notes:
1) This is going to be an evolving post and I’ll keep updating it. The latest update included second and third papers.
2) I’ve also taken inputs from Vivek@Artifacia, who specialises in visual understanding, and Rajarshee@Artifacia, who specialises in language understanding, to compile this list of essential papers.
3) The title of this post is inspired by a popular book series by Zed Shaw. His book Learn Python the Hard Way remains one of the most recommended books for people starting with Python or programming in general.
4) If you’ve already read most of these papers and understood all of it, you should really consider applying to Artifacia!
