Tags
The arc of Deep Learning is an interesting one. It could rightly be accused that “deep learning” is a rebranding, reminiscent of the introductory remarks delivered in class about the evolution of terminology from “business intelligence” to “data mining” to “data science.” Perhaps that accusation is accurate, but perhaps this evolution of terminology is necessary to convey the effect when incremental changes (such as distributed computing and massive data storage) result in a qualitative transition. Indeed, according to Abdel-Rehman Mohamed, a PhD computer scientist trained at the University of Toronto and slated to join IBM Research, “we’re now at the intersection of so many things that we didn’t have in the past.”(1) He’s speaking about computer hardware, algorithm sophistication, and data acquisition and storage.
In addition to the changes above, development of Graphics Processing Unit (GPU) designs has also aided the shift from traditional ANNs to modern deep learning. According to Jürgen Schmidthuber of the Swiss Artificial Intelligence Lab, “GPUs … accelerate learning by a factor of 50.”(4)
That concept – that the practical application of deep learning is, indeed, qualitatively different from previous attempts at applying ANN (artificial neural network technology) in part because of the simultaneous development of “so many things” serves as the key to understanding the future of deep learning. However, another kernel from the above – Mohamed’s departure from the University of Toronto and into IBM Research – is also key to understanding the state of Deep Learning in the present moment.
(Pallister ’13)(2)
The figure above, adapted from a publication by the embedded systems company EMBecosm, illustrates the relationship between basic research and applied research. Deep learning is quickly making the leap into the domain of applied, commercial research and development. Interestingly, it is often the very same figures who are quite literally making the transition in their careers: In 2013 Yann LeCun, an expert in deep learning from NYU, announced that he would transition from his post at New York University to join Facebook. In the 1980’s, LeCun was instrumental in the development of the back-propagation neural networks that are directly responsible for what is now called deep learning. (1) It seems fitting, then, that he would take the step into industry in 2013.
LeCun is not the only member of the NYU computer science faculty to make the move to Facebook. Rob Fergus, director of NYU’s M.S. in Data Science program, also announced that he would join the company alongside his colleague.(3)
Taken along with the numerous startup companies mentioned in our second blog post, all vying to commercialize the technology, it seems that the time has come for deep learning to prove whether it will succeed in building profits and providing competitive advantages or whether this wave of interest will serve as another bust in the tumultuous relationship between “artificial intelligence” and private industry.
One can find a glimmer of hope, however, in the possibility that perhaps this time the technology will be deployed to solve those problems – and only those problems – that are peculiarly well-suited for deep learning. Indeed, a 2014 article from Business Insider calls attention to marketing department efforts in “social listening” in the domain of social media images and photos, using deep learning to take a step beyond the kind of straightforward sentiment analysis that can be tackled with bag-of-words representation and information retrieval techniques.(5)
In conclusion, the debate over whether “deep learning” should be kept conceptually distinct from ANN is functionally a moot point. Whatever the technology is called, it is currently experiencing a renaissance as developments across several fronts (algorithms, computational power, data availability, storage capacity/cost, and GPU design) have allowed for quick and accurate unsupervised learning on data types previously inaccessible such as images and videos. The technology has burst from the walls of academia and is, as of 2013 and 2014, in the midst of a transition from academia and basic research to industry and applied research. Finally, the success or failure of deep learning is likely to hinge upon the ability of firms to leverage it in the right spot in their overall analytics strategy. Deep learning is likely to be a strong contender for problems such as “social listening,” a task that fits the problem paradigm nicely due to the highly unstructured nature of inputs and relatively nominal nature of outputs.
(1) Metz, Cade. “60 years later, Facebook heralds a new dawn for artificial intelligence.” Wired. 12/10/2013. http://www.wired.com/2013/12/facebook-deep-learning/
(2) Pallister, James. “Pasteurized computing: the relationship between academia and industry.” EMBecosm. 2/1/2013. http://www.embecosm.com/2013/02/01/academia-and-industry/
(3) Fergus, Rob. “I’m joining Facebook!” (blog post) http://cs.nyu.edu/~fergus/pmwiki/pmwiki.php
(4) Angelica, Amara. “How bio-inspired deep learning keeps winning competitions.” 11/28/2012. http://www.kurzweilai.net/how-bio-inspired-deep-learning-keeps-winning-competitions
(5) Smith, Cooper. “Social Media’s Big Data Future.” Business Insider Australia. 2/8/2014. http://www.businessinsider.com.au/social-medias-big-data-future-from-deep-learning-to-predictive-marketing-2014-2