Deep learning: What’s changed?

Deep learning has been making headlines as the UK AlphaGo team beat Lee Sedolin the Go board game. Go being a more complex game, it is hard to discover new strategies and thus winning it is difficult.

Deep learning made the headlines when the UK’s AlphaGo team beat Lee Sedol, holder of 18 international titles, in the Go board game. Go is more complex than other games, such as Chess, where machines have previously crushed famous players. The number of potential moves explodes exponentially so it wasn’t possible for computers to use the same techniques used in Chess. In learning Go, the computer would have to create millions of games, competing against itself and discovering new strategies that humans may never have considered.

Deep learning itself isn’t that new, and researchers have been working on algorithms for many years, refining the approach and developing new algorithms. What has stimulated it recently is the convergence of massively parallel processing, huge data sets and superior performance against traditional machine learning algorithms.

How does deep learning differ from traditional algorithms?

Let’s take a few examples. A credit scoring model based on logistic regression will typically use around ten to fifteen input parameters, such as age, income, time at address etc. More complex decision trees or neural networks used to detect fraud may use hundreds of parameters. Deep learning takes this to a whole new level and may use hundreds of thousands or even millions of parameters. This can only really work when there are thousands or even millions of examples to train the models.

The internet is an ideal place to find examples. For instance, when you search for images of cats, dogs, trains, and so on it will probably be a deep learning algorithm that’s been used to classify the image. Other uses extend to natural language processing, translation, facial recognition – Google and Facebook are known to be extensive users of these algorithms. Interestingly, humans were used to classify the initial images through such techniques like Captcha® where the user confirms they’re a human by identifying which images are dogs, buildings, areas of water etc. Each batch of images would include some known images, but also some that were unknown – once a few users agree on an image, it can be marked as classified and the process repeated on new images. In this way, thousands of images can be quickly classified for use in algorithm training.