Really Deep Neural Networks with PyTorch

Modern neural networks consist of hundreds of computation layers! These very deep architectures consistently outperform shallower networks in a variety of tasks. However just simply stacking layers on top of each other won't work because the gradients are either vanishing or exploding during optimisation procedure. This talk explains the exciting math, cool ideas and elegant code that modern neural network architectures such as ResNets, HighwayNets and DenseNets are applying to circumvent the problem using PyTorch. PyTorch is a relatively new deep learning framework that is deeply integrated into Python. Unlike other frameworks such as TensorFlow and Theano, it uses tape-based automatic differentiation to run computation immediately, supports dynamic neural networks and provides a powerful GPU-accelerated Tensor library. The talk concludes with some real-world use-cases for very deep neural networks in chemical-genetic profiling and autonomous driving.