Convolutional Neural Networks for Image Style Transfer

In this thesis we will use deep learning tools to tackle an interesting and complex problem of image processing called style transfer. Given a content image and a style image as inputs, the aim is to create a new image preserving the global structure of the content image but showing the artistic patterns of the style image. Before the renaissance of Arti�cial Neural Networks, early work in the �field called texture synthesis, only transferred limited and repeatitive geometric patterns of textures. Due to the avaibility of large amounts of data and cheap computational resources in the last decade Convolutional Neural Networks and Graphics Processing Units have been at the core of a paradigm shift in computer vision research. In the seminal work of Neural Style Transfer, Gatys et al. consistently disentangled style and content from different images to combine them in artistic compositions of high perceptual quality. This was done using the image representation derived from Convolutional Neural Networks trained for large-scale object recognition, which make high level image informations explicit. In this thesis, inspired by the work of Li et al., we build an efficient neural style transfer method able to transfer arbitrary styles. Existing optimisation-based methods (Gatys et al.), produce visually pleasing results but are limited because of the time consuming optimisation procedure. More recent feedforward based methods, while enjoying the inference efficiency, are mainly limited by inability of generalizing to unseen styles. The key ingredients of our approach are a Convolutional Autoencoder and a pair of feature transforms, Whitening and Coloring, reflecting a direct matching of feature covariance of the content image to the given style image. The algorithm allows us to produce images of high perceptual quality that combine the content of an arbitrary photograph with the appearance of arbitrary well known artworks.