Can you imagine that you can change any photo you want into a piece of art on your own? It is made possible now! Recently, researchers from University of Tubingen, Leon A. Gatys and his colleagues have found a new way to automatically transfer an image to any style while preserving the original content.
Actually, there are many methods in previous studies used to do the style transferring work. However, resultant images are not statisfiable because only low-level image features are transferred, leading to unnatural images. This time, the researchers applied convolutional neural network (CNN), a kind of neural network popularly used in photo manipulation, in style transferring. As a result, new styles could be fit in photos more perfectly without any abrupt feelings.
The image synthesized with Van Gogh’s The Starry Nigtht
Credit: Leon A. Gatys, Alexander S. Ecker and Alexander S. Ecker
Before the transferring stage, their study is mainly focused on separating content representation and style representation of images, which are features extracted from images for further recombination. Usually, the semantic content and style are hard to separate. Even for artists themselves, it is really hard to tell which part is exactly content and which is exactly style. In this study, both of them are caught by the CNN, which captures object information accurately so that both the content and the style could be used independently to synthesize new meaningful images.
Style Represetation extracted from The Starry Nigtht
Credit: Leon A. Gatys, Alexander S. Ecker and Alexander S. Ecker
The final step is not easy either. The synthesis is not simply combining representations got from the previous step, you need to decide which, original content or new style, to be dominant in the new image. Naturally, we want the new style to dominate, but in fact, a strong emphasis on style could result in images matching the appearance of artwork but losing the original content.
The presentation of the study
Viedeo Credit: Leon A. Gatys, Alexander S. Ecker and Alexander S. Ecker
Finally, their work paid off. The synthesis images using their method show a high perceptual quality. Thanks to their study, some image style transferring services based on the study, such as deepart.io, allow everyone to upload any image to produce a new artistic picture. Now, everyone can enjoy what the new technology brings to us.
Style Transferred picture from deepart.io
Credit: deepart.io
There are still some limitations in this study, the major one is about resolution. The speed of production heavily depends on image resolution. So, costumers need to wait a long time before getting synthesis picture if they want a high-quality picture, which influence customers’ experience with the online service. Further studies could find ways to speed up the whole process so that all customers could get instant feedbacks with high quality. I believe it will popularize the service further so that more people can join in!
