Image creator program GauGAN

WHAT IT IS
Created by NVIDIA Research GauGAN project is a program which allows you to turn simple paintings into photorealistic pictures using AI technology.

HOW IT WORKS
GauGAN gives you blank space to draw on, where each brush type is labeled with different type of environment like sand, grass, sky, ocean. Option of changing daytime is also available. Then AI converts the sketch filling into desired elements basing on online pictures database. Using GANs technology program creates credible outcome. In the process two neural networks are used where one is called generator and the latter one discriminator. Generator creates image that later is corrected by discriminator in terms of authenticity.

GANs SYSTEM
Gaugan is using GANs (Generative Adversarial Networks). These are neural networks that cooperate in the process of deep learning, by competing with eachother. E.g one is creating a fake cat picture, while the other one determines whether it is real or fake. This eliminates the need for human to correct the AI and automatizes the process. This way human effort in advancing neural networks is minimized.

USE IN WORLD
In the future the application can help proffesions which require creating world representations like architects, urban planners or game developers. It would allow them to preview the outcome of their work with minimal effort and prevent from committing to bad ideas.

INSIGHTS
Current state of the project may be impressive but its very restrictive. The fact that you are allowed to use only certain labels which resemble specific parts of environment is disappointing and prevents from exploiting maximum potential of creativity. The most obvious way of solving that problem would be adding more labels but connecting elements from various types of environments could corrupt the final landscape. E.g covering a desert with a snow or placing a star on the dirt background would distort the picture, because AI would have no data to refer to. Project’s advancement is dependent purely on describing parts of the world with new algorithms and putting them to the GANs test.

 

EXAMPLES – Here are some masterpieces I’ve created.

REALISTIC EXAMPLE – easy to make, low chance of corruption

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

ABSTRACT EXAMPLE – harder to make, high chance of corruption

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

CORRUPTED EXAMPLE – elements added to habitat in which they not occure will distort the picture. Here is a house in the middle of the ocean with a bridge connected to a pile of mud above.

 

 

 

 

 

 

 

 

 

 

 

 

Here are few breathtaking possible outcomes.

 

 

SOURCES

My sketch using NVIDIA’s GauGAN – using machine learning to turn doodles into realistic landscapes from MediaSynthesis

https://www.artstation.com/nicodemus/blog/O8Q4/nvidia-gaugan-fun

https://blogs.nvidia.com/blog/2017/05/17/generative-adversarial-networks/

The following two tabs change content below.
Avatar

Wiktor Ślęzak

Avatar

Latest posts by Wiktor Ślęzak (see all)

Leave a Reply