Generative models have revolutionized the AI landscape, with popular applications such as ChatGPT and Stable Diffusion. According to the NVIDIA blog, basic AI models and generative adversarial networks (GANs) have sparked leaps in productivity and creativity.
NVIDIA’s GauGAN, an AI model that transforms rough sketches into realistic artwork, is an example of this revolution. GauGAN powers the NVIDIA Canvas app, a cornerstone for creative professionals around the world.
How it started
GAN is a deep learning model that contains two complementary neural networks, a generator and a discriminator. The generator creates vivid images, and the discriminator tries to distinguish between real images and generated images. Through this competitive process, GAN improves its ability to generate realistic samples.
These models excel at understanding complex data patterns and producing high-quality results, which can be applied to image synthesis, style transfer, data augmentation, and image-to-image translation.
Named after the late Impressionist painter Paul Gauguin, NVIDIA’s GauGAN is an AI demo for photorealistic image generation. Built by NVIDIA Research, the demo directly led to the development of the NVIDIA Canvas app. Since its debut at NVIDIA GTC in 2019, GauGAN has gained widespread popularity, with millions of online users, including art teachers, creative organizations, and museums.
Van Gogh’s Sketch of a Landscape
Powered by GauGAN and local NVIDIA RTX GPUs, NVIDIA Canvas uses AI to turn simple brushstrokes into photorealistic landscapes in real time. Users start by sketching lines and shapes using a palette of real-world elements, called “materials,” such as grass or clouds.
The AI model then generates an enhanced image in real time on the other half of the screen. For example, a triangle sketched using the “Mountain” material appears as a realistic mountain range. Users can select the “Cloud” material to transform a clear environment into a cloudy environment with just a few clicks.
The creative possibilities are endless. If you sketch a pond, other elements like trees and rocks will be reflected in the water. If you change the material from snow to grass, the scene will change from a wintery background to a tropical paradise.
Canvas offers a Panoramic mode that allows artists to create 360-degree images for use in 3D applications. YouTuber Greenskull AI demonstrated Panoramic mode by drawing a seascape and importing it into Unreal Engine 5.
Canvas offers nine different styles, each with 10 variations and 20 materials to play with. The app can be downloaded from the NVIDIA website.
In addition to Canvas, NVIDIA offers other AI-based content creation apps, such as NVIDIA Broadcast, which is free for RTX GPU owners and turns any room into a home studio.
Generative AI is revolutionizing gaming, video conferencing, and conversational experiences. Subscribe to the AI Decoded newsletter to get the latest developments.
Image source: Shutterstock