Tony Kim
15 Aug 2024 01:58
NVIDIA Edify dramatically enhances content creation capabilities by enabling developers to create custom AI models using their own data.
According to the NVIDIA Blog, NVIDIA has unveiled Edify, a cutting-edge AI architecture designed to enable developers to build custom models using their own licensed data. This innovation aims to empower the creative community by providing tools to generate high-quality content from a variety of media types, including images, videos, and 3D assets.
Key Features of NVIDIA Edify
Edify stands out for its multimodal capabilities, allowing it to generate a variety of content types from simple text prompts. The system can generate images, videos, 3D models, 360-degree high dynamic range imaging (HDRi), and physically based rendering (PBR) material. One of its most notable features is its training efficiency, allowing it to produce high-quality content with fewer images.
Edify can also fine-tune models to match a specific style or train on specific characters and objects. This flexibility makes it a powerful tool for a wide range of applications, from artistic endeavors to commercial projects.
Applications and Use Cases
An example of Edify’s best-practice is its integration with Getty Images’ generative AI service. Getty Images used NVIDIA AI Foundry to train Edify on licensed content, ensuring that no copyrighted characters or products were included. The service allows users to create and modify images while maintaining commercial safety. Contributors to the dataset also receive a share of the revenue, benefiting from a new revenue stream.
Edify’s capabilities extend beyond image creation. It can also create artist-ready 3D meshes with clean topology and up to 4K PBR materials. These meshes are ideal for prototyping scenes, creating background objects, or using them as a starting point for 3D sculpting. The system’s quick preview mode can produce results in just 10 seconds, which can then be refined into full 3D models.
Advanced features for image editing
Edify Image offers advanced features for image editing, such as InPaint, which allows users to add or modify content within an image. The Replace function, a more rigorous version of InPaint, can change details such as clothing. OutPaint can expand an image to fit different aspect ratios, and the Segment function simplifies object masking with text prompts.
The system also supports advanced prompt compliance and camera control, allowing users to specify focal length or depth of field. ControlNet, such as Sketch and Depth, guides the creation process, allowing for controllable and customizable output.
360 degree HDRi and multimodal capabilities
Edify 360 HDRi generates environment maps of natural landscapes that can be used for scene lighting, reflections, and backgrounds. The model can generate up to 16K HDRi images from text or image prompts, saving users hours of searching for the right backplate.
One of Edify’s unique strengths is its multimodal capabilities, enabling advanced workflows that combine different asset types. For example, users can prototype entire scenes in minutes with a simple text prompt, as demonstrated in NVIDIA’s Research SIGGRAPH demo. Artists can create scenes in 3D, frame the desired shot, and then use Edify Image to turn the prototype into a photorealistic image.
Generative AI from Getty Images
Getty Images, a leading provider of creative visuals, has trained Edify Image models for its generative AI service using NVIDIA AI Foundry. Available through Getty Images’ Generative AI for businesses and iStock’s Generative AI for small businesses, the service allows users to generate and modify images using models provided by Edify.
Edify Image’s latest update introduces new camera controls to improve creation speed and rapid compliance. Users can now edit and modify iStock’s library of visuals to quickly iterate and perfect their content. These features will soon be available on the Getty Images platform.
For more information, visit the NVIDIA blog.
Image source: Shutterstock