Terrill Dickey
December 17, 2024 18:31
NVIDIA is introducing small language models to enhance digital human responses, enabling improved interactions with agents, assistants, and avatars on RTX AI PCs.
NVIDIA has announced a new SLM (Small Language Models) series aimed at strengthening digital human capabilities. These models are part of NVIDIA ACE, a suite of technologies designed to harness the power of RTX AI PCs to bring agents, assistants, and avatars to life.
Introduction to multi-mode function
New models include NVIDIA Nemovision-4B-Instruct, a multimodal SLM that allows digital humans to interpret visual images and provide context-sensitive responses. Built using the latest NVIDIA VILA and NeMo frameworks, these models are optimized for a range of NVIDIA RTX GPU performance, maintaining the high accuracy levels essential to developers.
Large-scale context language model
NVIDIA’s new large-scale context SLM is designed to manage a wide range of data inputs, making complex prompts easier to understand. Available in 8B, 4B, and 2B parameter versions, the Mistral-NeMo-Minitron-128k-Instruct family balances speed, memory usage, and accuracy on NVIDIA RTX AI PCs. These models can process significant amounts of data in a single pass, improving accuracy by reducing the need for data segmentation.
Audio2Face-3D NIM enhancements
NVIDIA has also updated the Audio2Face-3D NIM microservice to improve the realism of facial animation, which is critical for true digital human interaction. This microservice now supports real-time lip sync and facial animation, increasing customization options through a single, downloadable optimization container.
Streamline deployment on RTX AI PCs
Deploying digital humans on RTX AI PCs requires efficient coordination of animation, intelligence, and voice AI models. NVIDIA is introducing new SDK plugins and samples to facilitate on-device workflows, including Unreal Engine 5 sample applications powered by NVIDIA Riva automatic speech recognition and Audio2Face-3D. These tools are part of the NVIDIA In-Game Inference SDK, currently in beta, which simplifies AI integration by managing model and dependency downloads and enabling hybrid AI operations.
Developers interested in these advancements can access these tools through the NVIDIA Developer Platform.
Image source: Shutterstock