Jessie A. Ellis
December 4, 2024 20:28
AWS and NVIDIA will scale NIM microservices across the AWS platform and enhance AI inference capabilities by increasing the efficiency of generative AI applications and reducing latency.
Amazon Web Services (AWS) announced that it is expanding its collaboration with NVIDIA by integrating NVIDIA NIM microservices into its core AI services. According to NVIDIA, the move, unveiled at the AWS re:Invent conference, aims to accelerate AI inference and reduce latency for generative AI applications.
Enhanced AI Inference with NVIDIA NIM
NVIDIA NIM microservices are now easily accessible through AWS Marketplace, Amazon Bedrock Marketplace, and Amazon SageMaker JumpStart. This availability simplifies deployment of NVIDIA-optimized inference for popular models at scale. NIM microservices, part of the NVIDIA AI enterprise software platform, provide secure, high-performance deployment of AI model inference in a variety of environments.
These pre-built containers leverage advanced inference engines such as NVIDIA Triton Inference Server and NVIDIA TensorRT to support a wide range of AI models. Developers can leverage these services across a variety of AWS platforms, including Amazon EC2 and Amazon EKS, to increase model deployment flexibility and performance.
Extensive support model
Developers can explore more than 100 NIM microservices, including models from NVIDIA, Meta’s Llama 3, Mistral AI, and more. These services are optimized for deploying NVIDIA accelerated compute instances on AWS, providing a powerful solution for AI model inference.
In particular, NVIDIA Nemotron-4 and Llama 3.1 models are now available directly on AWS and offer advanced features for data synthesis and multilingual conversation, respectively. These models are designed to improve AI application performance and reliability in a variety of areas.
Industry adoption and use cases
Industries are increasingly adopting NIM on AWS to accelerate time to market, ensure security, and reduce the cost of generative AI applications. For example, IT consulting firm SoftServe has developed several AI solutions using NVIDIA NIM, now available in AWS Marketplace. This includes applications for drug discovery, industry support, and content creation, all leveraging NVIDIA AI Blueprint for accelerated development and deployment.
Getting Started with NIM on AWS
Developers interested in deploying NVIDIA NIM microservices can get started by exploring the NVIDIA API catalog, which offers a variety of NIM optimization models. You can begin deploying these microservices across the AWS platform by requesting a developer license or trial license for NVIDIA AI Enterprise. This initiative highlights AWS and NVIDIA’s commitment to advancing AI technology and facilitating seamless integration for developers.
Image source: Shutterstock