Iris Coleman
October 13, 2024 02:37
AMD has released ROCm 6.2.3 to enhance the AI capabilities of Radeon GPUs and improve AI development efficiency with improved support for Llama 3, Stable Diffusion, and Triton frameworks.
AMD has released the latest version of AMD ROCm™ 6.2.3, open computing software specifically designed to improve the performance of Radeon GPUs on native Ubuntu® Linux® systems. According to AMD.com, this update aims to provide superior inference performance for AI models, specifically the Llama 3 70BQ4, and allow developers to integrate the Stable Diffusion (SD) 2.1 text-to-image conversion feature into their AI projects.
Key features of ROCm 6.2.3
The new ROCm 6.2.3 release brings several advanced features aimed at accelerating AI development.
- Llama 3 support via vLLM: This feature provides excellent inference performance on Radeon GPUs using the Llama 3 70BQ4 model.
- Flash Attention 2 integration: Designed to optimize memory usage and improve inference speed, this feature supports forward activation.
- Stable Spread 2.1 Support: Developers can now integrate SD text-to-image models into AI applications.
- Triton Framework Beta Support: This allows developers to efficiently utilize AMD hardware to write high-performance AI code with minimal expertise.
Advances in AI Development
Erik Hultgren, software product manager at AMD, emphasized that ROCm 6.2.3 targets specific features to facilitate generative AI development. This release includes expert-level performance improvements for Large Language Model (LLM) inference with vLLM and Flash Attention 2. We’re also introducing beta support for the Triton framework, expanding the scope of AI development on AMD hardware.
Evolution of ROCm support
AMD’s ROCm support for Radeon GPUs has evolved significantly over the past year, starting with the 5.7 release. Version 6.0 expands functionality by integrating the ONNX runtime and officially certifying more Radeon GPUs, including the Radeon PRO W7800. The 6.1 update marks another milestone with support for multi-GPU configurations and integration with the TensorFlow framework.
In the current release, ROCm 6.2.3 continues to focus on Linux® systems, with plans to soon introduce support for Windows Subsystem for Linux® (WSL 2). This strategic approach aims to further enhance the ROCm solution stack for Radeon GPUs, positioning it as a powerful option for AI and machine learning development.
For additional information and resources, visit AMD’s official community page.
Image source: Shutterstock