Warp 1.5.0 introduces tile-based programming for improved GPU efficiency.

The latest release of Warp 1.5.0 introduces tile-based programming primitives that promise to improve GPU efficiency and productivity. According to NVIDIA, new tools leveraging cuBLASDx and cuFFTDx enable efficient matrix multiplication and Fourier transform within the Python kernel. These advances are particularly important for accelerated simulation and scientific computing.

The Evolution of GPU Programming

Over the past decade, GPU hardware has improved efficiency by moving from a purely Single Instruction, Multiple Threads (SIMT) execution model to one that relies heavily on cooperative operations. As Tensor Core math units become integrated into GPU computing, it is important to program them efficiently. Existing high-level APIs such as BLAS provide extensive abstractions but often lack integration and efficiency when interfacing with user programs.

Tile-based programming in Warp

Tile-based programming models, such as those introduced in Warp 1.5.0, allow developers to express operations on tiles that can be executed cooperatively by multiple threads. This model extends Warp’s kernel-based programming to include tile-based operations, allowing a smooth transition from SIMT to tile-based execution. Supports automatic differentiation for training while reducing the need for manual indexing and shared memory management.

warp tile primitive

Warp’s new tile primitives include composition, load/store, linear algebra, and map/reduce operations. These primitives naturally extend Warp’s existing kernel-based programming model. NumPy-style operations can be used to construct tiles inside a Warp kernel, allowing data to be managed efficiently across CUDA blocks.

Improved matrix multiplication

One of the main advantages of tile-based programming is the ability to perform cooperative matrix multiplication. Warp 1.5.0 introduces: wp.tile_matmul() This is the building block that leverages cuBLASDx to deliver the appropriate Tensor Core MMA instructions for optimal performance. These advancements significantly improve performance, achieving approximately 70-80% of cuBLAS performance for larger matrices.

Case studies and applications

Warp’s tile-based programming is very useful for applications that require dense linear algebra, such as robot simulation and signal processing. For example, in robot simulations, Warp’s tile primitives can efficiently compute the matrix products required for forward dynamics and outperform existing frameworks such as Torch by reducing global memory round trips and execution overhead.

future development

Future versions of Warp and MathDx will include additional support for rowwise reduce operators, tile generation from lambda functions, improved GEMM computational performance, and new linear algebra primitives. These improvements will continue to optimize GPU programming efficiency.

For more information, please refer to the NVIDIA official blog.

Image source: Shutterstock