NVIDIA Explores RAPIDS cuVS IVF-PQ for Accelerated Vector Search

In a detailed blog post, NVIDIA provides insight into the RAPIDS cuVS IVF-PQ algorithm, which aims to accelerate vector search by leveraging GPU technology and advanced compression techniques. This is the first part of a two-part series that follows our previous exploration of the IVF-Flat algorithm.

Introduction to the IVF-PQ Algorithm

In this blog post, we introduce IVF-PQ (Inverted File Index with Product Quantization), an algorithm designed to improve search performance and reduce memory usage by storing data in a compressed form. However, this method comes at the expense of accuracy, a trade-off that we will explore in more detail in the second part of the series.

IVF-PQ builds on the concept of IVF-Flat, which uses an inverted file index to limit search complexity to smaller subsets of data through clustering. Product Quantization (PQ) adds a compression layer by encoding the database vectors, making the process more efficient for large data sets.

Performance Benchmarks

NVIDIA shared benchmarks using the DEEP dataset, which contains 1 billion records and 96 dimensions, and is 360GiB in size. A typical IVF-PQ configuration compresses it to a 54GiB index without much impact on search performance, or as little as 24GiB with a slight slowdown. This compression allows the index to fit into GPU memory.

Comparing with the widely used CPU algorithm HNSW on a 100 million-item subset of the DEEP dataset, we show that cuVS IVF-PQ can significantly accelerate both index building and vector search.

Algorithm Overview

IVF-PQ follows a two-step process: coarse search and precise search. The coarse search is the same as IVF-Flat, but the precise search calculates the distance between the query point and the vector of the explored cluster, but the vector is stored in a compressed format.

This compression is achieved by PQ, which approximates vectors using two-stage quantization. This allows IVF-PQ to fit more data into GPU memory, improving memory bandwidth utilization and accelerating the search process.

Optimization and Performance

NVIDIA has implemented several optimizations in cuVS to ensure that the IVF-PQ algorithm performs efficiently on GPUs. These include:

• Fuse tasks to reduce output size and optimize memory bandwidth utilization.
• When possible, store lookup tables (LUTs) in GPU shared memory for faster access.
• Use custom 8-bit floating point data types in LUTs for faster data conversion.
• To optimize data transfer, data is aligned in 16-byte chunks.
• Implement an “early stop” check to avoid unnecessary distance calculations.

NVIDIA benchmarks on a 100 million-size dataset show that IVF-PQ outperforms IVF-Flat, delivering up to 3-4x more queries per second for larger batch sizes.

conclusion

IVF-PQ is a powerful ANN search algorithm that leverages clustering and compression to improve search performance and throughput. The first part of the NVIDIA blog series provides a comprehensive overview of how the algorithm works and its benefits on GPU platforms. For more detailed performance tuning recommendations, NVIDIA encourages readers to explore the second part of the series.

For more information, visit the NVIDIA Technology Blog.

Image source: Shutterstock