Terryl Dickey
28 Jul 2024 07:58
NVIDIA launches VISTA-3D NIM microservice that enhances CT scan analysis with advanced organ and disease segmentation capabilities.
More than 300 million computed tomography (CT) exams are performed worldwide each year, 85 million of which are in the United States alone. Radiologists are constantly looking for ways to accelerate their workflows and produce accurate reports. To address this need, NVIDIA Research has developed a new foundation model, VISTA-3D, which is integrated into an optimized microservice called NVIDIA NIM, designed for scalable deployment, according to the NVIDIA Technical Blog.
VISTA-3D Model
The VISTA-3D (Versatile Imaging Segmentation and Annotation) model is trained on over 12,000 volumes and contains 127 types of human anatomical structures and a variety of lesions, including lung nodules, liver tumors, and bone lesions. It provides accurate out-of-box segmentation and state-of-the-art zero-shot interactive segmentation, making it a versatile tool for medical imaging.
This model provides three core workflows:
- Break everything down: It helps us understand complex diseases that affect multiple organs by comprehensively exploring the body.
- Segments using classes: Provides detailed views by specific classes, essential for analyzing specific diseases.
- Segment Point Prompt: Improve segmentation accuracy and accelerate accurate real-world data generation through custom selection.
The architecture of VISTA-3D includes an encoder layer followed by two parallel decoder layers, one for automatic segmentation and the other for point prompting. This structure ensures high accuracy and adaptability in various anatomical regions.
VISTA-3D NIM Microservices
The VISTA-3D NIM microservice, hosted on the NVIDIA API Catalog, allows users to test the functionality with sample data. It can segment over 100 organs or specific classes of interest, providing views in the axial, coronal, or sagittal planes.
Using NIM Microservices
Users can run VISTA-3D on their data by signing up for a private key from NVIDIA, which gives them 1,000 free credits to try out all NIM microservices. Detailed instructions on how to generate an API key and run models are available, as well as sample code in a variety of programming languages.
If you want to run VISTA-3D on your own data, you will need to set up an FTP server to serve the medical images. This approach accommodates the large size of medical images, which are typically too large to send the API payload directly.
Running the NIM microservice locally
To run the NIM microservice locally, users must apply for NVIDIA NIM access. Once approved, they will receive a Docker container that can run the VISTA-3D NIM microservice on their preferred hardware. Prerequisites include installing Docker, Docker Compose, and the NVIDIA drivers.
Instructions are provided on how to set up an NGINX server to serve a sample Docker Compose file and image to help users get started quickly.
conclusion
NVIDIA’s VISTA-3D-based model represents a significant advance in medical imaging, accurately segmenting over 100 organs and a variety of diseases in CT scans. NVIDIA NIM microservices streamline the deployment and use of this powerful model, improving the workflow and accuracy of radiologists.
Interested parties can apply for access to the VISTA-3D NIM microservice to leverage its capabilities in hardware to streamline their medical imaging processes.
Image source: Shutterstock