Edgeless Systems has launched Continuum AI, a pioneering generative AI (GenAI) framework designed to maintain encrypted prompts at all times through confidential computing. According to the NVIDIA Technology Blog, this innovative solution combines confidential virtual machines (VMs) with NVIDIA H100 GPUs and secure sandboxing.
Data Privacy and Security Guarantee
The launch of Continuum AI represents a significant step forward in AI deployment, allowing enterprises to leverage powerful large-scale language models (LLMs) without compromising data privacy and security. Edgeless Systems is working with NVIDIA to help organizations across a wide range of industries securely integrate AI. The platform is not only a technological breakthrough, but also a significant step toward a future where AI can be safely leveraged even on the most sensitive data.
Continuum AI’s Security Goals
Continuum AI has two primary security goals: protecting user data and protecting AI model weights for infrastructure and service providers. Infrastructure includes all the underlying hardware and software stacks on which AI applications run, such as cloud platforms like Microsoft Azure. Service providers control AI applications, such as OpenAI for ChatGPT.
How Continuum AI Works
Continuum AI relies on two key mechanisms: confidential computing and advanced sandboxing. Confidential computing is a hardware-based technique that ensures that data remains encrypted while being processed, thereby verifying the integrity of the workload. Powered by NVIDIA H100 Tensor Core GPUs, this approach creates a secure environment that separates infrastructure and service providers from data and models. It also supports popular AI inference services such as NVIDIA Triton Inference Server.
Despite these security measures, third-party AI code can still leak prompts, either accidentally or maliciously. The complexity and frequent updates of AI code make thorough review of AI code impractical. Continuum addresses this by running AI code within a sandbox of AI workers protected by confidential computing, using a modified version of Google’s gVisor sandbox. This ensures that AI code can only process encrypted prompts and responses, preventing plaintext data leaks.
System Structure
Continuum AI consists of two main components: the server side, which hosts AI services and securely processes prompts, and the client side, which encrypts prompts and verifies servers. The server-side architecture includes worker nodes and an attestation service.
The worker nodes, which are the heart of the backend, host AI models and process inference requests. Each worker runs within a confidential VM (CVM) running Continuum OS, a minimally verifiable system via remote attestation. The CVM hosts workloads in a sandbox and mediates network traffic through an encrypted proxy to ensure secure data processing.
The attestation service ensures the integrity and authenticity of worker nodes, ensuring that both service providers and clients are interacting with a secure deployment. This service runs on the CVM and manages key exchange for fast encryption.
Workflow and user interaction
The administrator verifies the integrity of the attestation service via the CLI and configures the AI code using the worker API. Verified workers can receive inference secrets and securely process requests. Users interact with the attestation service and worker nodes to verify deployments and send encrypted prompts for processing. The encryption proxy decrypts these prompts, processes them in the sandbox, and re-encrypts the responses before sending them back to the user.
Visit our Continuum page to learn more about this cutting-edge technology.
Image source: Shutterstock