jack anderson
October 2, 2024 19:53
Brave browser users can now experience enhanced AI interactions through the integration of RTX-accelerated Leo AI and Ollama, which provides local processing for improved privacy and efficiency.
The Brave browser, known for its focus on privacy, has partnered with Ollama to introduce Leo AI, a powerful AI assistant powered by RTX-accelerated local large language models (LLMs), according to an NVIDIA blog. This integration aims to improve user experience by providing efficient, locally processed AI capabilities.
Enhanced AI Experience with RTX Acceleration
Powered by NVIDIA’s RTX technology, Brave’s Leo AI gives users the ability to summarize articles, extract insights, and answer questions directly within the browser. This is achieved using NVIDIA’s Tensor Cores, which are designed to handle AI applications by processing numerous computations simultaneously. Collaborating with Ollama allows Brave to leverage the open source llama.cpp library, which facilitates AI inference tasks that are specifically optimized for NVIDIA’s RTX GPUs.
Advantages of Local AI Processing
Running AI models locally on your PC has significant privacy benefits by eliminating the need to send data to an external server. This local processing approach ensures that user data remains private and accessible without the need for cloud services. Users can also interact with a variety of specialized models, such as bilingual or code generation models, without the cost of cloud services.
Technology Integration and Performance
Brave’s integration with Ollama and RTX technology delivers responsive AI experiences, and the Llama 3 8B model achieves processing speeds of up to 149 tokens per second. This setting improves Leo AI’s overall browsing experience by ensuring quick responses to user queries and content requests.
Getting started with Leo AI and Ollama
Users interested in taking advantage of these advanced AI features can easily install Ollama from the official website. Once installed, Brave’s Leo AI can be configured to use a local model through Ollama, giving you the flexibility to switch between cloud and local models as needed. Developers can learn more about using Ollama and llama.cpp through resources provided by NVIDIA.
Image source: Shutterstock