NVIDIA and Infosys Transform Telecom Network Operations with AI Solutions

Telecommunications companies face the ongoing challenge of meeting service level agreements (SLAs) to ensure network quality and quickly resolve complex network issues. These challenges often result in long-term network downtime, impacting operational efficiency and customer experience.

To address these challenges, Infosys developed a generative AI solution using NVIDIA NIM and NVIDIA NeMo Retriever that aims to streamline network operations centers (NOCs) by automating network troubleshooting, minimizing downtime, and optimizing performance.

Building a Smart Network Operations Center Using Generative AI

Infosys, a global leader in digital services, has built a smart NOC using a generative AI customer engagement platform. The platform provides essential vendor-independent router commands for diagnostics and monitoring, supporting NOC operators, network administrators, and IT support staff. This intelligent chatbot reduces mean time to resolution and improves customer service.

Challenges in vector embedding and document retrieval

Infosys faced several challenges in developing the chatbot, including balancing high accuracy and low latency while handling network-specific taxonomy and complex device documents. The time-consuming nature of vector embedding process on CPU and the latency issues of LLM were also significant obstacles.

Data collection and preparation

To overcome these challenges, Infosys built a vector database of network device manuals and knowledge artifacts, initially focusing on devices from Cisco and Juniper Networks. Custom embedding models and fine-tuned parameters populate the vector database to ensure accurate and contextual responses to user queries.

Solution Architecture

Infosys’ solution architecture included several key components:

• User Interface and Chatbot: We developed an intuitive interface using React for custom chatbots and advanced query scripting.
• Data Configuration Management: Provides flexible settings for chunking and embedding using NVIDIA NeMo Retriever.
• Vector Database Options: We implemented options like FAISS for high-speed data retrieval.
• Backend Services and Integrations: We’ve built a robust backend service, including a RESTful API for integration with external systems.
• Integration with NIM: We leveraged NIM microservices to improve accuracy and performance.
• composition: It utilizes 10 NVIDIA A100 80GB GPUs, 128 CPU cores, and 1TB of storage.
• Guardrail: Enhanced security and stability with NVIDIA NeMo Guardrails.

AI Workflows with NVIDIA NIM and NeMo Guardrails

Infosys fine-tuned and deployed a basic LLM using self-hosted instances of NVIDIA NIM and NeMo. NeMo Retriever powered vector database search and re-ranking workflows, enabling the company to connect custom models to business data and deliver accurate answers. Read more on NVIDIA’s blog.

Using NeMo Retriever and NV-Embed-QA-Mistral-7B, Infosys achieved over 90% accuracy on text embedding models. The models excel at a variety of tasks, improving accuracy and performance.

result

Infosys measured LLM latency and accuracy with and without NVIDIA NIM. Without NIM, LLM latency was 2.3 seconds, with NIM it was reduced to 0.9 seconds, a 61% improvement. Integrating NeMo Retriever embedding and reranking microservices improved accuracy from 70% to 92%.

conclusion

By integrating NVIDIA NIM and NeMo Retriever, Infosys has significantly improved the performance and accuracy of its smart NOC. These improvements simplify network troubleshooting, reduce downtime, and optimize overall network performance.

Learn more about how Infosys eliminates network downtime with NVIDIA-powered automated workflows on NVIDIA’s official blog.

Image source: Shutterstock