Peter Jang
7 November 2024 17:10
Mistral AI introduces a new moderation API that aims to improve content safety through a scalable and robust LLM-based system and support multiple languages for a variety of applications.
Mistral AI announced the release of its new Moderation API, a tool designed to improve the safety and scalability of content management systems. According to Mistral AI, the API aims to help users detect undesirable text content across a variety of policy dimensions.
Enhanced Safety Measures
The Moderation API is built on the same framework that powers the moderation service in Mistral AI’s Le Chat platform. It provides users with a flexible tool that can be customized to meet specific safety standards and application requirements. As demand for Large Language Model (LLM)-based coordination systems grows, Mistral AI’s products seek to provide scalable and robust solutions.
Multilingual function
The API has an LLM classifier that can classify text input into nine distinct categories. It contains endpoints for raw text and conversation content, allowing you to classify messages within a specific conversation context. This model supports multiple languages, including Arabic, Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Russian, Spanish, etc., making it suitable for users all over the world.
Focus on policy relevance
Content moderation classifiers integrate relevant policy categories to establish effective safeguards against potential harms such as unqualified advice and disclosure of personally identifiable information (PII). Mistral AI’s approach to LLM safety is practical, comprehensive, and addresses the nuanced nature of undesirable content in a variety of contexts.
Performance and Collaboration
Mistral AI shared performance metrics, including area under the precision-recall curve (AUC PR) for internally tested policies. The company is committed to contributing to safety advancements in the field of AI by collaborating with its customers and the broader research community to improve and expand its intervention tools.
This release is part of Mistral AI’s ongoing effort to provide lightweight, customizable orchestration solutions that can adapt to the changing needs of the industry.
Image source: Shutterstock