Felix Pinkston
December 6, 2024 06:02
LangSmith SDK v0.2 introduces simplified evaluation methods, improved performance, and improved documentation, significantly improving the developer experience across Python and TypeScript.
In a recent update, LangChain announced the release of LangSmith SDK version 0.2, which aims to improve the developer experience in evaluating applications. According to LangChain, this release introduces several key improvements, including simplified use of evaluation methods, performance improvements, and expanded documentation.
Simplified Evaluation Methodology
The update simplifies use. evaluate() and aevaluate() methods. Developers can now take advantage of a simpler evaluator that allows functions to directly score application output without complex argument structures. This change applies to both Python and TypeScript SDKs, streamlining the process for developers.
Additionally, the SDK now supports direct evaluation. langgraph and langchain Further simplifies the evaluation process. This feature allows developers to pass these objects directly to the evaluation method, reducing the steps required to evaluate application performance.
Integrated evaluation function
In the new version, LangSmith consolidated the evaluation methods and reduced them to a single one. evaluate() method. The method now includes features previously spread across multiple methods, making it easier for developers to run applications on datasets, evaluate existing results, and perform comparative evaluations.
Local evaluation and performance improvement
LangSmith SDK v0.2 introduces a beta feature that allows you to run assessments locally without uploading results. This is especially useful for developers who want to quickly test a prompt or verify functionality without recording an evaluation. However, this feature is currently only supported in the Python SDK.
This update also includes performance improvements to the Python SDK, resulting in approximately 30% speedup. aevaluate() For large cases, the efficiency of the evaluation process is significantly improved.
Improved documentation
The LangSmith team overhauled the documentation, rewrote most of the assessment guides and introduced new ones to bring them in line with the latest updates. This revamp includes a unified Python SDK API reference that aims to provide developers with a comprehensive and accessible resource.
Breaking changes in the Python SDK
This update introduces several major changes to the Python SDK, including modifications to the default concurrency settings and dataset identification process. Additionally, support for Python 3.8 has officially been discontinued after support ends in October 2024.
LangSmith SDK v0.2 provides a simpler, more efficient experience for developers by simplifying the evaluation process, improving performance, and providing improved documentation, making it an invaluable tool for software development in both Python and TypeScript environments.
Image source: Shutterstock