Exploring UX for AI Agents: Chat Interfaces

At Sequoia’s AI Ascent conference in March, the LangChain blog highlighted three important limitations for AI agents: planning, UX, and memory. The blog has now embarked on a deeper exploration of these issues, starting with the user experience (UX) of the agent, with a particular focus on chat interfaces. This in-depth discussion is divided into three parts, the first of which is dedicated to chat, thanks to insights from LangChain’s founding engineer, Nuno Campos.

Streaming Chat

The “streaming chat” UX has emerged as the most dominant interaction pattern for AI agents. Examples of this format include: ChatGPTStream your agent’s thoughts and actions in real time. While seemingly simple, streaming chat offers several advantages.

It removes the barrier between the user and the LLM by interacting directly with the language model (LLM) primarily through natural language. This interaction is similar to early computer terminals, providing low-level direct access to the underlying system. While more sophisticated UX paradigms may be developed over time, the low-level access that streaming chat provides is beneficial, especially in the early stages.

Streaming chat also enhances transparency and understanding by allowing users to observe LLM’s intermediate actions and thought processes. It also provides a natural interface for LLM to correct and guide, leveraging users’ familiarity with repetitive conversations.

However, streaming chat has its drawbacks. Existing chat platforms like iMessage and Slack don’t natively support streaming chat, making it difficult to integrate. It can also be awkward for long-running tasks, as users may not want to wait and watch for an agent to do its job. Plus, streaming chat typically requires human intervention, keeping users in the loop.

Non-streaming chat

Non-streaming chats, although seemingly old-fashioned, share many features with streaming chats. They allow direct interaction with the LLM and facilitate natural corrections. The main difference is that responses are received in batches, so the user is unaware of the ongoing process.

This opacity requires trust, but it also allows for delegation of tasks without micromanagement, as Linus Lee emphasizes. It is also better suited for long-running tasks, as users do not expect immediate responses, and it is consistent with established communication norms.

However, non-streaming chat can lead to issues such as “double texting,” where a user sends a new message before the agent has finished the task. Nevertheless, it integrates more naturally into existing workflows, as people are accustomed to text and AI can easily adapt to it.

Is there anything else besides chatting?

This blog post is the first in a three-part series that suggests there are many more UX paradigms to explore beyond chat. Chat is still a very effective UX due to its direct interaction and the ease of asking follow-up questions or making corrections, but as the field evolves, other paradigms may emerge.

In conclusion, both streaming and non-streaming chat offer unique advantages and challenges. Streaming chat provides transparency and immediacy, while non-streaming chat matches natural communication patterns and supports longer-term tasks. As AI agents continue to evolve, the UX paradigms for interacting with them are likely to expand and diversify.

For more details, please visit the original post on the LangChain blog.

Image source: Shutterstock