How do I make agents respond faster?

AI agent response time is directly linked to the combined length of input and output text, measured in tokens for each transaction. For example, 100 tokens is roughly 75 words, though this ratio varies by model, writing style, and language. By being deliberate with your instructions, you ensure that your agents reason clearly and avoid unnecessary processing.

Here are some recommendations to help your agents respond faster.

• Remove the summarization prompt when complex post-processing isn't required. For simple post-processing, include the instructions directly in the user prompt instead of using a separate summarization prompt. This approach reduces the number of LLM processing steps and improves response time.
• Minimize the use of input and output tokens with concise prompts. Avoid passing the entire output from upstream nodes to downstream nodes. Instead, use an output schema to pass only the data required for the next step.
• Specify response length to avoid overly verbose answers.
• Choose the smallest model that meets your requirements. Use lightweight models for routine tasks and larger models only when additional reasoning or higher quality responses are needed.
• Configure reasoning effort based on task complexity. Use Minimal or Low reasoning effort for routing, extraction, and simple question answering. Reserve higher reasoning effort for complex analysis, as higher settings can increase response time and token usage.
• Use Code nodes for deterministic tasks with predictable outcomes, such as filtering structured data, validating inputs, performing calculations, formatting outputs, and parsing or transforming structured data such as JSON. Use LLM nodes for tasks that require language understanding, interpretation, summarization, classification, extraction from unstructured content, or reasoning.
• Use smaller, specialized agents or workflow nodes to handle distinct tasks, such as research, sending communications, or Q&A. Where possible, execute independent tasks in parallel instead of building long sequential workflows.

By carefully selecting relevant context and breaking complex tasks into smaller, focused agents, you can improve both the speed and efficiency of your AI agents.