1. How do I use AI Agent Studio?
  2. Add and Configure an LLM Node in a Workflow

Add and Configure an LLM Node in a Workflow

You can add LLM nodes in your workflows to handle tasks that need language understanding or generation within a workflow. They can extract information from unstructured text and turn it into structured JSON, summarize, interpret, or explain information, map data into a request payload, and generate responses, such as confirmations or clarification messages. For example, a purchasing workflow can use an LLM node to extract supplier and item details from a quotation, then pass that output to a Business Object Function node to validate the supplier and create a requisition.

For more information about LLMs in AI Agent Studio, see Large Language Models (LLMs) in AI Agent Studio.

  1. Go to AI Agent Studio and open the Workflows Workflows tab.
  2. Edit a workflow and add an LLM node to it.
  3. Enter a name and description for the node. The code is automatically populated based on the name.
  4. Select an error handler node to define failure handling.
  5. Enable the Answer in user's language option to override the value set in the Chat Experience tab of the workflow's settings.
  6. Enable the Include chat history option to include previous conversation turns when this node runs.
  7. Enable the Enable guardrails? option to instruct the LLM to respond based on grounded enterprise facts and evidence. Guardrail instructions are added to the prompt automatically. You can view the added instructions from the detailed LLM input during debugging runs.
  8. In System Prompt, add instructions to set the model behavior for this node. You can also use context-sensitive expressions in the prompt. For more information, see Expressions.
  9. In User Prompt, enter the the user request or task for the LLM node. You can also use context-sensitive expressions in the prompt. For more information, see Expressions.
  10. From the LLM section, use the default LLM configured for the workflow or select another model and set its parameters.
  11. In the Output section, define the JSON output specification for downstream steps and expression tools. To use natural language for generating the JSON, select Generate, describe the schema, and select Go.
  12. Publish the workflow.

When the workflow runs, the LLM node handles the tasks according to the specified node configurations.