1. How do I use AI Agent Studio?
  2. Run ATLAS Tests

Run ATLAS Tests

Run ATLAS tests to validate workflow behavior, identify failures, and evaluate test results. You can run individual tests, complete workflow suites, or tests selected by tags. Configure the data source, evaluation mode, judging method, and workflow status based on your testing requirements.

Run a Test During Investigation

Run an individual test to investigate specific workflow behavior, identify assertion failures, and review operation details. Use the following prompts with your coding assistant:

Run <test name> for <workflow name> using recorded file data and deterministic checks only.

Show the route, failed assertions, warnings, and individual HTML report.

Use deterministic-only operation for focused investigation. Complete required semantic evaluation before accepting the final result.

Run the Workflow Suite

Run the complete workflow test suite to validate all configured tests and review the consolidated results. Use the following prompts with your coding assistant:

Run the complete ATLAS suite for <workflow name> using recorded file data.

Preserve each test's configured evaluation mode, complete required judging, and show the suite report.

Select Tests by Tag

Run tests associated with specific tags to validate a selected group of test cases. Use the following prompts with your coding assistant:

Run file-replay tests tagged release-upgrade.

Require every requested tag and show the suite report.

Specify whether tests must match all requested tags or any requested tag.

Choose Data Source and Evaluation Independently

Select the data source, evaluation mode, and judging method independently based on the testing requirements. Use the following guidelines:
  • File or live data: Select based on repeatable regression versus current integration validation.
  • Deterministic or semantic evaluation: Preserve the test’s configured evaluation mode for normal suites.
  • Local or remote judging: Request remote judging when required by the test plan or release process.

Choose Draft or Published

Select workflow status at runtime. Ensure to check the resolved code, status, and version in the report. Align local and remote draft content before recording or testing. Selecting published doesn’t publish anything.