Review ATLAS Test Reports

Review ATLAS test reports to verify test outcomes, investigate failures and warnings, and determine whether the evaluated workflow or model meets the required checks.

Choose a Report

Select the appropriate report based on the test results or evaluation details you want to review. Use the following guidelines:

• result.html: Why did a scenario pass, fail, warn, or require judging?
• suite-result.html: Is the workflow, application, or selected suite ready?
• comparison.html: How did the candidate model differ from the baseline?
• sweep-result.html: Which evaluated model placements are recommended or rejected?

Use HTML for human review, JSON for automation, and Markdown for concise sharing.

Review Results in Order

Review the test results in the following order to verify correctness, identify issues, and evaluate performance.

1. Confirm the target, inputs, data source, and run label.
2. Compare expected and actual routes.
3. Inspect failed checks and warnings.
4. Check replay-data origin and readiness.
5. Review the response contract and semantic findings.
6. Examine consumption and latency after establishing correctness.

Interpret the Result

Review the test result status to determine whether the required checks succeeded or further action is needed. Use the following guidelines to interpret each status and identify the appropriate next steps:

• Passed: All required checks succeeded and required judging is complete. Review coverage and retain evidence.
• Failed: Operation or a required check failed. Investigate the individual report.
• Warning: A concern requires review. Determine whether corrective action is needed.
• needs-judge: Required semantic evaluation remains incomplete. Complete the judging process or attach the judging results.
• needs-model-test-data: Replay data is incomplete. Prepare the required data.

A plausible final answer alone doesn’t establish success. Verify that the workflow reached the expected route and met the required checks.