XPRESSO

Comparing Run Results

This topic provides information about comparing test results from two different Run Requests with no baseline set.

See "Baseline and Baseline Comparison" for information about comparing test results when a baseline is set.

Also refer to this table which outlines the use cases for both types of comparison features: baseline runs vs. no baseline runs.

Why Comparing Run Results Matters?

Comparing the test results from one Run Request to another can help you quickly determine what impact (positive or negative) occurred between two testing scenarios. Along with detailed comparison data that can help you pin-point which change influenced the results, the comparison also provides details for all other metadata and all generated files/logs that were generated as part of the Run Request.

Also refer to "Why Setting a Baseline Matters?" for additional information about why comparing test results matters.

Comparing Job Results

The following conditions are required to compare test results:

  • You can compare the results of the same Job and Profile (they must have the same name).

  • Both Job Requests must contain test results.

  • The results must have successfully completed (for Aborted/Errored Run Requests, a best-effort to compare is attempted; if no useable results are found, no comparison is made).

To Compare Job Test Results:

  1. From the Main Navigation Bar, choose Main Menu→EXECUTION→Request menu to open the Requests page.

  2. Locate the two Job Requests that you want to compare and click the checkbox adjacent to the Job Request ID column to mark them.

  3. Click the Compare icon button located at the top-right side of the Request page.
    When the comparison details are complete, proceed to the Results Comparison page showing the summary of the two test runs.

  4. To see the differences between the specific test results, hover over a specific Job Profile parameter to query/compare and select the Testcase Diff floating action button on the right-side of the Results Comparison page.

The specific test results differences display in a side-by-side comparison format. You can scroll up or down as required or use the Quick Navigator on the right-side of page to locate specific differences.

Next