XPRESSO

Configuring and Executing Job Requests

About Configuring and Executing Job Requests

When you save a Job in XPRESSO and the Job is submitted for execution and queued (therefore, the job is set to “Run DEFAULT” on the Registered Jobs page), the Job is now considered to be a "Job Request" (also known as a Run Request and is truncated to just "Request" on the XPRESSO dashboard).

You do not directly create Job Requests via the XPRESSO main menu; they are created as a result of performing a "Run" request. You can however, configure additional parameter settings with the Job Request to further define the test environment your Job will be executed in.

For example, you can select/de-select which testbeds to use, specify your clean file requirements and Job/Environment/Harness arguments, and configure optional parameters such as establishing a baseline test run or running your tests on multiple testbeds.

Job Requests can be run individually, in Job Groups, or in Job Bundles. When XPRESSO creates a Job Request, they are registered in XPRESSO and saved with a unique ID for identification and tracking purposes.

The Job Request executes when the testbed resources become available or when the Job is scheduled to run. Once the Job Request completes the run, it contains your test results.

You can optionally view job execution statistics in real-time using LiveView when a Job Request is running.

Job Requests Attributes

Job Requests have the following attributes:

  • Each Job Request is assigned a unique Request ID in the format of REQ########.

  • A Job Request may contain one or more jobs/profiles.

  • Each Job Request indicates which testbeds to be queued against.

  • All Job Requests have a maximum lifetime (e.g: best-before date). If the Job Request has not been fully completed by the value specified in this parameter, all remaining Job Request executions are terminated.

Job Request Events

Once a Job Request is submitted for execution, it transitions through the following events:

  1. Request is queued against the specified testbeds (with the defined environments and parameters associated with the job).

  2. The Job Request is run/executed as dispatched by the priority queue.

  3. Results are collected.

  4. The Group Member who submitted the Job Request and interest list are notified.

Ways to Configure and Execute a Job Request

You can run a Job Request in the following variations:

  • As a single Job Request: see the next sub-topic for details.

  • As a Group Run: Are created on-the-fly and cannot be saved; see the "Running a Job Group" sub-topic in "Creating XPRESSO Jobs" for more information.

  • As a Group Bundle Run: Likewise to Job Groups, Job Bundles allow multiple Jobs to be selected so they can be run concurrently; the key difference is that Job Bundles can be saved. See "Creating Job Bundles" for more information.

Configuring and Executing Single Job Requests

This is the quickest way to run a single job/profile combination. You are notified when the Job executions is complete and the test results are available.

To Configure and Execute a Single Job Request:

  1. From the Main Navigation Bar, choose Main Menu→EXECUTION→Jobs to open the Registered Jobs page.

  2. Locate and highlight the Job that you want to run a single Job Request for and click the Run DEFAULT floating action button located to the right of the Job. The New Job Request wizard opens.

  3. Configure each wizard step as required. For each new Job Request, you can configure (or reconfigure in some cases), the following options and parameters. All of the parameters in these steps are optional.

  4. Step 1 - Request Options: Allows you to set the following parameters:

    • Branch: Select the branch image that is required for this Job run (required if the Job Request is used as a baseline (now or in the future)).

    NOTE:
    When you designate this Job Request as a baseline, you must associate a Branch identifier with the baseline in order to track comparison results/create a usable baseline. This is required because a single Job Request can be used to compare against many Job Requests; the data results from each test run needs to be mapped to the specific Job Request.

    • Label: If required, enter a Label for the branch. Always add a label if the Job Request will be used as a baseline (now or in the future). Labels ensure proper classification of baseline data and is helpful if you are maintaining a large database of baselines.

    • Description: Provide a meaningful description to describe the purpose of the Job Request to assist with identifying and locating your Job Request.

    • Request Priority: Set the override priority for all jobs/profiles associated with this Job Request.

    • Interest List: Specify which users the Job results to be carbon copied (Cc) when the job Request completes.

    • Tags: As required, enter a tag(s) (keyword category) to create a searchable soft link to help you quickly find data within the Job Request using the Global Search tool.

    • Maximum Request Lifetime: Specify the maximum time limit to wait for the Job Request to finish running after which the Job results become irrelevant. The Job Request is aborted if the Job Run does not completely execute within the specified time frame.

    • Baseline (checkbox): Allows you to set the test results of this Job run as a baseline for comparison purposes. (OR)

    • Conditional Baseline (checkbox): Allows you to set the test results of this Job run as a baseline if the results meets or exceeds the conditions set in the Baseline Criteria parameter (see the Profile Options below in step 3 for condition details).

      NOTE:
      If the Conditional Baseline parameter is enabled and the Baseline Criteria parameter (see the Profile Options below in Step 3) has the "Match or Improved Compared to Baseline" parameter also set to enabled, a baseline comparison is run against the selected comparison option or latest baseline that matches with the branch and label.

    • Compare to Baseline: Specify which baseline to compare the test results from this Job Run against; two options are available:

      • Baseline of your choice: If you specified a Label with the Job Request, select an existing baseline image from the dropdown list.
      • Latest: If you did not provide a specific baseline to compare with, XPRESSO uses the latest baseline that matches with the branch and label (if provided) of your Job Request to perform the baseline comparison.

    • CDET ID(s): As required, add Cisco Defect and Enhancement Tracking System IDs to be referenced to the job run. When adding CDETS:

      • Type in the "Defect ID" or the comma separated list of defect IDs under the CDETS section using this format: CSCtu31082, CSCtu54321.
      • Click the Add icon button that appears (or simply press enter button in your keyboard) to save.
      • Follow the same steps to add "CDETS Attributes" if necessary.
      • "Enable CDETS Attachment" if you need to get the DDTs included in your job results. This will be auto-enabled if your group admin has enabled CDETS attachments in Group default preferences.

  5. Step 2 - Profile (informational): Provides a summary of the associate Job Profiles assigned to a Job Request. This summary is view only; you cannot change any parameters in this step. To change any parameters on this page, return to the Job Details page, select the Job Profile and choose Edit.

  6. Step 3 - Profile Options: Allows you to set the following parameters:

  • Execute on all Testbeds (checkbox):

    • (Off position): When you performed a Job Run, XPRESSO executes your Job Run on the first available testbed defined in your Job Request.
    • (On position): When you performed a Job Run, XPRESSO executes your Job Run on all testbeds defined in your Job Request. Job execution is still subject to the testbed availability and priority queues.

  • Baseline Criteria: Allows you to set the condition(s) for the test results for this Job run to be marked as a baseline based on the following criteria:

    1. The overall percentage of passed test cases.
    2. The minimum number of passed test cases.
    3. If the test results match or have improved when compared to the latest baseline.

    NOTE 1:
    Baseline criteria are cumulative; all set criteria must be met or exceeded for a Job Run to be marked as a baseline.

NOTE 2:
You can set multiple conditions for the Job Run. To ignore a condition, leave the parameter field blank.

  • Pass Rate (%): Allows you to set the overall percentage of passed test cases required to mark the test results as a baseline.
  • Min. No. of Passed TCs: Allows you to set the minimum number of passed test cases (TCs) required to mark the test results as a baseline.
  • Match or Improved Compared to Baseline: Mark the test results as a baseline if the test results match or have improved when compared to the latest baseline.
  • Webhooks
    • Enable Webhooks: Allows you to choose a Webhook from the pulldown menu to apply to the Job Run. You can apply multiple Webhooks to a Job Request if required.

  1. Step 4 - Resources: As required, select the required testbed or topologies resources to meet your Job Run requirements and execution engine type. The following rules apply to the Resources parameters in this step:

    • You can only access these options if you previously configured the associate attributes (testbeds or topologies) when you created the Job.

    • If you have configured a number of testbeds or topologies to use for the Job, you can select or de-select any objects to include/exclude from your Job Run.

    • If you only configured one of the resources for your Job Run (for examples, testbeds), this information is accessible via this step and can be selected or de-selected. To add other resources (including any new testbeds), return to the Job Details page, select the Job and choose Edit. Add the appropriate resource as required. When you run your Job, the other resources are accessible on the New Job Request page and you can select or de-select them as required.

  2. Step 5 - Clean: As required, configure the testbed clean options if a corresponding clean YAML is associated with a testbed. Make sure your clean YAML is valid and in accordance with the pyATS clean schema.

  3. Step 6 - Arguments: As required, configure the following custom command line arguments/variables to be set when running this job file; you can configure multiple arguments by clicking the + Add string.

    • Job Arguments
    • Environment Variables
    • Harness Arguments: Select the pyATS specific arguments to set for this job request.
    • Cloud Mount Points

Job Request States

Once a Job Request is submitted to run, its stays in the priority queue until one of the testbeds/topologies it was queued on becomes free/available (and is also subject to when it is scheduled to run). Once the Job starts running, you can view its current status via the Job Request page.

A Job Request displays its current state to give you a real-time update as it transitions from start to finish. Job Request States are defined in the following table:

State Definition
Preparing: Indicates your Job Request is communicating with the system and related instances.
Queuing: Indicates your Job Request is being added to the testbed queue.
Queued: Indicates:
  • On submission, all jobs stay in the priority queue and are subject to the testbed queuing rules.
  • If a job is submitted to be queued on multiple testbeds at the same time, the first available testbed will be used.
  • Jobs will stay in the priority queue until its maximum queue retention period is reached.
  • While a request is in queue, its run priority can be modified.
Running: Indicates:
  • Each job/profile is limited to a maximum runtime (defined by its Job profile).
  • You can access a LiveView of the Job Run (available on the Job Request details page) that provides detailed information of how the Job Run is progressing.
  • Stop Action: You can request to stop a Job Request while it is running. You will be prompted to provide a reason as confirmation for a stop.
Passed: If the Job execution completes with passed statistics, you can view the execution results on the Job Request detail page. The Job Request statistics may include:
  • Testbed it ran against.
  • List of all arguments used.
  • Results and logs.
  • Archived files (including downloads).
  • Jenkins console log (if available).
Failed: Indicates that the Job execution completed but contains elements that contains incomplete statistics. You can view the reason(s) for the failure on the Job Request/Job Results detail page.
Stopped: Indicates that the Job Request/Job Results was stopped during the Job Run. You can view the reason for the stoppage on the Job Request/Job Results details page. You can view any execution statistics recorded before the stop if applicable or available.
Errored: Indicates that an error occurred with the Job Request/Result during a Job Run. You can view the reason(s) for the error on the Job Request/Job Results detail page. You can view any execution statistics recorded before the error if applicable or available.
Expired: Indicates that the Job Request could not be completed within the defined maximum Job Request lifetime/maximum runtime or could not be dispatched within the maximum queue retention lifetime. When the Job Request Expires, it will be terminated or cleared off the priority queue.
Next