Creating Job Profiles

About Job Profiles

Test automation suites need to be flexible, dynamic, and give Users the ability/means to facilitate/conduct a wide-breath of different test scenarios. XPRESSO meets this requirement by adding an additional layer on top of the Job layer in order to perform varying degrees of tests/coverages using the same pyATS test script but with different parameters/arguments/environment variables. This layer is called a Job Profile.

A Job Profile contains the metadata information associated with a Job that defines "how" to run your Job; information that is specified via the Job Profile includes:

• Your test environment (test harness, Execution Engine, and location-details).
• Your lab resources (testbeds or topologies).
• Any localized arguments/environment variables.
• You can also optionally set:
  • The condition(s) for the test results for a Job run to be marked as a baseline Job Run.
  • For Jenkins Job Profiles; specify the path and file to the Easypy Configuration YAML File to be used.

Job Profile Attributes

Job Profiles have the following attributes:

• Job Profiles define a job environment with precise arguments required to be launched.

• A single job can be registered with multiple Job profiles to run on/in different environments.

• At a minimum, a Job is always associated with a DEFAULT Job profile when you add a new Job to XPRESSO. This profile has the following attributes.

  • Is always named DEFAULT.
  • Created automatically during job registration.
  • The Default Job profile is used whenever a job is run without explicitly selecting a Job profile to run with.

• You can create or clone additional Job profiles and associate them to your Job so you can run your Job in different testing environments.

• When you create a Job Profile, they are registered in XPRESSO and saved with a unique name for identification and tracking purposes.

• Job Profiles are located under each registered Job on the Job details page.

• You can view current profiles under a job on the Jobs page.

Creating a New Job Profile

There are two methods to create a new profile for a given job:

• Create a new Job Profile
• Clone an existing Job Profile

To Create a New Job Profile:

1. From the Main Navigation Bar, choose Main Menu→EXECUTION→Jobs to open the Registered Job page.

2. Locate and double-click on the Job that you want to create the new Job Profile for. The Job Details page opens.

3. Choose one of the following:

   1. Click the Add Profile button at the top-right of the Job Details window to create a new Job Profile. The Adding New Profile wizard opens; the wizard parameters fields in this case are empty.
   2. Click the Clone floating action button located to the right of the highlighted Job. The Adding New Profile wizard opens; the wizard parameters fields in this case are pre-populated with the same values as the Job Profile it was cloned from.

4. Add or change any pre-existing parameters values if cloned each wizard step as required.

   NOTE:
   Job Profiles can be created for both Jenkins Jobs and Clouds Jobs; note the differences between them (the differences are noted in each step): (1) the required steps in two wizards are similar (Jenkins Job Profiles have one additional step) (2) some of the parameters in each step are unique to the specific execution engine type.

5. Step 1 - Profile Info:

   • Profile Name: Provide a unique identifier for the Job Profile.
     
   • Description: Provide a meaningful description to describe the purpose of the Job Profile.
     
   • Execution Engine: (Jenkins-Only) Select which Jenkins execution engine to run the Job Profile on.
     
   • Harness Instance: (Jenkins-Only) Select which test harness to use for the Job Run.
     
   • Supporting TCL Harness Instance (Jenkins-Only and optional): Select the appropriate TCL
     Harness Instances; only required if you intend to connect Tcl-ATS trees.

6. Step 2 - Baseline Criteria: Allows you to set the condition(s) for the test results for this Job Profile to be marked as a baseline based on the following criteria:

7. The overall percentage of passed test cases.

8. The minimum number of passed test cases.

9. If the test results match or have improved when compared to the latest baseline.

>**NOTE 1:** <br>Baseline criteria are cumulative; all set criteria must be met or exceeded for a Job Run to be marked as a baseline.<br>

NOTE 2:
You can set multiple conditions for the Job Run. To ignore a condition, leave the parameter field blank.

• Pass Rate (%): Allows you to set the overall percentage of passed test cases required to mark the test results as a baseline.

• Min. No. of Passed TCs: Allows you to set the minimum number of passed test cases (TCs) required to mark the test results as a baseline.

• Match or Improved Compared to Baseline: Mark the test results as a baseline if the test results match or have improved when compared to the latest baseline.

7. Step 3 - Profile Arguments: As required, configure the following custom command line arguments/variables to be set when running this job file; you can configure multiple arguments by clicking the + Add string.

   • Job Arguments
   • Environment Variables
   • Harness Arguments: Select the pyATS specific arguments to set for this job profile.
   • Cloud Mount Points

8. Step 4 - Testbed/Topologies:

   • Testbeds: Select the testbed(s) on which the Job Profile should be run against; (OR)

   • Topologies: Select the topology (or topologies) on which the Job Profile should be run against.

     NOTE 1:
     You can only assign testbeds/ topologies that are located in the same site/building as the selected execution engine and harness instance assigned to the job.
     NOTE 2:
     You can select multiple testbeds or multiple topologies but not both.

9. Step 5 - Configuration YAML: (applies to Jenkins Job Profiles only)

   • (Optional) Copy/paste (and if required, edit) the Easypy Configuration YAML File to be used.

10. More Job Profiles options: You can also optionally configure the following parameters for Job Profiles as required to meet your specific testing requirements:

• Maximum Runtime: Specify the maximum time period the job/profile should be completed within. If not completed within the maximum runtime, the job will be either terminated or allowed to expire.

• Priority: Defines the priority of your job run based on the testbed queueing rules.

• Support Mailer: Enter/update the email address used for the destination address for those users who are notified when the Job completes.

• Interest List: Specify which users the Job results to be carbon copied (Cc) when complete.

• Restrict the execution on Jenkins Node: (Jenkin-only) XPRESSO Jobs are executed on the master Jenkins execution engine and any connected slave nodes by default. This parameter allows you to designate which slave Jenkins execution nodes should be used for offloading instead of the default mode of executing to any slave nodes that have less resources or test cases waiting to execute.

  NOTE:
  You must enter the exact name(s) of the alternate Jenkins execution node(s) as used (defined) within the Jenkins dashboard. Names associated with a single slave node cannot have spaces in them. Names associated with multiple slave nodes must be separated by a space. Avoid special characters with slave node names such as these: !&|<>(), as other Jenkins features allow for defining label expressions.

• Coverage Type: Allows you to tag or categorize the type of test the Job profile applies to, for example, Sanity test, Performance test, or Regression test.

  NOTE:
  The Coverage Type option provides a method to append a test-type label to the Job details; XPRESSO does not implicitly do anything differently based on the selected test-type label. The label however, can be useful when searching for and analyzing test results returned by the Job execution.

Post-registration Job Profile Operations

Group Admins can perform the following post-registration management operations with Job Profiles after they are created:

To Perform Post-registration Operations on a Job Profile:

1. From the Main Navigation Bar, choose Main Menu→EXECUTION→Jobs to open the Registered Job page.

2. Locate and double-click on the Job that contains the Job Profile that you want to perform the post-registration operation on. The Job Details page opens.

3. Locate and highlight the Job Profile that you want to perform the post-registration operation on and click the appropriate floating action button located to the right of the Job Profile.

   • Quick Run: Provides the launch point to configure a new job request via the New Job Request wizard for each Job Profile. See "Configuring and Executing Job Requests" for more information.

   • Schedule. Provides a launch point to schedule a Job Profile via the New Schedule wizard. See "Job and Job Bundle Scheduling" for more information.

   • Edit: Allows you to edit all pre-configured parameters on the Job Profile page with the exception of the Profile Name parameter. To change the Profile Name parameter, delete the Job Profile and create a new one.

   • Clone: Allows you to replicate an existing Job Profile. See "To Create a New Job Profile" procedure above for more information.

   • Delete: Allows you to delete the Job Profile from the XPRESSO database.

     NOTE:
     You cannot delete the DEFAULT Job Profile; all other Job Profiles can be deleted.

   • Disable: Allows you to temporarily disable a Job Profile from being used by other Group Members.

   • Maintenance: This state has the same attributes as a Job Profile in a Disabled state except Group Admins and System Admins are able to use them to troubleshoot or leverage them in emergency cases.

   • Enable: Allows you to re-active a Job Profile if the Job Profile was disabled or place into a Maintenance state.