You can configure/manage the following objects to define or describe your test framework to support the creation of new test suites, the execution environment, the supported platforms, and the integration of new features in future releases; this ensure maximum flexibility of the system:
Docker Images: Docker is a set of platform as a service (PaaS) products that uses OS-level virtualization to deliver software in packages called containers. You can use a Docker Image to add (register) existing Docker Images or build new Docker images to provide a virtual equivalent of a test execution environment to execute Job Runs in the Cloud. This eliminates the need for a physical (Jenkins) execution engine or test harness; the Cloud-equivalent test execution environment provides a portable, stable, and reproducible (controlled) testing environment.
Test Harnesses: In XPRESSO, a test harness defines the test framework properties (name, type, path, location, supported features) for the Python script execution environment associated with your pyATS instance or Tcl-ATS tree. This topic describes how to create a new test harness instance and which test harness types are supported.
Execution Engines: Execution engines host the actual run of each job on a server using the provided Test Harness, then waits for the run to complete and submits the results back to XPRESSO. XPRESSO supports the following Execution engines:
Jenkins Execution Engine: The default execution engine in XPRESSO is Jenkins which serves as the workflow engine to manage/provide a CI/CD solution/process to execute each XPRESSO Job Run on a server using the provided Test Harness. Jenkins is a self-contained, open source, orchestration and automation execution engine (server) used to automate the various tasks related to the building, testing, delivery, and deployment of software.
Cloud Execution Engine: In XPRESSO, you can run automation tests in two ways: client-side (via Jenkins) or cloud-side execution. You can use a Docker Image to add (register) existing Docker Images or build new Docker images to provide a virtual equivalent of a test execution environment to execute Job Runs in the Cloud.
Branches: In XPRESSO, a Branch is a parallel instance of an OS revision. Each branch can have multiple builds associated with it; each build in turn can be tagged with a unique metadata label to implicitly identify it. Branches form part of a build/release management technique; the goal of this methodology is to ensure significant builds can be validated to ensure image stability by executing a given set of Job Runs on your testbed (with or without a clean file) against the image.
Platforms: When XPRESSO is installed, it comes pre-populated with most of the prevalent Cisco platform types registered in XPRESSO in anticipation of deployment to meet the testing requirements of most users. In the event your Platform type is not registered in XPRESSO, you can make a request to the Group Administrator to add it. This allows you to associate/apply Platform metadata with your Job Runs to assist with tracking purposes and to make working with test results easier.
OS: When XPRESSO is installed, it comes pre-populated with the most used/prevalent device/platform Operating Systems (OS), for example: aireos, ios, ioxr, and linux in anticipation of deployment to meet the testing requirements of most users. In the event your OS type is not registered in XPRESSO, you can make a request to the System Administrator to add it. This allows you to associate/apply OS metadata with your Job Runs (via the applicable Branch or Platform) to assist with tracking purposes and to make working with test results easier.
Locations: All resources in your test environment (test harnesses, Execution Engines, testbeds, and topologies) are tagged with location-specific metadata (a location code), for example SJC/SJC23, to prevent Users from making geo-location operational and testing errors. This ensures your site and resource dependencies align with each other to ensure smooth and accurate operations within XPRESSO. This topic describes why location geo-tagging is important when your work with XPRESSO, how to confirm your location details associated with your test site is registered in the XPRESSO database, and how to request to register a location.
Webhooks: Webhooks (also called a web callback or HTTP push APIs) allows XPRESSO to provide third-party applications with custom callbacks so it can deliver real-time information as it updates. This differs from typical APIs where you need to poll for data frequently to get real-time information. Some Webhooks allow third-party applications to reply to XPRESSO, and can modify its behavior in some way. All Webhooks extended through XPRESSO are maintained by their third-parties.