XPRESSO

About XPRESSO

What is XPRESSO?

XPRESSO is a comprehensive and complementary network test automation web-based "dashboard" that's designed to maximize, expedite, and simplify your test execution in a fashion that aligns well with the stringent timeline and resource requirements.

XPRESSO provides a unique set of at-a-glance, simple-to-use set of features to help you prepare, schedule, and execute network test automation scripts while providing a comprehensive and user-friendly way to manage and search test results so you can quickly isolate, interpret, and analyze test results.

Why use XPRESSO?

These are the core use cases and benefits of why/how XPRESSO can make your test automation experience easier:

Use Case Scenarios:

  • Fully automates your testing and images/releases certification; compresses your DevOps/NetDevOps cycle time from weeks/months to hours/days:

    • Creating Jobs: Lets you prepares Job Runs in advance of execution by providing the means to associate all required elements to run the Job such as: your test environment (test harness, Execution Engine, and Location-details); your lab resources (testbeds or topologies); and any localized arguments/environment variables.

    • Creating Job Schedules: Allows you automate the execution of Jobs instead of having to manually initiate (kick-off) individual Jobs every time.

    • Creating Reservations: Lets you guarantee your system resources for a specific date, time, and duration in advance of your testing requirements.

    • Testbed Queuing: Provides automatic queuing of Job Runs, ensures efficient use of testbed Resources and provides testbed/devices utilization stats.

    • On-the-fly Job Test Result Comparisons: Allows you to compare the test results from one test run against another and determine the root cause of failures.

    • Verification Testing: Allows you to easily validate new software updates from Cisco prior to upgrading your network.

    • Baselining Job Test Results: Allows you to set the test results of a Job as a baseline in order to establish a starting point for automated test result comparison and analysis of future runs of the same Job.

  • Likewise for release certification, use XPRESSO and its automated tool set to reproduce, test, and troubleshoot networking bug scenarios. XPRESSO has the required plugins that enables integration with the Cisco's CDETS bug tracker; you can automatically attach XPRESSO generated test results as attachments to the bug CDETS.

Other Core Features:

  • Provides advanced result comparison, triage, visualization and analytics metrics and help you quickly isolate, interpret, and analyze test results data.

  • Can be deployed to work in both a private or DMZ'ed lab, or Cisco-hosted lab facilities. You can also run automation tests in two ways: client-side via a Jenkins Engine or cloud-side execution via a Docker Image.

  • Provides maximum flexibility to define your test framework and lab resources to support the creation of new test suites, the execution environment, platforms supported, and the integration of new features in future releases.

  • Designed for agile and rapid development network testing iterations and has the ability to handle a wide variety of testing scenarios ranging from white/black-box unit testing, feature and release deployment testing, performance and scaling testing, and more.

  • Platform, test-methodology and feature/scope independent. You can run any test suite, on any test platform, and with any release.

  • Enforces global and consistent network and security constraints of users in your test environment, thereby not only adding ease-of-security management but also improving the accuracy of application testing and reducing the time to execute tests.

  • Gives you the ability to determine the real-time, proactive at-a-glance readiness of downstream test equipment and the resource operational states prior to executing your test cases. From a reactive perspective, you can also view/use operational state information to help you determine what caused the problem in the event your Job Run ran into an unexpected problem or to troubleshoot any other system-related problems.

  • Provides a holistic view of your test resource usage at any given time to help you manage and maximize your lab resources and optimize your resource allocation and prevents scheduling conflicts. You can also control the configuration state of supported objects to perform basic equipment and resource management.

  • XPRESSO is designed to not only work seamlessly with the pyATS end-to-end test automation ecosystem but also with Tcl-ATS, Jenkins, Docker, LaaS, CDETs, Kafka and more.

What makes XPRESSO Unique?

XPRESSO is not "just" another test automation tool... it is the next generation automation dashboard built to make your test-automation life easier with the emphasis on reliability, a user-friendly and harmonized UI/UX, seamless integration with your test environment and other test tools, ease-of-deployment, modularity and scalability, and advanced security as one of the core fundamental design principles.

Design Principle Details
Architecture
  • Modernized, micro-services based architecture.
  • Easily writeable YAML representations of components (e.g.: testbeds)
  • REST API access
  • CLI access
Transparency
  • XPRESSO can be deployed to support one team or shared among the entire business unit.
Modularity and Scalability
  • XPRESSO supports independent modules, webhook plugins, pluggable apps, Kafka support for external connectivity and much more.
Simple and Harmonized UI
  • Centralized entry for executions, triage, and analytics.
  • A flexible and user-focused web interface.
  • Visualized progress analytics with LiveView dashboard
  • Test result baselines
Seamless Integrations
  • LaaS, Jenkins, VM and more.
  • Trade uploads
  • CDETs reference
Deployment Ease and Reliability
  • Click and run Jobs/Profiles: configure one-time and your sanity/regression test are just a click away
  • Reservations, schedules, and queuing mechanism.
  • Clean operations
  • Geo-location tagging
Underlying Security
  • Automation tokens with prompted expiry.
  • Managed user levels and Groups with explicit permissions control and customization.
  • History tracking

Key Feature Highlights

XPRESSO has the following key features to help you with your test automation needs:

  • Searching in XPRESSO: To simplify and expedite searching through large amounts of test results and to help you isolate specific equipment details, you can use the following search methods/tools to search for information in XPRESSO:

    1. Global Search Tool: Provides a quick, all encompassing method to search most registered objects in the XPRESSO database; search results are limited to key fields in each registered object type.

    2. Searching Localized Registered Objects: Allows you to search for information at a more finite level in each discrete registered object; object details can be further searched/refined using a secondary search field, custom filters, sorting schemes, and table customization.

    3. Custom Tags: You can also create or assign Custom Tags to flag important information with user-defined (personalized) keywords to create a searchable soft link to help you quickly return to flagged information. You can only search for Custom tags in the Global Search Tool.

  • History Tracking: XPRESSO keeps a historical record of all significant events and changes in registered components. History Tracking events and data collected can be to troubleshooting failures or changes by pin pointing details as to when, where, and why.

  • Location Geo-tagging: All resources in your test environment (test harnesses, Execution Engines, testbeds, and topologies) are associated with location-specific metadata (a location code), for example SJC/SJC23. This ensures the test execution resource dependencies align with each other on a given site, as well as enabling Group and System administrators be aware of site-dependencies of their test environment in case of schedule lab maintenance, and other events.

  • Operational & Configuration States:XPRESSO constantly monitors all equipment and resources managed by XPRESSO in to report their current status (state). Two reporting schemes are supported:

    • Operational states: Reports the readiness/availability of test equipment, the transitional states as a Job Run executes, what happens when the Job Run completes, and if a problem occurs, the reason why.

    • Configuration states: Reports the current configuration status of supported objects that can be configured by users. Three configuration states are supported: Enabled (default configuration state), Disabled, and Maintenance. Group Admins (or permitted users) can change the configuration state of XPRESSO Jobs, Resources, and Test Environment to meet their specific testing requirements.

  • Job Scheduling You can use scheduling to control when and how often (daily/weekly/monthly/custom) to execute a single Job or a Job Bundle instead of having to manually initiate (kick-off) the Job Run each time. Job Scheduling provides a single point-of-control mechanism to create a new schedule, track all pending schedules (in tandem with the Calendar), and manage all pending/in-progress/expired schedules as they are created, updated, executed, and managed by XPRESSO.

  • Reserving Testbeds and Topologies: Reservations guarantee your system resources for a specific date, time, and duration in advance of your testing requirements. Booking time slots for utilizing testbeds and topologies is usually allocated by a queue manager on a first-come; first served basis (based on the priority of request). You can create "First Available" or "Pick My Time" Reservations in XPRESSO in advance of your testing requirements to guarantee system resources for a specific date, time, and duration. You can view all configured Reservations on the Calendar or on the Reservations page.

  • Testbed Queuing: XPRESSO actively manages the runtime of registered testbeds by entering all jobs submitted to each testbed. In turn, the Testbed Queues manage testbed time by controlling the prioritization and dispatching of job requests and runs.

  • Comparing Run Results: Comparing the test results from one Run Request to another can help you quickly determine what impact (positive or negative) occurred between two testing scenarios. Along with detailed comparison data that can help you pin-point which change influenced the results, the comparison also provides details for all other metadata and all generated files/logs that were generated as part of the Run Request.

  • Baselining Run Results: You can configure (set) the Job Request as a baseline Job Request in order to establish a starting point for comparison purposes. You can then compare the test results from other Job Requests against the baseline Job Request. Once you establish your baseline Job Profile/Job run, you can use this baseline as a means to:

    • Test new device features to prove their conformance to all functional, performance, reliability, and security requirements. This greatly improves/expedites Cisco Software development workflow by automatically catching regression errors and highlights areas for improvement.

    • Test any proposed changes or pending fixes to your network to perform a verification/sanity check to prove or assess the effectiveness of a specific implementation.

    • Determine the health of your network at a certain point in time or at regular intervals to ensure that your network is working at an optimal level.

    • Provides the capability to track the important events via Notifications,Messages,Actionable Requests and Webex Bot Notifications

  • Devat Scripts:

Next