Many customers would like to start automating their testing. In this PoV, we are going to show how to use CML and pyATS for automated testing. In particular, we are going to showcase three automated testcases. In the first testcase, we test the reachability of the devices. In the second testcase, we compare the changes with the baseline through parsing show
commands and in case of changes, the testcase fails. In the last testcase, we parse a particular show
command and check that it satisfies the requirements that we have set.
In order to use pyATS, we have to create a testbed file, which contains the information on how to connect to the devices. The testbed file will include include the following elements:
An example testbed file has been provided in testbed_example.yaml
. If you want to read more on how to create a testbed file, then consult the documentation in the following link:
We have also created a separate file with the commands that you would like to test and the destinations for reachability. For the access devices, we have the following commands:
# Step 1: Specify the commands that you would like to test on the access devices access_commands = [ "show ip arp", "show ip interface brief", "show mac address-table", "show version", "show inventory", "show running-config", "show boot", "show ip route", "show ip ospf neighbor", "show ip ospf database" ]
And for the core devices, we have the following commands:
# Step 2: Specify the commands that you would like to test on the access devices core_commands = [ "show ip arp", "show ip interface brief", "show mac address-table", "show version", "show inventory", "show running-config", "show boot", "show ip route", "show ip ospf neighbor", "show ip bgp summary", "show ip bgp neighbors", "show ip ospf database" ]
We can change these commands in accordance with the test plan that you would like to create. Afterwards, we create a dictionary with the devices as a key and the commands as the value:
# Step 3: Create a dictionary with the devices as a key and the commands as the value device_commands = {}device_commands['access1'] = access_commands device_commands['access2'] = access_commands device_commands['core1'] = core_commands device_commands['core2'] = core_commands
And we specify the devices that we would like to test:
# Step 4: Specify all the devices that you would like to test in your topology devices = ["access1", "access2", "core1", "core2"]
Last, we have also included the destionations that you would like to test for reachability
# Step 5: Specify the destinations in a list that you would like to test for reachability destinations = [ [""], #access1 [""], #access2 [""], #core1 [""] #core2 ]
And add the testbed file name for the script that obtains the baseline:
# Step 6: Specify the testbed filename testbed_filename = ''
The following commands are executed in the terminal.
Create and activate a virtual environment for the project:
#WINDOWS:
$ py -3 -m venv [add_name_of_virtual_environment_here]
$ [add_name_of_virtual_environment_here]/Scripts/activate.bat
#MAC:
$ python3 -m venv [add_name_of_virtual_environment_here]
$ source [add_name_of_virtual_environment_here]/bin/activate
For more information about virtual environments, please click here
Access the created virtual environment folder
$ cd [add_name_of_virtual_environment_here]
Clone this repository
$ git clone [add_link_to_repository_here]
Access the folder gve_devnet_pyats_config_diff_comparison
$ cd gve_devnet_pyats_config_diff_comparison
Create a testbed file. For an example, please see the testbed_example.yaml
Update the env_var.py
file in accordance with your testplan and add the devices from your topology.
Install the dependencies:
$ pip install -r requirements.txt
We obtain a baseline with the script get_baseline.py
$ python get_baseline.py
Make changes to the devices, e.g., update the firmware of the routers
Perform an automated test using pyATS. We will automatically go through the pre-defined testcases.
$ easypy test_scripts/compare_changes_job.py -testbed_file [add_name_of_testbed_file]
View the logs
$ pyats logs view
In this PoV, we will follow the following workflow:
Below is a screenshot of the output of an automated test with pyATS:
And below there is a screenshot of the pyATS GUI, where you can see the results of the current and previous tests:
Provided under Cisco Sample Code License, for details see LICENSE
Our code of conduct is available here
See our contributing guidelines here
Please note: This script is meant for demo purposes only. All tools/ scripts in this repo are released for use "AS IS" without any warranties of any kind, including, but not limited to their installation, use, or performance. Any use of these scripts and tools is at your own risk. There is no guarantee that they have been through thorough testing in a comparable environment and we are not responsible for any damage or data loss incurred with their use.
You are responsible for reviewing and testing any scripts you run thoroughly before use in any non-testing environment.
This solution provides scripts to automate the testing by using pyATS. We create a baseline, by taking an initial snapshot, and then after our changes, we take another snapshot and we compare it to the initial snapshot. Afterwards, we generate a report, which highlights the differences and the results of the tests.
Provide links to related white papers:
Provide a link to a related DevNet Sandbox:
Provide links to related Learning Labs or modules on DevNet:
Provide links to related solutions on DevNet Ecosystem Exchange:
Owner
Contributors
Categories
Products
Cisco Modeling Labs (CML)pyATSProgramming Languages
PythonLicense
Code Exchange Community
Get help, share code, and collaborate with other developers in the Code Exchange community.View Community