- Overview
- Product Documentation
- CML 2.3 Release Notes
- CML 2.3 Installation Guide
- CML 2.3 User Guide
- CML User's Guide
- Overview of CML 2.x
- Using CML and the HTML5 UI
- Dashboard
- Workbench
- Adding Nodes to a Lab
- Starting, Stopping, and Wiping Nodes
- Deleting Nodes
- Creating Links
- Rules for Creating Links and Interface Overprovisioning
- Adding Interfaces and Overprovisioning
- Overprovisioning Interfaces with Link Creation
- Hiding Links
- Starting Simulations
- Connecting to a Node's Console
- Changing Global Console Settings
- Download the Console History
- Setting CPU limit on node
- Launch sequencing and CPU limiting
- Stopping Simulations
- External Connectivity for Simulations
- Link Packet Capture
- Lab Sharing
- Console Server
- Breakout Tool
- Custom VM Images
- Creating a New Node Definition
- CML 2.3 Admin Guide
- Resources
CML 2.3 Release Notes
Cisco Modeling Labs (CML) is a network simulation platform. CML 2.3.1 is the latest release of CML, and these release notes include information for all CML 2.3.x maintenance releases.
New and Changed Information
This table shows any published changes to this CML 2.3 release notes document.
Date | Changes |
---|---|
March 14, 2022 | Initial version for the CML 2.3.0 release. |
March 21, 2022 | Adding known issue about VMware Workstation/Player and the CML 2.3.0 VM's memory. |
March 28, 2022 | Adding a workaround for the CML 2.3.0 VM's memory usage to Known Issues. |
April 09, 2022 | Adding Known Issue about CML 2.3.0 compatibility with Apple Mac OS X Big Sur and above. |
April 27, 2022 | Updated bug fixes and known issues for CML 2.3.1 release. |
May 9, 2022 | Added new known issues for link hiding and IOSvL2 port-channels. |
May 27, 2022 | Added known issues for Breakout Tool version compatibility and for link conditioning and VLAN tags. |
Installing or Migrating to CML 2.3
Upgrading to CML 2.3.1
If you already have a CML 2.3.0 installation, you can upgrade it to CML 2.3.1. See the In-Place Upgrade page for upgrade instructions. In-place upgrades from CML version 2.2.3 or earlier are not supported.
Migrating from an Existing CML 2.2 Installation
It is not possible to upgrade a CML 2.2.x installation to CML 2.3. You must migrate to a new CML 2.3 installation. If you have an existing CML 2.2.x installation, the migration tool will help you migrate your data from your existing CML installation to the new CML 2.3 installation. If your existing CML installation is CML 2.0.x or 2.1.x, we recommend first following the CML 2.2.x documentation to perform an in-place upgrade to CML 2.2.3 before migrating to a new CML 2.3 installation.
Note: If you choose to migrate an existing, licensed CML 2.2.x installation to a new installation of CML 2.3, you must deregister the old CML instance before decommissioning it. See Licensing - Deregistration for instructions on deregistering a CML instance.
- Deregister your existing CML 2.2.x instance.
- Install CML 2.3 (see below).
- Since you are migrating from an existing CML 2.2.x installation, mounting the refplat ISO before running the CML 2.3 installation is optional.
- If the reference platform ISO is mounted before starting the CML 2.3 installation, the set-up script will copy reference platform images from a mounted refplat ISO to the local disk.
- You can skip the copying of reference platform images during installation because the migration tool will copy all of the reference platform images from your CML 2.2.x installation to the new CML 2.3 installation.
- Only mount the new refplat ISO if you want to add the new versions of the reference platforms in addition to the existing versions from your CML 2.2.x installation.
- Follow the instructions in the migration tool's README.md to migrate data from your existing CML 2.2 installation to your new CML 2.3 installation.
- License your new CML 2.3 installation.
The migration tool can be found on your CML 2.3 installation at /usr/local/bin/virl2-migrate-data.sh. You can also download the latest version of the migration tool from https://github.com/CiscoDevNet/cml-community/tree/master/scripts/migration-tool.
The migration tool also supports offline migration. If you want to install CML 2.3 on the same server that's currently running CML 2.2.x, you can use the Offline Version Migration instructions to back up the data to a .tar
file. That way, you can transfer the .tar
file to a separate datastore temporarily while you reinstall with CML 2.3. As always, make sure that you deregister your old CML 2.2.x instance before decommissioning it and starting the CML 2.3 installation.
Installing CML 2.3
The CML Installation Guide provides detailed steps for installing your new CML 2.3.x instance.
- In CML 2.3.x, all of the reference platform VM images must be copied to the local disk of the CML instance. Be sure to provision enough disk space for your new CML instance to accommodate the reference platforms.
- Downloading the refplat ISO file is optional if you are migrating from an existing CML 2.2.x instance. The migration tool will migrate all of your existing image definitions and VM images from your old CML 2.2.x instance.
- Be sure to check the System Requirements for updated system requirements.
- Important: After the reference platforms are copied to disk during the Initial Set-Up, wait a couple of minutes for the CML services to finish restarting before clicking OK to continue. If you click OK too soon, the initial set-up can fail. (See the Known Issues below.)
Summary of Changes
This section details new and changed features in the new release. Bug fixes are listed in the subsequent section. You can see a brief demo of some of the new features in the CML 2.3 Feature Walkthrough video:
UI Changes
The CML 2.3 UI introduces new features for the lab Workbench.
Feature | Description |
---|---|
Link Bundling | When there are multiple links between two nodes, the Workbench can now show them either as a single bundled link or as individual links directly in the canvas. |
Hiding Links | You can set a node to hide its connected links in the Workbench. This feature is especially useful for hiding links that were created for out-of-band management access or external connectivity to the node. |
Node state badges | There is an additional badge state for when a node is in the queue, waiting to be started. |
The behavior of some Workbench actions have changed compared to the CML 2.2 UI. The styling of some elements has also been updated:
- Add a link is now a single click on the link button in the node action ring vs a click-and-drag in all previous versions.
- Link label styles have changed
- Badge styles have been changed
- Node positions are no longer snapped to a grid.
Canvas Actions in CML 2.3
CML 2.3 introduces new canvas actions and changes the behavior of some of the existing canvas actions in the Workbench. This table summarizes the CML 2.3 changes compared to previous versions of CML.
Action | <=2.2 | 2.3 |
---|---|---|
Hover link | Link color changes from grey to Cisco blue | Link size increases |
Left click link | Select Link + color changes to Cisco blue | Select link + color changes to Cisco blue |
Double-click link | None | For a bundled link, hide the bundle and show individual links. For a single link that is bundleable, show bundle and hide individual links |
Double-click node | None | Toggle hiding of links for node |
Left click node | Select node and connected links | Select node and connected links |
Left click empty area on canvas | Unselect all elements | Unselect all elements |
Double-click empty area on canvas | Canvas zooms by one level | None |
Hover node | Display radial menu + change node outline color from grey to Cisco blue | Display radial menu |
Left click on menu item in node hover menu | Take action and keep menu displayed | Take action and hide menu |
Mouse wheel scroll | Zoom in/out | Zoom in/out but more aggressively |
Toolbar Changes
This table summarizes changes to the Workbench toolbar in CML 2.3. In addition to the toolbar button changes, the toolbar is now accessible via screen readers.
Toolbar Buttons | <=2.2 | 2.3 |
---|---|---|
Undo | Undo last lab action | N/A - button removed from toolbar |
Screenshot | N/A - button added in 2.3 | Takes screenshot of canvas with current color scheme |
Center | Center canvas with long animation | center canvas without animation |
Fit | N/A - button added in 2.3 | Fit to canvas with short animation |
Zoom In | Single click zoom only | Single click + click and hold for continuous zoom |
Zoom Out | Single click zoom only | Single click + click and hold for continuous zoom |
Full Screen | Make canvas full screen | N/A - button removed from toolbar. |
Toggle Link Labels | Show/hide link labels | Show/hide link labels |
Change link style | Toggle link styles between straight and bezier | N/A - button removed from toolbar |
Bundle links | N/A - button added in 2.3 | Toggle bundling parallel links between nodes |
Single click node | select node and connected links | select node and connected links |
LDAP Authentication
With version 2.3, a new user authentication method has been added: LDAP. Prior to version 2.3, all authentication and authorization requests to CML were checked against the local user database.
A CML admin user is created during installation. For the personal edition of the product, this is typically the one and only user account needed.
For multi-user CML-Enterprise or CML-Education instances, a CML administrator can change CML's authentication method to LDAP. You can integrate CML with your existing directory services, such as Microsoft Active Directory, as long as it provides LDAP access. With the LDAP authentication method, CML authenticates logins and authorizes users via the configured LDAP server, and the CML administrator does not create local accounts on the CML instance.
See Configuring LDAP Authentication for instructions on configuring CML to use LDAP for authentication.
API Changes
CML 2.3 introduces several changes to the REST-based web service APIs. For details about a specific API, log into the CML UI and click Tools > API Documentation to view the CML API documentation page.
APIs Modified in CML 2.3
In previous versions of CML, the JSON format for many of the request bodies or responses had no defined format. In CML 2.3, the API specification now provides a JSON schema for the request and response bodies, if any. The API specification also specifies any constraints on the path parameters for the API endpoints. While defining these schemas, we normalized the object structures and made the ID handling more consistent. Therefore, the behavior of some of the existing APIs will be a little different in CML 2.3.
Lab IDs in CML 2.3 have been converted from short, unique identifiers to full, 128-bit UUIDs. The UUIDs are represented in the APIs by 36-characters strings in five hyphen-separated groups of hexadecimal digits. Note that this means that the IDs of any existing labs will be different after migration to CML 2.3.
There are some important changes in the behavior of the APIs related to users and groups. We updated these APIs as part of the changes for LDAP support, adding properties and making them more consistent. In addition to the user and group API endpoints that were added or removed in CML 2.3 (see below), the user_id
parameter in the user and group APIs is now the user's UUID, not the username
. The group_id
in these APIs is now the group's UUID and not the group's name
. This change affects both the path parameter in the API endpoints and references to users and groups in the objects returned by the APIs.
The members
property of a group now lists the UUIDs of the users in the group, not their usernames. Similarly, the groups
property on user objects is a list of group UUIDs to which the user belongs, not a list of group names. There is a new labs
property in the user objects returned by the APIs. The labs
property list the IDs of the labs owned (created) by the user. There is also a new groups
property on the lab objects returned by the GET
/labs/{lab_id} API. Similar to the groups
property in the user object, this property is a list of the UUIDs of the groups to which the lab belongs.
The DELETE
/groups/{group_id} and DELETE
/users/{user_id} endpoints return an HTTP/204 (No Content) on success instead of an HTTP/200. In general, when a CML API endpoint completes successfully but returns no response object, it uses the return code HTTP/204 instead of HTTP/200 to indicate success.
The POST
and PATCH
API endpoints for /groups/{group_id} and /users/{user_id} return the group or user object that was created or updated. In CML 2.2, these API endpoints returned the object's ID or just the string "OK"
on success.
Another significant change in the CML 2.3 APIs involves the structure of the node definition object. The JSON object returned by the GET
/node_definitions and GET
/node_definitions/{def_id} is different in CML 2.3 compared to previous releases. The JSON object for a node definition used to nest most of the details under the data
property, but the extra layer of nesting was inconsistent with the YAML version of the node definition. The JSON object for a node definition in CML 2.3 no longer has the top-level data
property, and the properties that it used to contain are now at the top-level of the node definition. This change makes the JSON representation of a node definition in CML 2.3 equivalent to the YAML representation. If you use the JSON response from GET
/node_definitions or GET
/node_definitions/{def_id} APIs, you will need to adjust your code to handle the flattened data structure.
New APIs in CML 2.3
This table summarizes the API endpoints that are new in CML 2.3.
API Endpoint | Description |
---|---|
GET /users/{username}/id |
Get user unique identifier. |
GET /groups/{group_name}/id |
Get group unique identifier. |
GET /system/auth/config |
Retrieve the current system authentication configuration |
PUT /system/auth/config |
Set the system authentication configuration |
POST /system/auth/test |
Test provided system authentication with a username/password |
GET /image_definition_schema |
Returns the JSON schema that defines the image definition objects. |
POST /users |
Creates user |
PATCH /users/{user_id} |
Updates user |
PATCH /groups/{group_id} |
Updates a group. |
POST /licensing/reservation/discard |
Discard a reservation authorization code for an already cancelled reservation request. |
PATCH /labs/{lab_id} |
Update the specified lab, if it exists. |
GET /labs/{lab_id}/links/{link_id}/capture/status |
Returns the status of the packet capture running on the specified link. |
APIs Removed in CML 2.3
This table summarizes the API endpoints that have been removed and are no longer available in CML 2.3.
API Endpoint | Description |
---|---|
GET /user/roles |
Get roles for this user |
PUT /users/{user_id}/change_password |
User password change. |
PUT /users/{user_id}/roles |
Update the user roles. |
GET /wait_for_lld_connected |
This web service blocks and does not return until the LLD connects to the controller. |
POST /feedback |
Sends feedback to the product team. |
GET /lab_schema |
Returns the JSON schema that defines the lab. |
POST /users/{user_id} |
Creates user |
PUT /groups/{group_id} |
Updates a group. |
PUT /clear_backup_and_shutdown |
Stop the services and discards all backups of the controller state. |
APIs Deprecated in CML 2.3
The following APIs are now deprecated and will be removed in a future release:
Deprecated API Endpoint | Use this API Endpoint Instead |
---|---|
DELETE /licensing/reservation/discard |
POST /licensing/reservation/discard |
PUT /labs/{lab_id}/description |
PATCH /labs/{lab_id} |
PUT /labs/{lab_id}/title |
PATCH /labs/{lab_id} |
PUT /labs/{lab_id}/notes |
PATCH /labs/{lab_id} |
PUT /labs/{lab_id}/nodes/{node_id}/image_definition |
PATCH /labs/{lab_id}/nodes/{node_id} |
GET /labs/{lab_id}/nodes/{node_id}/config |
GET /labs/{lab_id}/nodes/{node_id} |
PUT /labs/{lab_id}/nodes/{node_id}/config |
PATCH /labs/{lab_id}/nodes/{node_id} |
Reference Platforms
The new refplat ISO file available with the CML 2.3 release includes a CAT 8000V image. To reduce the size of the refplat ISO file, only a single NX-OS 9000 VM image is included. The refplat ISO file also no longer includes the node definitions or VM images for NX-OS or IOS XRv. Both of those reference platforms are EOL and are no longer being updated. If you would like to continue using these images, the NX-OS and IOS XRv node and image definitions from CML 2.2.x still work in CML 2.3.x. If you used the migration tool to migrate your data from CML 2.2.x, then you will still have these images for use in your new CML instance. If you did not migrate data from an earlier CML release and do not have these VM images, you can get them from the CML 2.2.x refplat ISO file and add them to your new CML instance.
CML 2.3 includes the vmxnet3
NIC driver. With this driver, CML now supports subinterfaces on CSR 1000v nodes. If you migrated your data from CML 2.2.x, where CSR 1000v used the VirtIO
NIC driver, CML 2.3 will automatically update the node definition to the vmxnet3
driver. Subinterfaces are also supported by the new CAT 8000V image.
The documentation on VM Images for CML Labs has been expanded to include additional information about each of the VM images on the refplat ISO file. We have also added sample labs with working configurations for each of the bundled Cisco reference platforms. Look for the sample labs with Feature Tests in the lab name in the Tools > Sample Labs page in the CML UI. These are the same labs that Cisco uses when testing the bundled Cisco reference platforms with a new CML release.
Other Changes
CML 2.3 is based on Ubuntu Linux 20.04LTS. From CML 2.0.0 to CML 2.2.3, the CML release was based on CentOS 8.x. CentOS 8 is now EOL and no longer releases security fixes. Moving CML to Ubuntu provides a stable and secure platform for the current and future CML releases.
In CML 2.3, the CML services can load the state of running VMs when the services start. Therefore, CML no longer stops any nodes or labs when you restart the CML services (without rebooting the CML server). When the CML services restart, they will scan the system to determine the state of all nodes and labs.
This table provides some details on other individual changes in CML 2.3.
Summary | Details |
---|---|
Change lab element IDs to UUID | Internal lab elements (i.e., labs, nodes, interfaces, and links) are represented by UUIDs instead of IDs. CML 2.2.x .yaml files can still be imported to CML 2.3. |
Expose all image definition parameters via the UI | New values are exposed in the Image Definition page of the UI: schema_version, disk_image_2 and disk_image_3 |
Add Hiding Links feature to UI | The Hiding Links feature allows a user to hide links for a given node:
|
Cytoscape dark mode | Adjusted node colors for dark mode |
Cytoscape toolbar |
|
New sample labs | New "Feature Test" sample labs with working node configurations are available in Tools > Sample Labs for each of the bundled Cisco reference platforms. |
Notification when a node is launched without mounting the refplat ISO | We raise a human-friendly message when starting a node if a refplat ISO is not attached. |
Update validation for image/node definitions | Set limits for various node and image definition properties. |
Licensing validation | The Licensing API call for setting an explicit reservation count for individual features had an incorrect input format definition. In addition to its previous mapping of feature id to count , it now allows the use of a more standard array of objects with id and count properties. In the Licensing API call for retrieving a return code for a cancelled reservation authorization code now allows the use of the more standard POST method instead of DELETE (with the same input body). Clients automating Licensing API are encouraged to make use of these new style forms. |
Enhancement to logout and clear all sessions | New button in the Settings page of the user menu to clear all login sessions for the current user. |
Lab Console Improvements |
|
Unified UI for the Settings page | Redesigned Settings page to look like the System Administration page. Updated form validations for password change (disallowing the use of the same password) |
No error message was shown in the UI when attempting to create or update a node definition without providing all mandatory fields. | Longer UI forms, such as the node definition form, can display a global error message near the Update or Save button. When you save the form with missing values for any mandatory fields, the form updates to show the error message and a button that will scroll the form to the first invalid field. |
Minor UI improvements for Workbench page |
|
Show loading animation as the UI loads | Prefetch UI resources and show a loading progress bar at the top on initial page load. |
Migrate the /settings page to subroutes like /sys_admin |
|
Disable interface simulation tab for unmanaged switch | Unmanaged Switch and External Connector have disabled interface Simulate tab |
Truncate lab name and node label |
|
Add a confirmation popup when deleting node/image definitions | Added confirmation dialog for deletion of an image/node definition and uploaded image |
Remove .ng lab support in UI | Removed import lab support for .ng lab files which used an older, pre-FCS format. |
Remove .ng lab support | Removed support for json based (.ng) topologies. Converted all sample labs to yaml format. |
Terminal server displays element labels rather than IDs. | The console server used to use the lab and node IDs, but the change to UUIDs in CML 2.3 made that approach unreadable. The terminal server now uses the lab and node labels. |
Make API schema for GET users match the data | API schema says List[userobject]. Fixed and now it returns correct result. Previously it returned Dict. |
Add support for IPV6 address and port override when generating pyats testbed | The GET /api/v0/labs/{lab_id}/pyats_testbed API accepts a hostname query parameter for overriding the default host or IP of the terminal server. This parameter can now be used to specify IPv6 addresses and/or an alternate port. |
User group openapi input validation | user and group API endpoints now use robust type, range (for integers) and length (for strings) validation - users can learn more about particular values in swagger POST /users and POST /groups |
Validation of node and image definitions | node and image definitions API endpoints now use robust type, range (for integers) and length (for strings) validation - users can learn more about particular values in swagger POST /image_definitions and POST /node_definitions |
Add descriptions to node and image definition schemas | API documentation updated |
Disable group API calls in UI | Fixed "unable to get groups" error in lab list view for unlicensed/personal instances. Disabled User Administration for unlicensed/personal instances. |
Forbid enterprise calls in controller | Forbid various user, group and lab APIs on unlicensed products and products that use a personal license. |
Remove SLR licenses once they expire | For CML-Enterprise customers who use Specific License Reservation (SLR), the validity of the licenses is properly enforced. A node expansion license entitlement outside its validity period does not count towards the limit, and a valid base license is required to allow node launches in the first place. If you use SLR licensing, you are encouraged to review the official status of your license reservations in the Cisco Software Central Smart Licensing portal. |
Remove node definition properties from the JSON blob | Store only configurable properties in the json blob. All properties that can be inherited from node and image definitions are no longer stored in the json blob. |
Make the logs for the virl2-* services persistent | Logs for virl2 services are persistent. |
Document limitation of NXOSv 9000 cannot send Jumbo Frames | Jumbo frames work by default in the NX-OS 9000 nodes in CML, but configuring mtu on an interface is not supported by NX-OS 9000 until version 9.3(3). To use jumbo frames, change your lab to use a NX-OS 9000 VM image version of 9.3(3) or higher. |
Update virl2_client for the node_definition API changes in CML 2.3 | Because of the change in the GET /api/v0/node_definitions response structure, the structure of the node definitions in the list returned by client.definitions.node_definitions() is different in the new version of the virl2_client. For example, if you had this code in your old code label = client.definitions.node_definitions()[0]["data"]["ui"]["label"] , you would change it to remove the "data" layer: label = client.definitions.node_definitions()[0]["ui"]["label"] . |
Generate a lab title on .virl import when no title is provided | If no title is provided when importing a VIRL 1.x topology, an empty title will be generated for it. |
Include the migration tool in CML 2.3. | The migration script can be found at /usr/local/bin/virl2-migrate-data.sh. We recommend checking the cml-community repository for updates before using the bundled version of the migration script. |
Bug Fixes
Bug Description | Resolution |
---|---|
CML does not support '-' for DNS search domains. | Fixed in CML 2.3.1. |
Subinterfaces were not working in CSR1000v | Node definition network interface drivers add vmxnet3 as an option. The node definitions for CSR1000v and CAT8000v have been switched to use vmxnet3 network interface drivers. This enables vlan subinterfaces to work on these nodes. |
CML reported Transport not registered message for the Registration Status on the Tools > Licensing page. |
Only applies to CML deployments that use OnPrem SSM licensing. Fixed a bug with CML's that caused failures with registration and authorization renewals when using OnPrem SSM. This bug could eventually cause issues with manual removal of the CML instance from the OnPrem SSM. |
Core driver fails to start if an invalid/corrupted image definition is present | virl2-core-driver service will not fail anymore if an invalid image definition is present. |
Request timeout in login/auth page is too short | Increase the time limit from 5s to 15s when logging in and fixed error message during timeout on login page |
Configuration extraction failed / timed out for IOS XRv 9000 nodes whose node label and hostname did not exactly match. | This known issue was caused by a bug in pyATS. It is resolved in CML 2.3.0. |
Configuration extraction did not properly dispose of file handles, which could eventually cause the controller service to crash. | Fixed in CML 2.3.0. |
Vulnerability: X-frame option missing in Cockpit:9090 | X-frame header was added in CML 2.2.3 (CentOS 8.4) and the vulnerability is a false positive. The vulnerability is not present in CML 2.3.0. |
Ubuntu images uses a different interface name from what is defined in the node definition. | A new Ubuntu cloud image is included in the new refplat ISO with predictable interface naming that matches what is shown in the UI |
No warning was shown when CML could not start a queued node because of insufficient resources. | If CML cannot start a node because there are insufficient resources (vCPU, memory, or disk space) available, the node remains in the QUEUED state, and CML adds a warning notification to the Notifications in the Workbench every minute. If the CML server has insufficient resources to ever start the node (i.e., the node requires more hardware resources than the total hardware capacity of the CML server), the node can never be started, and CML shows an error notification in the Workbench. |
When the lower pane of the Workbench is showing the Nodes pane, clicking the console button in a node's radial action menu does not switch to the Console tab. | Fixed in CML 2.3.0. The console button in the node's radial action menu switches the lower pane to the Console tab for the node even when the Nodes pane is open. |
Missing error message on the login page after a failed login. | Fixed in CML 2.3.0. This bug only occurred when you closed the error message after a failed login. Subsequent logins failed to show the error message. |
Confusing notifications when a node fails to wipe | Failed node wipe will show more detailed error message (including the name of the node) |
Confusing notifications when a lab fails to wipe | Failed lab wipe will show more detailed error message (including the name of the lab) |
Lab's name allows empty string | PATCH /labs/{lab_id}/title and PATCH /labs/{lab_id}/ do not allow null/empty string for lab title Added a validation rule in the UI for lab title fields to prohibit empty string |
Lab notes editor doesn't resize on init | Fixed wrongly calculated height of the lab notes editor on initialization |
Image definition delete confirmation message contains null |
Fixed missing ID in the confirmation message to delete image definition |
User allowed to set empty node name | PATCH /labs/{lab_id}/nodes/{node_id} doesn't allow null/empty string for node label Added a validation rule in the UI for node label field to prohibit empty string |
CommitFields don't disable the Apply button with an invalid value | Commit fields will now properly disable the commit action when the field has an invalid value. |
Missing validation messages for node labels in the Node Info pane | When an invalid Node Name is entered in the Workbench's Node Info pane, the field shows the validation error message. |
After deleting a node definition, the UI returned to the Node Definitions page, but the deleted node definition was still shown in the list. | Fixed in CML 2.3.0. |
After failing to import an invalid node definition, importing a new one appears to do nothing. | Fixed in CML 2.3.0. When you try to import an invalid node definition file, the UI will clear the file selection field. |
Import lab hangs on when trying to upload a file with an invalid extension | Fixed hanging lab import with unsupported file extension (only .yaml, .virl and .ng are supported) |
Automatic Image Definitions has an empty info pop up in the Workbench. | Image Definition tooltip is disabled for the "Automatic" option in the Workbench |
Info button in the Image Definition group on the Simulate pane of a node shows nothing until node is wiped. | Fixed a bug when some tooltips (by disabled fields) were not showing up on mouseover. |
Image Definition group is not bound to the Simulate menu of the node and moves. | The options (if open) for the selection type field will move along with the other elements as the user is scrolling in the workbench footer. |
Info icon is missing in Node Definitions and icon with invalid text is present in Image Definitions | Fixed tooltip for the "i" icon on the image definition page. |
Wipe button in Dashboard is not working correctly | Disabled wipe button in Dashboard Tile View and Workbench for DEFINED_ON_CORE and STARTED lab states |
Only wipe nodes if they are stopped | Running nodes and labs can no longer be wiped. If an API request is made to wipe a running node or lab, it returns HTTP 400 / Bad Request . The same error is returned when an API request is made to wipe a node or lab that was already wiped. |
Missing error messages for User Administration | Added validation for checking if the new password is not the same as the current one in the User Administration page. |
The UI shows an empty error message when trying to login with the wrong password | Fixed empty error message at login with wrong credentials |
Deregistering when already deregistered results in weird licensing state | Deregistration is not possible when already deregistered anymore. |
Non-existent groups should redirect user to /notfound |
The administrator will be redirected to /notfound when he tries to visit a non-existent group in System Administration. |
When you deleted a node from the Nodes pane in the Dashboard, the node was removed from the list, but the now-deleted node remained selected. Subsequent actions in the Nodes pane will generated errors because they failed for the deleted node. | This bug is fixed in CML 2.3.0. When nodes are deleted in the lab's Nodes pane, the nodes are also removed from the list of selected nodes in the UI. |
When you deleted a lab in the Dashboard, the lab was removed from the list, but the now-deleted lab remained selected. Subsequent actions generated error notifications because they failed for the deleted lab. | This bug only affected the Dashboard when Show List was enabled. The bug is fixed in CML 2.3.0. When a lab is deleted, the lab is also removed from the list of selected labs in the Dashboard. |
If you are logged in as a non-admin user, deleted labs are not removed from the Dashboard until you refresh the page. | Fixed in CML 2.3.0. |
Wrong exception type in virl2-client | Raise a correct exception when None is provided as the URL in client library initialization. |
Licensing module crashed with SLR licensing. | Fixed a bug that caused the licensing module to crash if you entered an invalid or expired authorization code. |
HTTP urls were not allowed in licensing. | Permit URLs in the Transport Settings on the Tools > Licensing page to use HTTP protocol. |
Known Issues and Caveats for CML 2.3
Bug Description | Notes |
---|---|
After upgrading CML to 2.3.x, the Breakout Tool shows API error and Network error and does not show any labs. |
The older versions of the Breakout Tool are incompatible with CML 2.3.x. To use the Breakout Tool with CML 2.3.x, replace your local version of the breakout binary or EXE file with the version that's bundled with CML 2.3.1. With your CML 2.3.1 instance, follow the instructions on Installing the Breakout Tool. |
When using link conditioning and VLANs, packet forwarding appears to break. | When using link conditioning on a link, packets that cross the link lose their dot1q tags. The missing tags cause problems with switches that expect tagged packets, and packets may be dropped. Workaround: Use a WAN Emulator node instead of the Link Condition properties on a link's Simulate pane. For example, if a conditioned link connects SW-1 and SW-2, remove the link, add a WAN emulator node, and then connect both SW-1 and SW-2 to the wan-em node. Set the link conditioning properties in the wan-em node's Edit Config pane or configure the running wan-em node via its console. |
Port-channels do not come up in IOSvL2 nodes. | A bug in CML 2.3.x changed the way that CML generates mac addresses for interfaces in running labs. This change breaks some features in certain reference platforms, such as port-channels and SVIs on IOSvL2. The bug will be fixed in a future CML release. Workaround: For IOSvL2 port-channels, use channel mode on . For IOSvL2 SVIs, manually configure unique mac-addresses for each SVI. |
Link hiding feature does not work in CML 2.3.1. | Workaround: There is no workaround at this time. If you need the link hiding feature, continue to use CML 2.3.0 and do not upgrade to CML 2.3.1. |
VMware ESXi 6.5 will not import the OVA. | Fixed in CML 2.3.1. The CML 2.3.0 OVA was built with the incorrect VMware HW version. The CML 2.3.1 OVA is compatible with VMware ESXi 6.5+. |
virl2-controller crashes. The virl2-controller service logs indicate the error free(): corrupted unsorted chunks followed by Main process exited, code=dumped, status=6/ABRT . |
Fixed in CML 2.3.1. |
Even if you were able to run network simulations with CML 2.2.x on VMWare Fusion and Mac OS X Big Sur or above, the same labs no longer start after migrating to CML 2.3.0. | Workaround: There is no workaround at this time. Do not upgrade to CML 2.3.0. |
The CML 2.3 VM consumes all of the allocated memory even with no labs running. When you start the CML 2.3 VM in VMware, it will quickly allocate all of the memory assigned to the VM. For example, if you configured the VM to use 16 GB of memory, the VM process will show all 16 GB of memory in use shortly after it starts. If you check the guest OS itself (in the CML UI's status bar or by running Note: this problem affects all VM deployments of the CML 2.3.0 OVA. Earlier versions of the release notes incorrectly stated that the problem did not impact CML 2.3.0 deployments on VMware ESXi. |
Upgrade to CML 2.3.1. This problem only affects new deployments that used the CML 2.3.0 OVA. If you installed CML with the 2.3.1 OVA, your CML VM should not be affected. If you upgraded to CML 2.3.1 from CML 2.3.0, be sure to power down the CML VM from VMware after the upgrade. |
Your web browser shows an error that indicates an expired certificate when you try to use the CML UI or the Cockpit admin portal for your CML server even if you have previously added a local exception for the certificate. | The default SSL / TLS certificate included with CML 2.3.0 expires in June 2022. After that date, web browsers will report a warning or error because of the expired certificate. Upgrade to CML 2.3.1, which includes an updated certificate that is valid until 2024. Workaround: Install your own TLS / SSL certificate that's signed by a trusted Certificate Authority (CA). See the CML FAQ for instructions on installing your own certificate. |
CML can corrupt labs if virl2-lowlevel-driver is not available. The CML logs report a heartbeat exception because virl2-lowlevel-driver is unavailable. |
Fixed in CML 2.3.1. |
Installation fails if you click OK to continue installation after the reference platforms are copied but before the services finish restarting. | Workaround 1: After the reference platforms are copied, wait a couple of minutes before clicking the OK button to continue with the initial set-up. Workaround 2: If the installation failed because you pressed OK too quickly, reboot the CML instance and reconfigure the system again. All refplats are already copied, so the initial set-up screen won't restart the services this time. Note that the previously-configured admin user will already exist, so you will need to create a new admin account with a different username to complete the initial set-up. |
The uuidd.service is shown as failed in the services list in Cockpit. |
Upgrade to CML 2.3.1, which fixes this bug. Workaround1: Ignore this service failure. The uuidd service is not required for running CML. Workaround2: Edit the file /usr/lib/systemd/system/uuidd.service and set PrivateUsers=no . Then restart the uuidd service. |
The Devices panel in the Cockpit > Storage page is empty. It does not show the LVM volume group as described in the CML documentation. | Upgrade to CML 2.3.1, which fixes this bug. Workaround:
|
Bare metal installation fails | Workaround: you must manually clear the server's disk before starting the installation. |
Bare metal installation always installs to /dev/sda. | There is no workaround at this time. |
Deleting a CML VM without first deregistering leaves the license marked as in use. New instances indicate that the license authorization status is out of compliance. | If you have deleted or otherwise destroyed a CML instance without deregistering it, then the Cisco Smart Licensing servers will still think that instance’s licenses are in use. Workaround 1: Before you destroy a CML instance, follow the Licensing - Deregistration instructions to deregister the instance. Properly deregistering an instance before destroying it will completely avoid this problem. Workaround 2: If you have already destroyed the instance, see the instructions for freeing the license in the CML FAQ's Licensing topic. |
Bug in OS X Big Sur. After upgrading to OS X Big Sur and VMware Fusion 12.x, simulations no longer start. | Big Sur introduced changes to the way OS X handles virtualization. While VMware Fusion 12 works with the new changes in OS X Big Sur, there is a bug that impacts some Apple systems. On those systems, after you upgrade the host OS to OS X Big Sur, labs will not start, and nodes just stay in QUEUED state. While not all systems exhibit this problem, we currently recommend that all CML customers remain with OS X Catalina until Apple and VMware fix this bug. Workaround 1: Do not upgrade to OS X Big Sur. Continue running the CML VM on OS X Catalina and Fusion version 10+. Note that CML appears to work correctly with Fusion 12 as long as the host OS is OS X Catalina. Workaround 2: If you already upgraded to Big Sur, and you experience the specified symptoms, then you may need to roll back your entire system to the last backup before you upgraded to Big Sur. |
When the refplat ISO is not mounted until after the initial set-up steps have started, CML will not load the node and image definitions after you complete the initial set-up. | Workaround: Log into Cockpit, and on the CML2 page, click the Restart Controller Services group to expand it, and click the Restart Services button. CML will load the node and image definitions when the services restart. |
When the incorrect product license is selected, the SLR (Specific Licensing Reservation) still can be completed. It eventually raises an error, but the SLR reservation will be complete with no entitlements. | Workaround: If you accidentally select the wrong product license, then you will need to release the SLR reservation and then correct the product license before trying to complete the SLR licensing again. |
If the same reservation authorization code is used twice, the CML Smart agent service crashes. | Workaround: Do not use the same reservation authorization code more than once. |
If you have read-only access to a shared lab, some read/write actions are still shown in the UI. Attempting to change any properties or to invoke these actions raises error notifications in the UI. | Workaround 1: Do not attempt to invoke the read/write actions when working with a lab for which you only have read-only permissions. Workaround 2: Ask the lab owner to share the lab with read/write permissions so that you can make those changes to the lab. |
If you have read-only access to a shared lab, you will still not be able to see the current link conditioning properties in the link's Simulate pane. | Workaround: ask the lab owner to share the lab with read/write permissions if you need to access the link conditioning properties. |
The algorithm for computing the available memory does not take into account maximal possible memory needed by all running nodes. It only checks the currently allocated memory. CML will permit you to start nodes that fit within the current available memory of the CML server. Later, if the memory footprint of the already-running nodes grows and exceeds the available memory on the CML server, those nodes will crash. | Workaround 1: Launch fewer nodes to stay within your CML server's resources. Workaround 2: Add additional memory to the CML server to permit launching additional nodes. Of course, this workaround just moves the point where the CML server will eventually hit its limit, so you must remain aware of the your CML server's resource limits or ensure that the CML server has sufficient memory to accommodate all labs that will be running on your CML server simultaneously. |
On a CML server or VM with multiple NICs, the CML services are active and running and do not report an error, but the CML UI and APIs are not reachable. The CML services are listening on the wrong interface because, when the CML instance has multiple NICs, CML chooses the first alphabetically-ordered interface and configures it as the CML server's management interface. That may not be the correct interface. For example, an interface name like enp10s0 would be selected before enp9s0 . |
Workaround 1: Change the network connections to the CML server so that the interface that CML selects is the one that you'd like it to use for its management interface. Workaround 2: After the CML server boots up, configure the network interface manually in the Cockpit Terminal: nmcli conThat command shows which interface bridge0-p1 is actually connected to. In this example, let's say that bridge0-p1 connected to ens10p0 , and we want it to be connected to ens9p0 . We need to remove the bridge0-p1 connection and then recreate it with a connection to the right interface. Adjust the following commands based on your interface IDs. check con del bridge0-p1In the end, you should see only 3 active connections (green with a device set). The first two are bridge0 and virbr0, and we do not touch these; the third one is bridge0-p1 which should now have the correct interface listed as the device. |
When starting a CSR 1000v node for the first time or after a Wipe operation, the interfaces are all in shutdown state. |
Workaround 1: Add no shut to each interface config in each CSR1000v node's Edit Config field in the UI. Workaround 2: Use the EEM applet on the CSR 1000v page. |
New nodes cannot be started when CPU usage is over 95%, when RAM is over 80%, or when disk usage is over 95%. | Workaround: Allocate additional resources to the CML server. Workaround 2: If you're hitting the disk usage limit, check the Remove Orphaned VMs function in Cockpit. Orphaned VMs can consume a significant amount of disk space even though CML is not longer using or managing those VMs. |
VM Images uploaded via the API can only be used in an image definition if they are transmitted via a multipart/form-data request. The problem is that the SELinux context is not set correctly. |
Workaround 1: Use the Python Client Library, which handles image uploads correctly. Workaround 2: Create a multipart/form-data request when calling this API. |
The journalctl output shows errors like this: cockpit-tls gnutls_handshake failed: A TLS fatal alert has been received |
This log message only occurs with Chrome browsers and has no functional impact on the use of Cockpit. No workaround needed. |