- Overview
- Product Documentation
- CML 2.6 Release Notes
- CML 2.6 Installation Guide
- CML 2.6 User Guide
- CML 2.6 Admin Guide
- CML Administrator's Guide
- Cisco Modeling Labs System Overview
- System Defaults
- Creating a New Node Definition
- Custom VM Images
- Clustering
- CML Admin Tools
- System Settings
- Networking
- Resources
CML 2.6 Release Notes
Cisco Modeling Labs (CML) is a network simulation platform. CML 2.6 is the latest feature release of CML and introduces a redesigned Workbench UI, a new maintenance mode for use on multi-user CML instances, and support for deployment to AWS. This release also includes further improvements for accessibility, external connectivity, cluster deployments, link conditioning, node and image definitions.
New and Changed Information
This table shows any published changes to this CML 2.6 release notes document.
Date | Changes |
---|---|
July 20, 2023 | Initial version for the CML 2.6.0 release. |
August 9, 2023 | Updating Known Issues. Updating advice about upgrading to CML 2.6. Adding screenshots to the Summary of Changes and a video overview of CML 2.6 features. |
October 6, 2023 | Updating Bug Fixes and Known Issues for the 2.6.1 maintenance release. |
October 20, 2023 | Updating Known Issues for the 2.6.1 maintenance release. |
Installing or Migrating to CML 2.6
Upgrading to CML 2.6
There are a few changes to the upgrade process for CML 2.6.0. Please read this section carefully before starting an upgrade to CML 2.6.0.
- We recommend reviewing the Known Issues and Caveats for CML 2.6 before upgrading to a new CML release. The latest release may include bugs that impact specific features or CML installation types. Depending on how you deploy and use CML, you may want to skip a release and wait until a specific bug is fixed in a future maintenance release.
- If your CML instance is already running CML 2.3.0 or later, you can upgrade it to CML 2.6.0.
- In most cases, you must first log in to Cockpit and install the latest software updates before upgrading to CML 2.6.0. Otherwise, the upgrade will fail due to unmet package dependencies because of a known problem in the CML 2.6.0 upgrade .pkg.zip file. Even if you are running an offline or air-gapped CML instance, you still need to install the latest software updates. If you cannot install the latest software updates before upgrading to CML 2.6.0, contact Support.
- After the upgrade completes, you must disable maintenance mode and change the state of the compute (on a cluster, at least one compute member's state) back to READY before you can start CML labs again. This post-upgrade step is new in CML 2.6.0 and is part of the maintenance mode feature, described in the Summary of Changes below.
- See the In-Place Upgrade page for detailed upgrade instructions.
In-place upgrades from CML version 2.2.3 or earlier are not supported. See migration instructions below.
Upgrading a CML Cluster to CML 2.6
If you have deployed a CML cluster, you may upgrade it to CML 2.6.0.
When upgrading from CML 2.4.0, which introduced cluster support, you must manually run the CML upgrade first on the controller and then separately on each compute member.
If you are upgrading from CML 2.5.0+, you only need to run the CML upgrade on the controller. The CML controller's upgrade automatically initiates a CML upgrade on the cluster's compute members.
Migrating from an Existing CML 2.2 Installation
It is not possible to upgrade a CML 2.2.x installation to CML 2.6. You must migrate to a new CML 2.6 installation. If you have an existing CML 2.2.x installation, the migration tool will help you migrate your data from your existing CML installation to the new CML 2.6 installation. If your existing CML installation is CML 2.0.x or 2.1.x, we recommend first following the CML 2.2.x documentation to perform an in-place upgrade to CML 2.2.3 before migrating to a new CML 2.6 installation. The following video demonstrates the migration process from CML 2.2 to CML 2.3, and you can use the same steps to migrate from CML 2.2.3 directly to CML 2.6.
Note: If you choose to migrate an existing, licensed CML 2.2.x installation to a new installation of CML 2.6, you must deregister the old CML instance before decommissioning it. See Licensing - Deregistration for instructions on deregistering a CML instance.
- Deregister your existing CML 2.2.x instance.
- Install CML 2.6 (see below).
- Since you are migrating from an existing CML 2.2.x installation, mounting the reference platform ISO before running the CML 2.6 installation is optional.
- If the reference platform ISO is mounted before starting the CML 2.6 installation, the setup script will copy reference platform images from a mounted reference platform ISO to the local disk.
- You can skip the copying of reference platform images during installation because the migration tool will copy all the reference platform images from your CML 2.2.x installation to the new CML 2.6 installation.
- Only mount the new reference platform ISO if you want to add the new versions of the reference platforms in addition to the existing versions from your CML 2.2.x installation.
- Follow the instructions in the migration tool's README.md to migrate data from your existing CML 2.2 installation to your new CML 2.6 installation.
- License your new CML 2.6 installation.
The migration tool can be found on your CML 2.6 installation at /usr/local/bin/virl2-migrate-data.sh. You can also download the latest version of the migration tool from https://github.com/CiscoDevNet/cml-community/tree/master/scripts/migration-tool.
The migration tool also supports offline migration. If you want to install CML 2.6 on the same server that is currently running CML 2.2.x, you can use the Offline Version Migration instructions to back up the data to a .tar
file. That way, you can transfer the .tar
file to a separate datastore temporarily while you reinstall with CML 2.6. As always, make sure that you deregister your old CML 2.2.x instance before decommissioning it and starting the CML 2.6 installation.
Installing CML 2.6
The CML Installation Guide provides detailed steps for installing your new CML 2.6 instance.
- In CML 2.6, all the reference platform VM images must be copied to the local disk of the CML instance. Be sure to provision enough disk space for your new CML instance to accommodate the reference platforms.
- Downloading the refplat ISO file is optional if you are migrating from an existing CML 2.2.x instance. The migration tool will migrate all of your existing image definitions and VM images from your old CML 2.2.x instance.
- Be sure to check the System Requirements for updated system requirements.
- Important: After the reference platforms are copied to disk during the Initial Set-Up, wait a couple of minutes for the CML services to finish restarting before clicking OK to continue. If you click OK too soon, the initial set-up can fail. (See the Known Issues below.)
Demonstration of CML 2.6 Features
You can see a brief demo of the new CML 2.6 features in the following video:
Summary of Changes
This section details new and changed features in the new release. Bug fixes are listed in the subsequent section.
UI Changes
CML 2.6 introduces a completely redesigned Workbench UI to provide a more streamlined and unified experience when working with a lab. A new onboarding tour makes all the changes easier to understand for both existing and new users. The tour is displayed to each user on your CML instance the first time they log in and visit the Workbench after the 2.6 upgrade.
For details on using the Workbench, see the Workbench section in the CML User's Guide.
Panes
The PANES panel is a versatile tool designed to give you a flexible workspace. When you open a console, VNC session, or packet capture, the Workbench opens it in a tab in the active pane of the PANES panel. You can open multiple tabs in each pane. You may also open multiple panes side-by-side in the PANES panel so that, for example, you can have a console session open in one pane and a packet capture session running in another.
For details on using panes, see the Panes section in the CML User's Guide.
Sidebar
The sidebar provides a single, consolidated hub for viewing and modifying all of your lab elements' properties. Whether you want to edit a node, link, or annotation, simply select it in the Workbench canvas, and the sidebar will display the relevant properties and editing options.
This streamlined way to manage your lab elements eliminates the need for the bottom panel to shift in response to the workbench state, which gives you a more focused experience.
With this setup, you can keep your attention on the bottom pane, while simultaneously editing properties of a lab element using the sidebar.
Right-click Context Menu
The Workbench now exposes the most common actions for each lab element in a context menu. Activate the context menu by right-clicking a single element in the canvas. This menu presents a list of actions specifically tailored to the selected element type.
In addition, you can right-click the canvas without selecting any elements to display a menu of lab actions, such as Start Lab and Stop Lab. You can also access the lab actions via the Workbench toolbar by clicking LAB in the toolbar.
Multi-select Action Menu
When you select multiple elements in the Workbench canvas and then right-click, CML shows the multi-select action menu. This context-sensitive menu shows a distinct section for each element type. Within each section, you can perform actions on all selected elements of that type. For example, you can open the console for multiple nodes at once via the multi-select action menu. Please note, however, that the UI currently does not verify whether the action is valid for the selected elements. For example, if you select multiple nodes in the STOPPED state, the multi-select action menu shows an action to open node consoles, but that action is only valid for nodes in the STARTED and BOOTED states.
In addition to the regular node actions, when you select multiple nodes, the multi-select action menu includes an Alignment section, providing options for aligning the selected nodes.
Note that the context menu for a single selected element typically provides a greater number of actions than the multi-select action menu does for elements of that type.
Adding Nodes
You can still add new nodes to your lab in the Workbench by using drag-and-drop to drop nodes onto the canvas. This interactive drag menu was previously located on the right-side menu. In CML 2.6, you can activate the menu simply by clicking the Add Nodes
icon in the Workbench toolbar.
In addition to this traditional method of adding nodes, CML 2.6 introduces a brand-new Bulk Add feature. It's designed with accessibility in mind, so you can add as many nodes as you need using only your keyboard, eliminating the need to use a mouse when adding new nodes.
Keyboard Shortcuts
Keyboard Shortcut | Function |
---|---|
DEL | Opens a dialog to delete all selected elements |
ALT + C | Opens the Connection dialog, where you to choose source and destination nodes. Your selections automatically populate the next free interface. If you select two nodes before you open the dialog, it also automatically populates the source and destination nodes. |
ATL + N | Opens the Add Node dialog in the bulk mode, allowing the addition of one or more nodes to the canvas using only keyboard navigation. |
Improved UX for CML Cluster Deployments
CML 2.6 improves user experience, especially in cluster deployments. Dialogs that display data for all members in a cluster are now less complex and more compact. CML 2.6 introduces scrollbars in elements that can overfill the CML UI, such as the footer bar or health status bar.
In addition, CML 2.6 introduces changes to cluster members' state handling. Expected states do not supersede the overall state of the rest of the cluster. For example, if a cluster compute member is expected to be disconnected and the remaining cluster computes are healthy, no warnings are displayed in the UI.
PATty
CML 2.6 introduces a flexible new tool for port forwarding and on-board device breakout. PATty provides an alternative approach to exposing and accessing the consoles for your labs' nodes. PATty is disabled by default, and you should understand the security implications of PATty versus the Console Server and the Breakout Tool before you enable it.
For more information, see PATty Tool.
AWS Support
CML 2.6 makes running a CML instance on AWS a supported deployment option.
For more information, see CML on Amazon Web Services.
Maintenance Mode
CML 2.6 introduces the ability to put the CML instance into maintenance mode. This feature allows CML administrators to prevent all non-administrator logins and changes on the CML instance while they perform an upgrade or performing other system maintenance. In addition, the CML administrator can control an individual compute member's lifecycle by changing its admission state. For example, the REGISTERED state is used to keep a host disconnected during maintenance.
For more information, see System Maintenance in the CML Administrator's Guide.
Notices
In addition to the maintenance mode feature, CML 2.6 introduces the ability to notify users about important upcoming or ongoing events. Administrators can configure a title and content for each notice and select which users should be shown the notice. Administrators can monitor the state of notices and see whether specific users have acknowledged a system notice.
For more information, see System Notices in the CML Administrator's Guide.
External Connectors
In the CML 2.6 Workbench, the configuration for an external connector in bridge mode now provides a dropdown list that includes bridge names or aliases on the controller. That way, you no longer have to enter the name of a bridge device manually.
CML 2.6 introduces the ability to configure bridge parameters, such as IP snooping and bridge protection, in the CML UI. CML administrators can provide a label for defined bridges to provide logical names for the bridges and to insulate your users from the specific bridge names in use. The IP snooping functionality has also been extended to all bridges instead of only the system bridge bridge0
.
For more information, see External Connectors in the CML Administrator's Guide.
Persistent Link Conditioning
CML 2.6 introduces persistent link conditioning. In previous releases, you could configure link condition properties, such as link loss and bandwidth, but those properties were lost when you stopped the lab. Starting with CML 2.6, CML persists all link condition properties with the lab. The properties are also included with the exported topology: if you download the lab and import it again, the link condition properties will be preserved.
For more information, see Link Conditioning in the CML User's Guide.
Improvements to Node and Image Definitions
CML 2.6 introduces an option to make node definitions read-only or read-write, enabling you to prevent or allow changes to a node definition.
For more information, see Creating A New Node Definition in the CML Administrator's Guide.
API Changes
CML 2.6 introduces several changes to the REST-based web service APIs. For details about a specific API, log in to the CML UI and click Tools > API Documentation to view the CML API documentation page.
APIs Removed in CML 2.6
This table summarizes the API endpoints that have been removed in CML 2.6.
Removed API Endpoint | Use This API Endpoint Instead |
---|---|
GET /labs/{lab_id}/external_connections |
GET /system/external_connectors |
APIs Deprecated in CML 2.6
This table summarizes the API endpoints that have been deprecated in CML 2.6.
Deprecated API Endpoint | Use This API Endpoint Instead |
---|---|
POST /labs/{lab_id}/nodes/{node_id}/tags/{tag} |
PATCH /labs/{lab_id}/nodes/{node_id} |
DELETE /labs/{lab_id}/nodes/{node_id}/tags/{tag} |
PATCH /labs/{lab_id}/nodes/{node_id} |
New APIs in CML 2.6
This table summarizes the API endpoints that are new in CML 2.6.
API Endpoint | Description |
---|---|
GET /system/compute_hosts/configuration |
Get administrative state for new compute members (REGISTERED or READY). |
PATCH /system/compute_hosts/configuration |
Set administrative state for new compute members (REGISTERED or READY). |
GET /system/external_connectors/{connector_id} |
Get an external connector configuration and state. |
DELETE /system/external_connectors/{connector_id} |
Remove the external connector configuration from DB (not the system). |
PATCH /system/external_connectors/{connector_id} |
Update and apply external connector configuration. |
GET /system/maintenance_mode |
Get current status of maintenance mode. |
PATCH /system/maintenance_mode |
Set the state of maintenance mode. |
GET /system/notices |
Get a list of all notices. |
POST /system/notices |
Add a system notice for all or specific groups of users. |
GET /system/notices/{notice_id} |
Get the details for the specified notice. |
DELETE /system/notices/{notice_id} |
Delete specified notice. |
PATCH /system/notices/{notice_id} |
Update attributes or acknowledgements of the specified notice. |
PUT /node_definitions/{def_id}/read_only |
Set specified node definition's read-only value. |
PUT /image_definitions/{def_id}/read_only |
Set specified image definition's read-only value. |
GET /labs/{lab_id}/connector_mappings |
Get list of external connector mappings for the specified lab. |
PATCH /labs/{lab_id}/connector_mappings |
Set partial list of external connector mappings for the specified lab. |
PUT /system/external_connectors |
Get a list of external connectors. |
APIs Changed in CML 2.6
This table summarizes the APIs that have been changed in CML 2.6.
API Endpoint | Applies to request/response | Summary of Change |
---|---|---|
POST /authenticatePOST /auth_extendedPOST /usersPATCH /users/{user_id}GET /users/{username}/idPOST /system/auth/testGET /users/{username}/id |
request request request, response request, response request request response |
The maxLength of username set to real limit (32). |
POST /authenticatePOST /auth_extendedPOST /usersPATCH /users/{user_id}GET /users/{username}/idPOST /system/auth/test |
request request request, response request, response request request |
Removed maxLength for password. |
GET /labs/{lab_id}/annotationsPOST /labs/{lab_id}/annotationsGET /labs/{lab_id}/annotations/{annotation_id}PATCH /labs/{lab_id}/annotations/{annotation_id}GET /labs/{lab_id}/topologyPOST /import |
response request, response response request, response response request |
Set explicit minLength for color, border_color, text_content and text_font. Set an enum for border_style property. |
PATCH /labs/{lab_id}/annotations/{annotation_id} |
request | Type property can't be updated. |
GET /usersPOST /usersPATCH /users/{user_id}GET /users/{user_id} |
response response response response |
Updated response schema by adding labs and directory_dn properties. |
GET /usersPOST /usersGET /users/{user_id}PATCH /users/{user_id} |
response request, response response request, response |
Added property tour_version. |
PATCH /users/{user_id} |
request | Property old_password is mandatory but ignored when admin is changing another user's password. |
GET /groups/{group_id}/labs |
response | Finalized response schema for items returned. |
GET /licensing/certificatePOST /licensing/certificateGET /system/auth/configPUT /system/auth/config POST /system/auth/test |
response request response request request |
Schema format and maxLength has been changed for licensing certificate. |
GET /system/external_connectors |
response | Added finalized response schema. |
POST /labsGET /labs/{lab_id}PATCH /labs/{lab_id} |
response response response |
Added properties owner_fullname and owner_username to the response. |
GET /labs/{lab_id}/topologyPOST /import |
response request |
Added/updated schema with link conditioning properties. Added new property enabled. |
GET /populate_labsGET /populate_lab/{lab_id} |
response response |
Added topology property to the response. |
GET /resource_pool_usage |
response | Added local to the pattern for valid external_connectors names. |
PATCH /labs/{lab_id}/interfaces/{interface_id}PUT /labs/{lab_id}/links/{link_id} |
request request |
Marked as Not Implemented Yet. |
GET /labs/{lab_id}/links/{link_id}/conditionPATCH /labs/{lab_id}/links/{link_id}/condition |
response request, response |
Added new query parameter operational. |
GET /node_definitionsPOST /node_definitionsPUT /node_definitionsGET /node_definitions/{def_id}GET /node_definition_schemaGET /simplified_node_definitions |
response request, response request, response response response response |
Changed enum value cat9k to cat9000v. Removed lxc as possible libvirt_domain_driver enum value for sim property. Updated enum list of possible nic_driver values. |
PUT /node_definitionsPOST /node_definitions |
request, response request, response |
Removed read_only property. |
GET /node_definitions/{def_id} |
request | Set minLength, maxLength and pattern for valid def_id parameter. |
Other Changes
Summary | Release Notes |
---|---|
Cockpit button to switch between LDAP authentication and local authentication | Added a button in Cockpit that switches the authentication method back to 'local'. For example, if a CML instance is configured for LDAP authentication, it becomes difficult or impossible to log in to the CML instance if the LDAP server's certificate changes. With this button, the CML administrator can temporarily switch the CML instance back to local authentication and use the local admin account to log in to the CML UI and adjust the CML authentication LDAP settings. |
Drop cat9kv DD from bundled node definitions | Dropped support for Cat 9000v Doppler D. |
MTU properties not inherited from Parent interface | Node interfaces which are put into external connector bridges on the controller host now have MTU set to 9000; this allows for the bridge, its ports to the outside, as well as nodes' internal interface configuration to work when configured to MTUs larger than default. |
Support for Cat 9000v nodes in Build Initial Configurations | Added configuration template for Cat 9000v nodes. |
Store CML version in browser to allow auto-refresh | Implemented a feature to check current CML version and to reload a newer version if available. Starting from CML 2.6, UI sources will be reloaded after upgrade to a newer version. |
Client Library missing option for setting definition.read_only property | Added methods for setting definition read-only attribute to the Client Library. |
Update Cockpit text for upgrade feature | During upgrades from version 2.6.0, the pre-requisites on CML instance state will change; refer to the CML Installation Guide for recommended upgrade procedures for all CML versions. |
With LDAP auth, some LDAP bind failures are not detected | Added a catch statement to catch and report all LDAP errors. Previously, some errors were silently ignored and CML reported no issues, but users could not log in. Now, if a user cannot log in because of an LDAP error, an error will be written to the log. |
Export/import link conditioning | Link conditioning is now included in the exported lab YAML. |
Add all kvm-supported NICs to node definition model | Added options for custom node definitions to use less-common simulated network interface models. |
Read-only definition improvements | Added switch for admins in Node/Image Definition Editor to set definition to Read-Only or Read-Write. |
Improvements for TOOLS menu | TOOLS menu is now scrollable and has maximal height set, so all options are always selectable, even on smaller screens. |
Use PATCH node endpoint to update tags | Deprecated API endpoints POST/DELETE tag - use PATCH node instead. |
UX improvement for cloning nodes | Node/Annotation cloning feature was improved. |
Close VNC websockets when moving away from lab | All VNC sessions are closed when leaving the Workbench page. |
Client Library Implement LDAP support | Added LDAP support to client library. |
Client Library methods should allow inserting None as parameter values | Client Library - None can now be passed to some parameter update functions to erase said parameter if the API allows it. (e.g., client_library.user_management.update_user(id, resource_pool=None) ) |
IPSnooper: learn static addresses with dhcp on | Added multiple improvements to the IP snooping tool. |
Move licensing tech info to diagnostics page | Smart Licensing technical information was moved from System Administration page to Diagnostics page. |
Simplify status popup display | Refactored the footer status bar to improve the user experience. The bar has a maximal height set, so it doesn't cover the whole screen. A new color scheme was introduced to distinguish between intentionally (GRAY) and unintentionally (RED) disconnected cluster members. Refactored the health status bar to improve the user experience. The bar was replaced with an aggregate bar that includes the most concerning state of all cluster members. The more complex and detailed health status was moved to diagnostics page. |
Support Events in Client Library | In the Client Library: * added Events - a websocket-based connection to the server, which receives events and automatically updates the data in the Client Library, instead of periodically synchronizing over REST API * added Event Handling, an extensible mechanism for handling received events * deprecated .remove_on_server() in Node s, Link s etc. in favor of .remove() * protected local element modification functions, as they are only useful internally * added the ability in some functions to use IDs instead of objects and vice versa * various fixes for bugs and documentation in the Client Library |
Find reference platforms on devices other than sr0 | Improved scripts handling refplat and configuration ISO images to properly look for such images based on their volume label (REFPLAT or CMLINIT). |
Add configuration file for client library | Client Library: Added a configuration file, '.virlrc'. |
Bug Fixes
Bug Description | Resolution |
---|---|
Assign unmapped external configuration keys if a matching bridge is detected | Fixed in 2.6.1. Fixed an issue where external connector bridges which were not yet used in a lab were not made usable in that lab. |
A crash of the virl2-licensing service can break SLR licensing | Fixed in 2.6.1. Fixed a bug that prevented expansion licenses from being properly reloaded after a crash. This bug affected only customers using expansion licenses mainly with SLR. |
Problems uploading the .pkg on MacOS | Fixed in 2.6.1. Changed PKG and ISO artifact extensions to resolve issue with some package managers on MacOS. Specifically, changed .pkg.zip to -pkg.zip and .iso.zip to -iso.zip. |
After completing a 2.6.0 upgrade on a standalone CML instance, the compute is not "ready" | Fixed in 2.6.1. Added a warning to prevent users from disabling the maintenance mode and leaving compute hosts in the REGISTERED state. Both conditions must be satisfied, and the warning advises admins to first move all hosts to the READY state and to disable the maintenance mode afterwards. |
Failed to load notices on 401 unauthorized | Fixed in 2.6.1. Fixed a bug that displayed "Failed to load notices" error notification on the login page. |
The health status reports issue when hosts are REGISTERED | Fixed in 2.6.1. Fixed some corner case false positives reported by the system status in UI. Specifically, the system status falsely reported issues (RED) after upgrade when all compute hosts are in the REGISTERED state. Now it correctly reports Not Ready state (GRAY). Similarly, when a single host is in the REGISTERED state and is faulty, the host is not considered in the system status and the system status reports no issues (GREEN) as long as there are no issues with compute hosts in the READY and ONLINE states. |
Node Editor Visual Issues | Fixed in 2.6.1. Fixed various visual issues in the node configuration editor. |
Create Nodes and Create Links keyboard shortcuts don't work on MacOS | Fixed in 2.6.1. Fixed broken "Option + N" and "Option + C" shortcuts on MacOS. |
Configuration of new nodes is reset to defaults after upgrade from 2.4 | Fixed in 2.6.1. Fixed a bug that reset configuration of new (wiped) nodes after an upgrade from CML 2.4.1 or earlier release to CML 2.6.0. The bug didn't affect stopped nodes or new nodes created in CML 2.5 releases. |
CML install skips initial setup and boots to the login prompt | Fixed in 2.6.1. Fixed one scenario where in OVA deployments, if the VM was shut down before reaching the initial setup dialogs, then initial setup could not be started after booting again. Despite this, customers should never interrupt the initial setup, and may have to redeploy in case the setup is broken after doing this. |
Ignore authorization header for calls which don't need one | Fixed an issue where the login page could not fetch maintenance mode and system information if it sent authorization from a previous login session. |
Add labels to node pane table elements | Fixed in 2.6.1. Added labels to the node pane table elements as a11y improvements. |
virl2-client's upload_image_file strips the .qcow2 extension | Fixed in 2.6.1. Fixed a regression that stripped the extension of images uploaded via client library. |
Bulk Add Nodes does not avoid node name conflicts | Fixed in 2.6.1. Fixed a bug in the new BULK node addition feature. The feature incorrectly tried to add existing nodes and failed. Now, it correctly skips existing nodes. |
Hostpubkey schema is too strict, locks UI if an SSH hostkey doesn't match | Fixed in 2.6.1. Fixed a bug where some additional public ssh keys installed by the admin into compute hosts caused compute registration to be rejected by API validator. |
A non-admin user must not be able to change some properties | Fixed in 2.6.1. Non-admins can no longer change their own usernames and resource pools. |
Generic error http 204 logged in controller log for multiple calls | Fixed in 2.6.1. Lowered log level for certain successful API calls that were logged as errors. |
Fix unhandled events in controller log | Fixed in 2.6.1. Removed unnecessarily logging certain ignored events as warnings. |
Fix lvm-maximize script for NVMe drives | Fixed in 2.6.1. Fixed a bug where the installation on NVMe drives would not expand onto all available space for the /var partition. |
NAT stops working after firewalld restart | Fixed in 2.6.1. Fixed an issue where NAT rules were not recreated after firewall reload/restart in some scenarios. |
virl2-sshd-cluster service is not restarted in v-i-s.py | Fixed in 2.6.1. Fixed an issue on cluster where changing the cluster internal network interface after initial setup did not restart the internal ssh service for cockpit integration. |
Remove tar and rm commands in /etc/sudoers.d/virl2-sudo |
Clarified the uses of sudo within system services and Cockpit maintenance scripts. |
Evaluate issues reported by rapid 7 scanner for CML UI | Fixed some issues reported by automated web security scanning software related to updated methods of declaring webpage content security policies and HTTP headers of redirect responses. |
CML upgrades failed when the firewalld service wasn't running. |
Fixed a bug where the CML2 package installation assumed that firewalld service is running; the required service is now started before firewall adjustments are made. |
Trying to reset link conditioning on stopped lab returns bad code | Fixed being unable to delete link conditioning on links that were never started. |
Cluster upgrade should not delete CML artifact unless completed | Provided upgrade package will not be deleted when software upgrade is unsuccessful. User can rerun upgrade, and it will be deleted once the upgrade was successful on all computes. |
Fabric segfault on remotemux UDP send | Fixed a segmentation fault in fabric's concurrent handling of link stop while sending traffic, causing inconsistent state for unrelated links. |
Failed node wipe on wiping its links | Fixed a bug where stopping, wiping and removing multiple connected nodes at the same time could fail when their common links were already removed by one of the processes. |
Handle ext conn with None configuration | Fixed an issue where external configuration node got into an inconsistent configuration state and failed to start. |
Some features are broken for non-admin users due to missing enterprise license | Fixed dashboard lab loading issues when enterprise license is missing. |
Dashboard list view does not show full name when LDAP is enabled | In the Dashboard when "Show List" is enabled, a new column "Owner Name" can be configured in the table settings containing the lab owner's full/display name. This is useful in case of some LDAP setups where the CML username, displayed in the default "Owner" column, does not clearly identify a user. |
Client Library auth methods missing or in wrong place | Client Library: Split LDAPManagement into AuthManagement (which handles synchronization and gives auth-method-agnostic ways to modify authentication settings) and auth method managers under AuthManagement.manager (which allows for each auth method to have its own parameters). |
Test Authentication LDAP error message leaking | Fixed the LDAP test validation dialog that was leaking response data for some errors (no security issue, just visual issue). |
Error message displayed after visiting User settings | Fixed a permission denied error on User Settings page that was displayed for non-admin users. |
Compute disconnect may not get propagated | Fixed a bug where a compute was not marked as disconnected in some communication failure and timeout scenarios. The controller was still trying to use the compute, accumulating failure reports. |
Cannot add links to unmanaged switch | Fixed a bug where firewall service restart would break creation of links to the NAT external connectors and unmanaged switches due to loss of non-persistent configuration generated for those bridges by libvirt. |
Repeated annotation cloning doesn't work as expected | Fixed a minor bug related to annotation cloning. Now the new annotation is correctly selected while in the past the original annotation was incorrectly selected. |
Cockpit cypress test leaking bridge memory on computes | Fixed memory leak on cockpit-bridge processes. |
Compute doesn't get IP when started after poweroff | Fixed a bug where compute member hosts' network connections failed after a poweroff; this is fixed for new installations only, or if the administrator completes the IP reconfiguration procedure. |
Fix Core controller add physical interface slot and label handling | Fixed bugs and clarified API documentation around lab node interface creation via API, lab import and node start. Interface slots and labels, where provided, must be consistent with the current node definition. The APIs will create all interfaces without gaps in the interface sequence, even if the sequence is out-of-order and incomplete on import. Changing a used node definition's interface list is not recommended. Any pre-existing wiped node whose interface list becomes inconsistent will be rejected from starting until the node is fixed by removing all invalid interfaces. |
Some LLD readiness data is static | Update compute info in System Status bar when compute is disconnected or its state is changed. |
Fix virl2.issue in setup with custom bridges | Fixed a small bug in CML controller console login message, where now only the IPv4 and IPv6 addresses of the "bridge0" interface will be displayed instead of an IP configured on a randomly-selected interface. |
Interface names do not match real names | Imported topologies will be rejected if they include interface labels which do not match the node definition. |
Wrong notifications order | Fixed wrong lab notification order. |
Known Issues and Caveats for CML 2.6
Bug Description | Notes |
---|---|
Newly-created custom bridges do not show up in the Config dropdown for an External Connector node. | Fixed in 2.6.1. This bug only impacts users who are using external connectors in bridge mode with custom bridges (i.e., bridge interfaces other than bridge0 ). |
In CML instances that use SLR licensing, a bug in the licensing implementation can cause an error that puts the CML instance into an Out of Compliance licensing state. | Fixed in 2.6.1. This bug impacts CML 2.5.0, 2.5.1, and 2.6.0 instances that use SLR licensing. There is no specific condition that triggers this bug, and it can occur at any time while the CML instance is running. When a CML instance hits this bug, the initial symptom is that the license status changes to Out of Compliance. In this state, CML will continue to run nodes that were already started, but users will not be able to start any additional. The error message will reference the license status or quota. When the CML administrator checks the Tools > Licensing page, the Count of the CML - Expansion Node License or CML - Expansion User License will have been reset to 0 . Workaround: When you encounter this problem, log in to Cockpit and Restart CML Services. You do not need to stop running labs. Once the services restart, log in to the CML UI as a user with administrator privileges check the Tools > Licensing page. The License Authorization Status should now be Authorized , and the Count should show the correct value for each license. |
External connectivity does not work in CML bare metal installations. | This bug impacts CML bare metal deployments from CML 2.3.1 up to and including CML 2.6.1. There is no specific condition that triggers this bug, and it can occur at any time while the CML instance is running. In some deployments, most notably on UCS bare metal servers in which the vNIC's vlan is set to access mode , the ingress traffic is tagged with vlan 0 . This vlan is meant to carry priority information as per the 802.1p standard. Network services can normally ignore this tag and handle vlan 0 packets transparently as untagged (no vlan), but some node types in CML do not handle it gracefully, ignoring the incoming packets. As a result, the virtual nodes connected to an external-connector node may not acquire an IP address from an external DHCP server or may not be able to communicate with external devices. Workaround: log in to Cockpit and click Terminal in the navigation bar. Identify the physical interface used for external connectivity, replacing the <bridge-name> in the following command to match the bridge used in your lab: ip link show master <bridge-name> Use that physical interface name to replace <interface-name> in each of the following commands to resolve the issue: tc qdisc add dev <interface-name> handle ffff: ingress tc filter add dev <interface-name> parent ffff: basic match "meta(vlan mask 0xfff eq 0)" action vlan pop Note: This bug will be fixed in CML 2.7, and you will need to revert these changes before upgrading to 2.7 by running this command: tc qdisc del dev <interface-name> handle ffff: ingress |
When you upload the CML upgrade PKG file on the Tools > Upgrade System page, the upload fails with an error indicating Insufficient space available on controller and Bytes Available 0 . |
If you're upgrading from CML 2.6.0 and put the controller into Registered state before you uploaded the PKG file, then you will always get this error. Workaround 1: You can put the controller back in Online state on the Tools > System Administration > Compute Hosts page. Then return to the Tools > Upgrade System page to upload the PKG file. Once the file finishes uploading, you can put the controller back into Registered state and proceed with the upgrade. Workaround 2: Alternatively, you can use the scp method to upload the PKG file to your CML server. For example: scp cml2_2.6.1-11_amd64-11.pkg admin@172.16.0.10:/ . |
After a restart, you cannot reach the CML UI. The console of the CML instance shows a login prompt, but it also indicates that CML does not have an IP address. | After you shutdown and restart a CML instance, it sometimes loses its IP address. When that happens, you will no longer be able to reach the CML UI even though the CML services are running. This bug affects all 2.6.X instances. Workaround: When you encounter this problem, open the CML console (e.g., the CML VM's console in VMware or the CML server's remote management console). Log in as the sysadmin user and run this command: sudo netplan apply You can verify the current/new IP address of the CML instance using this command: ip addr show bridge0 If the issue persists, run the following command to call the initial setup script and complete the initial setup wizard to resolve the issue: sudo HOME=/var/local/virl2 /usr/local/bin/virl2-initial-setup.py --reconfigure |
On MacOS, the Tools > Update System page won't let you select the cml2_2.6.0-5_amd64-6.pkg file for upload. | Fixed in 2.6.1. This problem is caused by the special treatment for folders with a name that ends in .pkg on MacOS and the way Archive Utility.app names files that it unzips. Workaround: If you extracted the CML .pkg.zip file by double-clicking the file in Finder.app , you will end up with a folder whose name ends in .pkg. Rename that folder from cml2_2.6.0-5_amd64-6.pkg to cml2_2.6.0-5_amd64-6-pkg , for example. You will then be able to drill down into that folder and select the CML2 .pkg upgrade file for upload. |
On MacOS, Create Nodes and Create Links keyboard shortcuts don't work. | Fixed in 2.6.1. Workaround: it is possible to work around this problem by creating an alternate keyboard map. See, for example, https://stackoverflow.com/a/16019737/5374843. |
Adding nodes in Bulk mode fails with the error: Node label must be unique . |
Fixed in 2.6.1. The new Bulk Add feature for adding nodes does not avoid name conflicts. It adds nodes one at a time and assigns names to the nodes by appending -0 , -1 , -2 , etc. to the user-provided Name prefix. Bulk Add stops adding nodes if the generated name is already used by another node in the lab. For example, you could not use the node name prefix iosv if there is already a node in the lab named iosv-0 . Workaround 1: When using the Bulk mode to add nodes, use a Name prefix that is not used by any other nodes in the lab. You change the node labels after the nodes are added to the lab. Workaround 2: When adding nodes, use the Drag mode instead of the Bulk mode. The drag-and-drop approach to adding nodes is not impacted by this bug. |
scp to port 22 fails with error subsystem request failed |
The service that runs on port 22 of your CML server only supports scp with the SCP protocol. Some scp clients use the SFTP protocol by default. Workaround: If you're using a newer version of an scp client that defaults to SFTP, see whether it provides an option, such as -O , to use the older SCP protocol. If so, retry the copy operation using scp -O . |
VM image upload with the virl2-client library renames the file so that the name of the file on the CML server is missing the file extension. | Fixed in 2.6.1. This bug affects both virl2-client 2.5.0 and virl2-client 2.6.0. When your code calls the client_library.definitions.upload_image_file function, the default behavior is to create a file on the CML server that is missing the suffix. For example, if you upload my_img-7.3.1.qcow2 , upload_image_file strips the .qcow2 extension. The file on the CML server after upload would be my_img-7.3.1 . Workaround: You can specify the filename that should be used on the CML server by passing the filename as a string with the optional rename parameter. |
Cannot run simulations on a macOS system that uses Apple silicon (e.g., the Apple M1 chip). | CML and the VM Images for CML Labs are built to run on x86_64 processors. In particular, CML is not compatible with Apple silicon (e.g., the Apple M1 chip). Workaround: There are no plans to support ARM64 processors at this time. Deploy CML to a different system that meets the system requirements for running CML. |
Even if you were able to run network simulations with CML 2.2.x on VMWare Fusion and macOS Big Sur or above, the same labs no longer start after migrating to CML 2.3.0+. | A default setting in the CML VM changed between CML 2.2.3 and CML 2.3.0. Because of the new virtualization layer in macOS 11+, this setting causes errors when running the CML VM on VMware Fusion. Workaround: follow the instructions to disable nested virtualization in the CML VM. |
Bug in macOS Big Sur. After upgrading to Big Sur and VMware Fusion 12.x, simulations no longer start. | MacOS Big Sur introduced changes to the way macOS handles virtualization, but the problems were resolved in later versions of macOS Big Sur and VMware Fusion. Workaround 1: If you are still experiencing problems, be sure to upgrade to the latest version of VMware Fusion 12.x and the latest bug fix release of your current macOS version. Workaround 2: If you are using CML 2.3+, then follow the instructions to disable nested virtualization in the CML VM. |
Deleting or reinstalling a CML instance without first deregistering leaves the license marked as in use. Your new CML instance indicates that the license authorization status is out of compliance. | If you have deleted or otherwise destroyed a CML instance without deregistering it, then the Cisco Smart Licensing servers will still report that instance’s licenses are in use. Workaround 1: Before you destroy a CML instance, follow the Licensing - Deregistration instructions to deregister the instance. Properly deregistering an instance before destroying it will completely avoid this problem. Workaround 2: If you have already destroyed the CML instance, see the instructions for freeing the license in the CML FAQ. |
When the refplat ISO is not mounted until after the Initial Setup steps have started, CML will not copy node and image definitions during the initial setup process. | Workaround: After you finish the Initial Setup steps and the CML services have restarted, you can copy the node and image definitions from the ISO image to your CML instance by following the instructions on Copy the Refplat ISO to Disk. |
When the incorrect product license is selected, the SLR (Specific Licensing Reservation) still can be completed. It eventually raises an error, but the SLR reservation will be complete with no entitlements. | Workaround: If you accidentally select the wrong product license, then you will need to release the SLR reservation and then correct the product license before trying to complete the SLR licensing again. |
If you have read-only access to a shared lab, some read/write actions are still shown in the UI. Attempting to change any properties or to invoke these actions raises error notifications in the UI. | Workaround 1: Do not attempt to invoke the read/write actions when working with a lab for which you only have read-only permissions. Workaround 2: Ask the lab owner to share the lab with read/write permissions so that you can make those changes to the lab. |
If you have read-only access to a shared lab, you will still not be able to see the current link conditioning properties in the link's Simulate pane. | Workaround: ask the lab owner to share the lab with read/write permissions if you need to access the link conditioning properties. |
The algorithm for computing the available memory does not take into account maximal possible memory needed by all running nodes. It only checks the currently allocated memory. CML will permit you to start nodes that fit within the current available memory of the CML server. Later, if the memory footprint of the already-running nodes grows and exceeds the available memory on the CML server, those nodes will crash. | Workaround 1: Launch fewer nodes to stay within your CML server's resources. Workaround 2: Add additional memory to the CML server to permit launching additional nodes. Of course, this workaround just moves the point where the CML server will eventually hit its limit, so you must remain aware of your CML server's resource limits or ensure that the CML server has sufficient memory to accommodate all labs that will be running on your CML server simultaneously. |
When starting a CSR 1000v node for the first time or after a Wipe operation, the interfaces are all in shutdown state. |
Workaround 1: Add no shut to each interface configuration in each CSR1000v node's Edit Config field in the UI. Workaround 2: Use the EEM applet on the CSR 1000v page. |
New nodes cannot be started when CPU usage is over 95%, when RAM is over 80%, or when disk usage is over 95%. | Workaround: Allocate additional resources to the CML server. Workaround 2: If you're hitting the disk usage limit, check the Remove Orphaned VMs function in Cockpit. Orphaned VMs can consume a significant amount of disk space even though CML is no longer using or managing those VMs. |
VM Images uploaded via the API can only be used in an image definition if they are transmitted via a multipart/form-data request. The problem is that the SELinux context is not set correctly. |
Workaround 1: Use the Python Client Library, which handles image uploads correctly. Workaround 2: Create a multipart/form-data request when calling this API. |
The journalctl output shows errors like this: cockpit-tls gnutls_handshake failed: A TLS fatal alert has been received |
This log message only occurs with Chrome browsers and has no functional impact on the use of Cockpit. No workaround needed. |