CML 2.7 Release Notes

Cisco Modeling Labs (CML) is a network simulation platform. CML 2.7 is the latest feature release of CML.

New and Changed Information

This table shows any published changes to this CML 2.7 release notes document.

Date Changes
January 31, 2024 CML 2.7.0 Beta test begins, build.1643.
March 27, 2024 Updated with the CML 2.7.0 FCS release.
April 15, 2024 Updated to warn about the Insufficient space error when upgrading from 2.6.x, make minor corrections, add some FAQs, and add pages for IOL and IOL-L2.

Installing or Migrating to CML 2.7

Upgrading to CML 2.7

If your CML instance is already running CML 2.3.0 or later, you can upgrade it to CML 2.7.0. See the In-Place Upgrade page for detailed upgrade instructions.

  • We recommend reviewing the Known Issues and Caveats before upgrading to a new CML release. The latest release may include bugs that impact specific features or CML installation types. Depending on how you deploy and use CML, you may want to skip a release and wait until a specific bug is fixed in a future maintenance release.
  • If your CML instance is already running CML 2.3.0 or later, you can upgrade it to CML 2.7.0. If you haven't upgraded to CML 2.6.x yet, see the CML 2.6 Release Notes for more details about changes to the upgrade process starting in CML 2.6.0.
  • Don't forget to disable maintenance mode once the upgrade is complete.
  • Note that if you are upgrading from CML 2.6.x, you must follow the steps in the order documented in In-Place Upgrade. Otherwise, you may encounter an Insufficient space available error when uploading the PKG file. See also the Troubleshooting FAQ.

In-place upgrades from CML version 2.2.3 or earlier are not supported. In that case, a migration is required. The migration process from CML 2.2.3 is detailed in the CML 2.6 Release Notes.

Upgrading a CML Cluster to CML 2.7

If you have deployed a CML cluster, you may upgrade it to CML 2.7.0.

When upgrading from CML 2.4.0, which introduced cluster support, you must manually run the CML upgrade first on the controller and then separately on each compute member.

If you are upgrading from CML 2.5.0+, you only need to run the CML upgrade on the controller. The CML controller's upgrade automatically initiates a CML upgrade on the cluster's compute members.

Installing CML 2.7

The CML Installation Guide provides detailed steps for installing your new CML 2.7 instance.

  • In CML 2.7, all the reference platform VM images must be copied to the local disk of the CML instance. Be sure to provision enough disk space for your new CML instance to accommodate the reference platforms.
  • Be sure to check the System Requirements for updated system requirements.
  • Important: After the reference platforms are copied to disk during the Initial Set-Up, wait a couple of minutes for the CML services to finish restarting before clicking OK to continue. If you click OK too soon, the initial set-up can fail. (See the Known Issues below.)

Summary of Changes

This section details new and changed features in the new release. Bug fixes are listed in the subsequent section.

IOL Support and Images

Starting in CML 2.7, the standard ISO of reference platform images includes two IOL node definitions and their associated images. Node definitions and images are included both for an L3 router (IOL) and for an L2/L3 switch (IOL-L2) that run IOS XE 17.12.01. Nodes of these two types do not run as full virtual machines (VMs). Therefore, they consume fewer resources, and using these new IOL node types will permit you to run more nodes at once (up to your license limit) on your existing CML server.

Node Types with Multiple Configuration Files

CML 2.7 expands the support for editing a node's day-0 configuration to permit modifying multiple day-0 configuration files on a node in your lab.

The first time you start a new node in a lab (or the first time you start a node after wiping it), CML can provision a day-0 configuration for the VM. The node definition associated with the node defines which file or files CML should provision for the new VM. You can specify the contents of each node's day-0 configuration in the Workbench, and the day-0 configuration contents are stored with the associated node in your lab.

In earlier versions, CML could provision multiple day-0 configuration files and provide them to the running node's VM, but CML only permitted you to edit one of those files on each node. Starting in CML 2.7, a CML administrator may now mark multiple filenames in a node definition's Provisioning section as Editable. You can then select which configuration file to edit in the Workbench and edit the day-0 configuration text for that file. You may specify the contents for each editable configuration file on all nodes of that type.

Examples of node definitions that can benefit from multiple editable configuration files are Ubuntu (user-data and meta-data files) and Cat9000v (iosxe_config.txt and conf/vswitch.xml files).

Specify SMBIOS Parameters

CML 2.7 introduces an option to specify SMBIOS parameters for a node definition or for a node itself. SMBIOS customization adds a layer of control, enabling users to tailor the virtualized environment to their specific requirements. This feature allows you to simulate different hardware profiles for nodes or influence how operating systems identify the virtualized environment by adjusting SMBIOS parameters, such as manufacturer, product name, version, and serial number. It also allows you to run VM images that require specific SMBIOS parameters, such as a Juniper Switch.

For more information, see Node parameters specification and Node definition parameters specification.

CML supports setting any of the SMBIOS system information values on a node.

UI Changes

Other than UI changes to support the new features mentioned above, there are some other small UI changes and bug fixes in CML 2.7:

  • You can now drag the Sidebar to change its width in the Workspace. In particular, this feature lets you make the Config pane wider while you're editing or reviewing the 0-day configurations for nodes in your lab.
  • Double-clicking a node no longer hides / unhides the node's links.
    • The functionality is still available in CML 2.7: right-click the node and select Hide Links from the node's context menu.
    • In previous releases, you could Hide Links by double-clicking a node, but CML users found it confusing when they accidentally double-clicked a node.
  • When you have enabled Hide Links on a node, all of its links are shown when the hidden node is selected.
  • You may now right-click a bundled link in the Workbench and invoke an action on all of the links in the bundle. For more about link bundling, see the Hiding Links page.
  • The Workbench saves your annotations when you leave the page.
    • If you are changing an annotation's property in one of your labs, and you navigate away from the page (for example, to return to the Dashboard), CML 2.7 saves your latest change to the annotation.
    • You no longer need to click elsewhere in the lab after updating an annotation property to ensure that the change will be saved.

External Connectivity Editor

CML 2.7 extends the functionality of L2 external connectors with the ability to modify the built-in NAT network IP range. CML system administrators can also use the new External Connectivity Editor to add new network bridges to the CML host. These managed bridges are attached to an internal DHCP server and optionally use network address translation to allow connected lab nodes to reach outside resources.

For more information, see External Connectors in the CML Administrator's Guide.

SD-WAN Support and Images

With the release of CML 2.7.0, CML provides Catalyst SD-WAN images to our users on a new ISO file of supplemental VM images. While previous versions of CML could run the Catalyst SD-WAN images, a CML license did not provide access to those VM images. Therefore, only users who had access to the SD-WAN images via some other entitlement could download them and add them to CML.

Note that if you were using the contributed SD-WAN node definitions from the cml-community repository, the node definitions have been updated along with this new release. The CML 2.7 upgrade will not migrate labs that use the old viptella-vmanage node definition to the new cat-sdwan-manager node definition. If you want to use the new node definitions and SD-WAN images, you will need to migrate your labs manually.

Manual Compute Host Selection or Node Pinning

CML 2.7 introduces the ability to pin a node to a specific compute host.

When you start a lab on a CML cluster, the CML controller assigns nodes to specific compute hosts automatically. This automatic compute host assignment can sometimes lead to suboptimal distribution of nodes to computes. For example, multiple resource-intensive nodes may all be assigned to the same compute host instead of being distributed evenly across the cluster's compute hosts. The compute pinning feature allows administrators to bypass the automatic compute assignment and choose the best compute host themselves.

A CML administrator may only pin a node to a compute host if the node has no persistent state on a compute host. That is, you can only pin a node to a compute host if the node is new and has never been started or if the node was wiped and has not been started since being wiped. A pinned node will only run on the specified compute host even if it is wiped, and a pinned node will fail to start if its associated compute does not have the capacity to run the node, even if there is capacity to run the node on another compute host.

For more information, see Compute Locking in the CML Administrator's Guide.

Running CML on AWS or Azure

CML 2.7 makes running a CML instance on Azure a supported deployment option. This option builds on the support for deploying CML to Amazon Web Services (AWS) that was introduced in CML 2.6.

For more information, see the cloud-cml page.

Smart Licensing Transport

CML 2.7 introduces a new transport module, designated as Smart Transport, within Smart Licensing. This module replaces the previous transport module, Call Home, utilized in CML 2.6 and below. The transition to Smart Transport is designed to ensure that CML can use the latest versions of Cisco's Smart Licensing technology.

The upgrade process to CML 2.7 seamlessly completes the transition from Call Home to Smart Transport. This ensures a smooth and hassle-free experience for CML administrators adopting the latest version. As part of the CML upgrade process, default direct transport settings are automatically updated. This involves switching the URL from Call Home to Smart Transport, aligning the system with the enhanced transport capabilities offered by the new module.

This change only impacts CML instances that utilize the default direct transport settings or a transport gateway (e.g., SSM OnPrem). CML instances configured to not communicate with CSSM (i.e., instances configured to use SLR) remain unaffected by this transition.

After you upgrade your CML instance to 2.7, we recommend using Actions > Renew License Authorization Now on the Tools > Licensing to verify that the new license settings are working.

API Changes

CML 2.7 introduces several changes to the REST-based web service APIs. For details about a specific API, log in to the CML UI and click Tools > API Documentation to view the CML API documentation page.

APIs Removed in CML 2.7

This table summarizes the API endpoints that have been removed in CML 2.7.

Removed API Endpoint Use This API Endpoint Instead
GET /licensing/certificate None
POST /licensing/certificate None
DELETE /licensing/certificate None

APIs Deprecated in CML 2.7

This table summarizes the API endpoints that have been deprecated in CML 2.7.

Deprecated API Endpoint Use This API Endpoint Instead
GET /diagnostics Any diagnostics endpoint mentioned in New APIs section.
GET /labs/{lab_id}/nodes/{node_id}/tags/ GET /labs/{lab_id}/nodes/{node_id}

New APIs in CML 2.7

This table summarizes the API endpoints that are new in CML 2.7.

API Endpoint Description
GET /diagnostics/computes Return diagnostic information for computes.
GET /diagnostics/labs Return diagnostic information for labs.
GET /diagnostics/lab_events Return diagnostic information for lab events.
GET /diagnostics/node_launch_queue Return diagnostic information for the node launch queue.
GET /diagnostics/services Return diagnostic information for services.
GET /diagnostics/node_definitions Return diagnostic information for node definitions.
GET /diagnostics/user_list Return diagnostic information for the user list.
GET /diagnostics/licensing Return diagnostic information for licensing.
GET /diagnostics/startup_scheduler Return diagnostic information for the startup scheduler.

APIs Changed in CML 2.7

This table summarizes the APIs that have been changed in CML 2.7.

API Endpoint Applies to request/response Summary of Change
GET /users
GET /users/{user_id}
POST /users
PATCH /users/{user_id}		 response
response
request, response
request, response		 Added property pubkey_info.
PUT /sample/labs/{lab_name} request Set minLength, maxLength and pattern for valid lab_name parameter.
PUT /sample/labs/{lab_name} response Finalized response schema for labs returned.
GET /system/compute_hosts
GET /system/compute_hosts/{compute_id}
PATCH /system/compute_hosts/{compute_id}		 response
response
response		 Removed pattern from property hostname and server_address.
GET /system/external_connectors
PUT /system/external_connectors
GET /system/external_connectors/{connector_id}
PATCH /system/external_connectors/{connector_id}
GET /labs/{lab_id}/connector_mappings
PATCH /labs/{lab_id}/connector_mappings		 response
response
response
response
response
response		 Added new property allowed.
GET /system/external_connectors
PUT /system/external_connectors
GET /system/external_connectors/{connector_id}
PATCH /system/external_connectors/{connector_id}		 response
response
response
response		 Removed pattern from property device_name
Removed pattern from operational property ip_networks.
PUT /system/external_connectors response Schema now recognizes multiple different types of data (string, object).
GET /node_definitions
GET /node_definitions/{def_id}
GET /simplified_node_definitions
GET /node_definition_schema
PUT /node_definitions
POST /node_definitions		 response
response
response
response
request
request		 Added new enum values to libvirt_domain_driver: iol, lxc.
GET /node_definitions
GET /node_definitions/{def_id}
GET /simplified_node_definitions
GET /node_definition_schema
PUT /node_definitions
POST /node_definitions		 response
response
response
response
request
request		 Added new property min_count.
GET /node_definitions
GET /node_definitions/{def_id}
GET /simplified_node_definitions
GET /node_definition_schema
PUT /node_definitions
POST /node_definitions		 response
response
response
response
request
request		 Added new property parameters (optional list of smbios key:value parameters).
PUT /node_definitions
POST /node_definitions
GET /node_definitions
GET /node_definitions/{def_id}
GET /simplified_node_definitions
GET /node_definition_schema		 request
response
response
response
response
response		 Removed driver enum values: iol, iol-2
POST /labs/{lab_id}/interfaces
GET /labs/{lab_id}/nodes/{node_id}/interfaces		 response
response		 Changed mac_address property from maxLength to pattern.
POST /labs/{lab_id}/interfaces
GET /labs/{lab_id}/nodes/{node_id}/interfaces
GET /labs/{lab_id}/interfaces/{interface_id}		 response
response
response		 Updated schema with operational property.
GET /labs/{lab_id}/topology
GET /nodes
GET /labs/{lab_id}/nodes
GET /labs/{lab_id}/nodes/{node_id}
POST /import
POST /labs/{lab_id}/nodes		 response
response
response
response
request
request		 Changed value of property configuration from False to null.
GET /labs/{lab_id}/topology
GET /nodes
GET /labs/{lab_id}/nodes
GET /labs/{lab_id}/nodes/{node_id}
PATCH /labs/{lab_id}/nodes/{node_id}
POST /import
POST /labs/{lab_id}/nodes		 response
response
response
response
response
request
request		 Added new parameters property in topology's "nodes" dict.
GET /labs/{lab_id}/simulation_stats response Updated schema for response code 404 (Not Found).
GET /nodes
GET /labs/{lab_id}/nodes
GET /labs/{lab_id}/nodes/{node_id}		 Updated schema parameters by including property exclude_configuration.
GET /nodes
GET /labs/{lab_id}/nodes
GET /labs/{lab_id}/nodes/{node_id}		 Updated schema parameters by updating property operational.
GET /nodes
GET /labs/{lab_id}/nodes
GET /labs/{lab_id}/nodes/{node_id}		 response
response		 Added new property iol_app_id.
GET /labs/{lab_id}/nodes/{node_id}
PATCH /labs/{lab_id}/nodes/{node_id}		 response
response		 Added new property pinned_compute_id.
GET /labs/{lab_id}/interfaces/{interface_id} response Changed mac_address property from maxLength to pattern.
PUT /labs/{lab_id}/links/{link_id}/capture/start request Updated schema with request body.

Other Changes

Summary Release Notes
Enhanced LDAP Error Reporting The LDAP test authentication API now provides detailed error messages for incorrect LDAP configurations, rather than generic errors.
Node Double-Click Behavior Removal The confusing double-click functionality that hid/showed node links and altered node outlines has been removed. This feature is now only accessible through the node's right-click context menu.
Cytoscape Performance Enhancements Optimizations have been made to improve the lab preview and page loading times, as well as the rendering of links in the workbench.
Improved Sample Lab Error Messaging The sample lab loader now displays a specific error message in the UI when required node definitions are missing, improving user feedback.
LXC and Network Migration Script A new migration script supports transferring CML lab data and compute hosts within clusters for version 2.7.0. It is not designed for other upgrades or merging multiple CML instances.
Simplified Workbench Pane Closure The cross icon in the workbench's PANES pane now closes the pane when only one is open for a more intuitive user experience.
PCL Annotation Support Annotations are now supported in the client library, enhancing documentation and note-taking capabilities.
Enhanced Dark Mode Text Selection Text selection within the node configuration tab is now more visible when using the workbench in dark mode.
SSH Public Key Authentication in virl2-sshd The virl2-sshd service now supports SSH Public Key authentication for secure and convenient access.
Workbench Sidebar Resizability The sidebar in the workbench has been improved to allow for both vertical and horizontal resizing, aiding in configuration tasks.
Anonymized Node Configuration in Events The Diagnostics -> Events section now anonymizes node configuration content, displaying only the length and alteration status rather than the full details.
Node Configuration Exclusion Option API calls for node data can now exclude the node configurations, reducing payload sizes where configurations are unnecessary.
VLAN Zero Tag Cleaning Service A new service is available for customers affected by 802.1p tags on bridged networks, particularly on UCS bare metal deployments, to correct external connectivity issues.
Reduced Node Data Requests in System Admin The automatic periodic GET requests for node data on the System Admin page have been replaced with data processed from websocket events to reduce unnecessary traffic.
Cat9kv DD Reshipment The Cat9000v series has been updated, with the reintroduction of the Cat9000v UADP (formerly Cat9000v DD) and an update to the Cat9000v Q200 model.
Intelligent CML Disk Selection During Install The baremetal installation process for CML has been improved to automatically select the appropriate disk, with a longer wait time for user input when multiple disks are present.
Specific Low Resource Launch Fail Messages Messages indicating high system load now differentiate between CPU, memory, and disk space constraints for clearer troubleshooting.
Linux Login Message Update Compute host console login messages have been updated to display more relevant system information.
Deprecated GET Tags Endpoint The GET /labs/{lab_id}/nodes/{node_id}/tags endpoint has been deprecated in favor of more modern and efficient alternatives.
CML Version Display in Cockpit UI The Cockpit UI has been updated to display the current CML version and logo for better user awareness.
Improved Service Health Status Handling Health status reporting and diagnostics have been improved for scenarios where compute host services are experiencing issues.
Enhanced NFS Health Detection The process for detecting NFS health has been improved to perform periodic checks and handle disconnections properly.
Automatic Image Re-mounting Service A new service has been introduced to re-mount reference platform images automatically after CML services restart or a host reboot in a cluster environment.
virl2-sshd SFTP Protocol Support SFTP protocol support has been implemented in the virl2-sshd service for secure file transfer capabilities.
Expanded Diagnostics API Endpoints The diagnostics API has been expanded with multiple endpoints, such as GET /diagnostics/computes and GET /diagnostics/labs, to provide targeted diagnostic data.

Bug Fixes

Bug Description Resolution
When you upload the CML upgrade PKG file on the Tools > Upgrade System page, the upload fails with an error indicating Insufficient space available on controller and Bytes Available 0. A bug in CML 2.6.x will always cause you to get this error if you put the controller into Registered state before you upload the PKG file. Fixed in CML 2.7.0. If you experience this problem while upgrading from CML 2.6.x, see the Troubleshooting FAQ for workarounds.
Delete Key Behavior in Settings Tab Resolved an issue where pressing the Delete key within settings tab input fields or text areas no longer triggers the node deletion dialog.
Swagger Interface Schema Improvement Fixed the API schema for topology interface responses to ensure accurate representation in Swagger documentation.
Multiple Link Selection Highlight Addressed a visual bug where occasionally, after deselecting multiple links, one link would remain highlighted.
Early Console Opening in Firefox Corrected a bug that caused Firefox to slow down when consoles were opened shortly after a node entered the started state.
Devicemux Recursive Libvirt Close Fixed a devicemux bug that prevented console start and stop operations following a restart of the Libvirt service.
Removal of Vulnerable SSH Algorithm Updated SSH server configurations to remove or disable the chacha20-poly1305@openssh.com algorithm due to CVE-2023-48795, thereby mitigating the Terrapin attack.
Reference Platform ISO Instructions Update Clarified instructions in the CML Install wizard for copying the reference platform ISO, indicating that it's not critical to have the ISO during installation as images can be added afterward.
Persistence of Deleted Custom Bridges in Cockpit Ensured that when a custom bridge is deleted, it is also removed from the list of available external connectors in the CML UI Administration Tools > External Connectors section.
Non-Admin User Removal with LDAP Enabled Fixed a bug that previously blocked CML administrators from removing users while LDAP authentication was enabled.
Username Case Sensitivity in CML vs. Active Directory Unified username case sensitivity by making all usernames case-insensitive, matching Active Directory's behavior. This may affect pre-upgrade phases of existing CML deployments.
NFS3 Usage with QEMU Base Images Improved the detection of NFS-related issues in cluster deployments, particularly concerning the use of QEMU base images with NFS3.
Concurrent Map Read/Write in Dispatcher Resolved a bug in the console dispatcher service that could cause a crash during concurrent map reads and writes, affecting lab node boot progress monitoring.
Debug Log Level and Fabric Service During Restart Separated component log levels to allow administrators to set LOG_LEVEL=DEBUG without breaking the fabric during virl2.target restarts.
Loss of Annotations Fixed an issue where annotations would be lost if a user left the workbench without clicking on the canvas; annotations are now automatically saved upon exiting.
UMS Start Behavior with Unavailable Compute Hosts Modified the start behavior for UMS and other nodes when compute hosts are disabled or unavailable in both single host and cluster deployments. Lab nodes will not start until compute hosts are available and a new start request is made.
Custom Column Reset in Nodes Pane Addressed a bug that caused custom column settings to reset when the resource pool column was enabled in the Nodes pane.
Node Start Error Messaging in Clusters Improved error messaging when a node start is rejected due to controller limitations, ensuring that errors are relevant and specific to the issue at hand.
Annotation Border Style Update on WebSocket Events Fixed a bug that prevented real-time updates to the annotation border style when changes were broadcasted via WebSocket events.
Validation Against Invalid Elements on Canvas Enhanced validation to prevent errors from invalid or random strings dropped onto the canvas; such input is now silently ignored if not valid.
Smart License Registration Enablement Post-Licensing Reset Resolved a bug that hindered the re-enabling of Smart License Registration (SLR) after a licensing reset while nodes were running.
VNC CTRL + ALT + DEL Function Restoration Restored the non-functional CTRL + ALT + DEL keyboard command for VNC access.
Node.remove() Method Behavior in Client Library Corrected the Node.remove() method in the Client Library to prevent incorrect NodeNotFound exceptions.
Suppressing Warnings for Old Lab Schemas in Client Library Eliminated unnecessary warnings when creating labs with old schemas using the Client Library.
Log Download During Service Failures Fixed the log download feature to function even when CML services are in a failed state.
PyATS Integration Test Handling Addressed a compatibility bug with PyATS in the Python Client Library, ensuring proper test-bed loading and disabling unnecessary variable replacement based on CLI arguments.
Migration Script Compute ID Consideration Improved the migration script to maintain compute host identifiers during migrations between hosts of version 2.4.0 or later.
ASAv Image Definition Configuration Update Updated the ASAv image definition configuration to ensure accuracy.
Enhanced LLD NFS Handling in CML Enterprise Clusters Improved NFS health checks in CML Enterprise clusters for more robust detection and handling of issues with reference platforms.
Removal of Incomplete Image Definitions Fixed a bug that prevented the proper removal of image definitions when their associated image file was missing.
Custom Bridge Protection Default State Correction Corrected a bug where the L2 bridge protection service for external connectors was reported as enabled by default but was not actually started.
Client Library Performance Syncing Addressed a performance issue in the client library when syncing operational parameters while listing lab nodes.
Improved Add Nodes Pop-out Search Functionality Enhanced the search tool in the Add Nodes pop-out to better match on labels.
External Connector Configuration with Resource Pool Restrictions Fixed an issue that now allows enterprise users to see which external connectors are disallowed by resource pool limits set by an administrator.
Disk Resizing While VM Is Running Fixed a bug to allow the maximization of lab storage after the disk is resized while the VM is running.
Deletion Order of Links and Nodes in UI Corrected a bug to ensure links are deleted before nodes when removing multiple objects in the UI simultaneously.
Configuration Upgrade on Installation Retry Resolved an issue that impeded configuration upgrades when attempting to reinstall the cml2 package after a previous failure.
AWS Installation and Libvirt Default Network Creation Fixed a bug in AWS installations/upgrades that prevented the default network from being properly recreated.
LDAP User Assignment to Resource Pool Fixed a bug that prevented existing users from being assigned to a resource pool when LDAP was enabled.
Local Authentication Data Handling Corrected an issue where LDAP data was unnecessarily included when configuring local authentication methods in System Administration.
Cat9kv Node Definition Patch Reapplication Resolved a bug that caused repeated failed attempts to reapply a patch to the cat9kv node definition.
IP Address Retention in VMware Fusion Addressed a problem where network connections could become misconfigured after a shutdown in VMware Fusion, with the fix applied to new deployments or those rerunning initial configuration.
IOSv and IOSvL2 Node Boot with Software RAID Fixed an issue preventing the boot of IOSv and IOSvL2 nodes on deployments with a block size of 4096.
Fabric Remotemux Bind Error After Reboot Corrected a bug in cluster deployments that prevented lab traffic from being received from other cluster compute hosts, impacting external connectors and unmanaged switch nodes.
Pre-Login Message Enhancement for Installations Implemented an informative pre-login message to guide users to support if the CML installation dialog fails to start.
Unmanaged Switch State Reflection in Controller Fixed a discrepancy in the displayed state of unmanaged switches for certain situations, such as when the corresponding bridge fails to activate.
Default GRPC Service Ports Adjustment Updated internal processes to bind to registered port ranges instead of dynamic ones to avoid port conflicts.
Copy Reference Platform Script Robustness Improved the reference platform copy script to handle cases where the ISO is already mounted or specified in an unexpected location.
Reference Platform Reload in Installation Script Fixed an issue allowing for the reload of the reference platform during installation if it's attached after the CML VM boots.
Workbench External Connector Mappings Update Resolved a bug that required a page refresh to update external connector mappings in the Workbench.
Diagnostics Page Display Fix Fixed the Diagnostics page to maintain content visibility when the screen width is reduced.
Licensing Feature API Validation Enhanced the licensing feature API to disable the save button when no license is selected in the "Choose Licenses" dialog.
File Upload Availability in REGISTERED State Fixed a bug that blocked file uploads when the controller was in the REGISTERED state, such as during a system upgrade.
Console Size Settings Consistency in Firefox Corrected inconsistencies with maximal console resolution settings in Firefox.
Persistent UI Indicator on Links Fixed a bug to ensure the UI indicator for links remains persistent and does not disappear in certain scenarios.
Link Menu Node Navigation Fix Corrected a navigation issue where links to nodes in the link menu did not open the node side menu.
Stopping an Already Stopped Link Modified behavior so that attempting to stop an already stopped link is treated as a no-op instead of producing an error.
Unmanaged Switch Configuration Handling Clarified that Unmanaged Switches (UMS) no longer accept configurations since such settings were not utilized.
Error Response Format for Large Image Uploads Adjusted API error responses (429, 413, 502) to return JSON instead of HTML during large image file uploads.
Annotations Input Validation Consistency Aligned the UI input validation for annotations with the API to ensure consistency.
Interface Deletion Restriction Fixed a bug that allowed deletion of interfaces while the node was running.
Dashboard Annotations WebSocket Update Resolved an issue where changes to annotations did not update in real-time on the lab preview in the Dashboard.

Known Issues and Caveats for CML 2.7

Bug Description Notes
Bootstrap Configuration for Catalyst SD-WAN Edge Nodes Identified a bug where bootstrap configuration is not functioning correctly for Catalyst SD-WAN Edge nodes. The fix is scheduled for CML 2.7.1.
Workbench Tour Color Issue in Dark Mode Discovered a UI inconsistency where the initial Workbench tour appears white even when dark mode is enabled. This will be corrected in CML 2.7.1.
Node Definition Configuration File Name Changes Noted a bug that causes issues with existing nodes when the file name of a configuration in a node definition is altered; the nodes become unable to start. A fix is planned for CML 2.7.1.
External connectivity does not work in CML bare metal installations. This bug impacts CML bare metal deployments regardless of release.

In some deployments, most notably on UCS bare metal servers in which the vNIC's vlan is set to access mode, the ingress traffic is tagged with vlan 0. This vlan is meant to carry priority information as per the 802.1p standard.

Network services can normally ignore this tag and handle vlan 0 packets transparently as untagged (no vlan), but some node types in CML do not handle it gracefully, ignoring the incoming packets. As a result, the virtual nodes connected to an external connector node may not acquire an IP address from an external DHCP server or may not be able to communicate with external devices.

Workaround: Follow the instructions from the Handling 802.1p Frames in UCS section of the Cisco Modeling Labs Administrator's Guide.
After a restart, you cannot reach the CML UI. The console of the CML instance shows a login prompt, but it also indicates that CML does not have an IP address. After you shut down and restart a CML instance, it sometimes loses its IP address. When that happens, you will no longer be able to reach the CML UI even though the CML services are running. This bug affects all 2.7.X instances.

Workaround: When you encounter this problem, open the CML console (e.g., the CML VM's console in VMware or the CML server's remote management console). Log in as the sysadmin user and run this command:

sudo netplan apply

You can verify the current/new IP address of the CML instance using this command:

ip addr show bridge0

If the issue persists, run the following command to call the initial setup script and complete the initial setup wizard to resolve the issue:

sudo HOME=/var/local/virl2 /usr/local/bin/virl2-initial-setup.py --reconfigure
Cannot run simulations on a macOS system that uses Apple silicon (e.g., the Apple M1 chip). CML and the VM Images for CML Labs are built to run on x86_64 processors. In particular, CML is not compatible with Apple silicon (e.g., the Apple M1 chip).

Workaround: There are no plans to support ARM64 processors at this time. Deploy CML to a different system that meets the system requirements for running CML.
Even if you were able to run network simulations with CML 2.2.x on VMWare Fusion and macOS Big Sur or above, the same labs no longer start after migrating to CML 2.3.0+. A default setting in the CML VM changed between CML 2.2.3 and CML 2.3.0. Because of the new virtualization layer in macOS 11+, this setting causes errors when running the CML VM on VMware Fusion.

Workaround: follow the instructions to disable nested virtualization in the CML VM.
Bug in macOS Big Sur. After upgrading to Big Sur and VMware Fusion 12.x, simulations no longer start. MacOS Big Sur introduced changes to the way macOS handles virtualization, but the problems were resolved in later versions of macOS Big Sur and VMware Fusion.

Workaround 1: If you are still experiencing problems, be sure to upgrade to the latest version of VMware Fusion 12.x and the latest bug fix release of your current macOS version.

Workaround 2: If you are using CML 2.3+, then follow the instructions to disable nested virtualization in the CML VM.
Deleting or reinstalling a CML instance without first deregistering leaves the license marked as in use. Your new CML instance indicates that the license authorization status is out of compliance. If you have deleted or otherwise destroyed a CML instance without deregistering it, then the Cisco Smart Licensing servers will still report that instance’s licenses are in use.

Workaround 1: Before you destroy a CML instance, follow the Licensing - Deregistration instructions to deregister the instance. Properly deregistering an instance before destroying it will completely avoid this problem.

Workaround 2: If you have already destroyed the CML instance, see the instructions for freeing the license in the CML FAQ.
When the incorrect product license is selected, the SLR (Specific Licensing Reservation) can still be completed. It eventually raises an error, but the SLR reservation will be complete with no entitlements. Workaround: If you accidentally select the wrong product license, then you will need to release the SLR reservation and then correct the product license before trying to complete the SLR licensing again.
If you have read-only access to a shared lab, some read/write actions are still shown in the UI. Attempting to change any properties or to invoke these actions raises error notifications in the UI. Workaround 1: Do not attempt to invoke the read/write actions when working with a lab for which you only have read-only permissions.

Workaround 2: Ask the lab owner to share the lab with read/write permissions so that you can make changes to the lab.
If you have read-only access to a shared lab, you will still not be able to see the current link conditioning properties in the link's Simulate pane. Workaround: ask the lab owner to share the lab with read/write permissions if you need to access the link conditioning properties.
The algorithm for computing the available memory does not take into account maximal possible memory needed by all running nodes. It only checks the currently allocated memory. CML will permit you to start nodes that fit within the current available memory of the CML server. Later, if the memory footprint of the already-running nodes grows and exceeds the available memory on the CML server, those nodes will crash. Workaround 1: Launch fewer nodes to stay within your CML server's resources.

Workaround 2: Add additional memory to the CML server to permit launching additional nodes. Of course, this workaround just moves the point where the CML server will eventually hit its limit, so you must remain aware of your CML server's resource limits or ensure that the CML server has sufficient memory to accommodate all labs that will be running on your CML server simultaneously.
When starting a CSR 1000v node for the first time or after a Wipe operation, the interfaces are all in shutdown state. Workaround 1: Add no shut to each interface configuration in each CSR1000v node's Edit Config field in the UI.

Workaround 2: Use the EEM applet on the CSR 1000v page.
New nodes cannot be started when CPU usage is over 95%, when RAM is over 80%, or when disk usage is over 95%. Workaround: Allocate additional resources to the CML server.

Workaround 2: If you're hitting the disk usage limit, check the Remove Orphaned VMs function in Cockpit. Orphaned VMs can consume a significant amount of disk space even though CML is no longer using or managing those VMs.
The journalctl output shows errors like this: cockpit-tls gnutls_handshake failed: A TLS fatal alert has been received This log message only occurs with Chrome browsers and has no functional impact on the use of Cockpit. No workaround needed.
In CML Cluster, the Cockpit Storage table for NFS mounts looks broken. This is a display bug of the Cockpit UI when it encounters an NFS share using an IPv6 address. The single mount is split into two rows, only the first of which is indicated as mounted. Your NFS setup is probably fine, as long as the CML UI shows in its Cluster Status view that each compute host's Images indicator is green. Trying to edit or repair the NFS mount in any way in Cockpit is more likely to break the configuration. Please contact TAC if you experience networking issues with clusters which cannot be resolved by verifying basic connectivity on the cluster network or by restarting CML services.