Hub and Spoke Topology

Cisco Catalysts SD-WAN Fabric by default creates a full-mesh topology between all WAN-Edge routers. By using Policies on the Manager, we can establish different topologies (like Hub and Spoke) per segment and create flows as per the design requirements.

There are multiple ways that a Hub-and-Spoke topology can be created for Sites for single or multiple VPNs. Here are three different ways that we can create a Hub and Spoke topology in the network.

1. Hub-and-Spoke Topology Policy
2. Custom Control Topology Policy

Pre-Check

By default, the traceroute shows that the topology between three sites (Site1, Site2, and Site3) is full mesh, where each site is directly reachable from the other site.

• Login to Cisco Catalyst SD-WAN Manager, go to Monitor > Devices, click on Site1-cEdge01 site.
• On the left side, scroll to the bottom and click on Troubleshooting option then Trace Route.

Hub and Spoke Topology Policy

Traceroute from Site1-cEdge01 VPN1 interface to a host 10.10.22.12 (Site2-cEdge01 VPN1)

Traceroute from Site2-cEdge01 VPN1 interface to Site1-cEdge01 VPN1 interface shows direct data-tunnel.

OMP routing table on Site1-cEdge01 shows that subnet 10.10.21.0/24 is learned from TLOC 10.10.1.13.

Hub-and-Spoke Topology Policy

This policy filters advertisements of Routes and TLOCs between the Spokes. Therefore, it is required that a DC Site advertises a default route to the Spoke Sites to facilitate the routing between the Spokes.

1. Go to Cisco Catalyst SD-WAN Manager > Configuration > Policies > Centralized Policy > Customer Options Click Lists under Centralized Policy.

2. Create Spokes Site List.

• Click Site > New Site List
• Enter Site List Name and Site IDs, then click Save.

REST API

3. Similarly create Hub Site List.

-Click Site > New Site List -Enter Site List Name and Site IDs, then click Save.

REST API

4. Get Site List IDs for Spokes and Hub.

REST API

REST API

VPN List allows us to create separate topologies per VPN.

5. Create VPN List for VPN 1.

• Click VPN > New VPN List
• Enter VPN List Name and VPN number, then click Save

REST API

6. Get the VPN List ID.

REST API

7. Create Hub and Spoke Topology.

• Go to Cisco Catalyst SD-WAN Manager > Configuration > Policies > Custom Options.

• Click Topology under Centralized Policy

• Click Add Topology then click on Hub-and-Spoke.

• Fill the Name and Description fields, add VPN List, Spoke and Hub Site Lists.

• Click Preview on the left bottom side to see the CLI of the Policy before saving.

• Save the Policy by clicking Save Hub-And-Spoke Policy.

REST API

Create Hub and Spoke Topology Policy

8. Get the List of Topology Policy.

REST API

9. Let’s add this Topology Policy in the list of Centralized Policy.

• Go to Cisco Catalyst SD-WAN Manager > Configuration > Policies > Centralized Policy.

• Click Add Policy, then click Next.

• Under Configure Topology and VPN Membership, click Add Topology to add choose Import Existing Topology option.

• Select the Policy from the drop-down, then click Import.

• Click Next > Next, fill the Policy Name and Description.

• Click Preview to view the complete Policy in CLI.

As we can see from the CLI, once the Policy will be activated, it will be pushed to all Controllers to be implemented in OUT direction to the Spokes.

• Click Save Policy.

REST API

Create Centralized Policy.

10. Get the Centralized Policy details.

11. Activate the policy by clicking “…” of Hub-and-Spoke Policy and then click Activate.

12. Click Activate again.

13. Hub-and-Spoke Policy is successfully pushed to the Cisco Catalyst SD-WAN Controller.

REST API Active Policy.

Verification/Testing

• On Site1-cEdge01, OMP routing table shows that it is not learning any routes from Site2.

• Traceroute from Site1-cEdge01 VPN1 interface to a host 10.10.22.12 (Site2-cEdge01 VPN1) takes the default route learned from DC site.

• Deactivate the policy when not needed.

REST API

Deactivate Policy.

Custom Control Topology Policy

You can also use Centralized Custom Control Policy to create a Hub and Spoke topology. In this policy, we’ll change the next hop TLOC address for the OMP routes of Spokes to the Hub’s TLOC before routes are advertised to other Spokes by the Controller.

To start a new policy:

1. Go to Cisco Catalyst SD-WAN Manager > Configuration > Policies > Centralized Policy > Add Policy

We have already created Site Lists and VPN List in the previous section of Hub-and-Spoke Topology Policy. (Create Site List for Spokes and Hub if not created already).

For guidance, refer to the previous section in this document.

-Click Next to go to Configure Topology and VPN Membership, click Add Topology > Custom Control (Route & TLOC)

-Click Sequence Type and then click Route.

-Add a Name and Description, click Sequence Rule, also rename the Rule Sequence. -Match on Site and select the Site List for Spokes.

-Change Action for the rule to Accept and set the TLOC to DC-cEdge01 TLOC IP, color to MPLS and encapsulation to IPSEC. This action changes the next-hop TLOC IP of the prefixes that are advertised by the Spokes to DC-cEdge01 TLOC. (To specify more than one TLOC, that can be used as a Nex-Hop, use TLOC List. Color and Encapsulation can be set for different values, if needed).

• Click Save Match And Actions to save to rule.

• Change the Default Action to Accept, then click Save Match And Actions.

• Click Save Control Policy.

• Click Next > Next.

• In Apply Policies to Sites and VPNs section, add Policy Name, Policy Description. -Click New Site/Region List and then add Site List of Spokes in Outbound direction.

Outbound direction is from Controller perspective as Controller is the Control plane that receives the prefixes from all branches and then advertises out them to other sites. This Policy is applied in outbound direction which means that Controller will update the TLOC of Site1 prefixes before they are advertised to Site2.

• Click Add.

• Click Preview to see the policy details in CLI. The CLI of the policy is below.

• Click Save Policy

REST API

Create Custom Control Topology

Get Custom Control Topology List

Create Custom Centralized Policy

Get Custom Centralized Policy

2. Activate Policy

• Click Activate

• Click Activate once again.

3. Policy is successfully pushed to the Controller.

REST API

Activate Policy

Let’s check the routing tables on Site1 and Site2 cEdges.

4. OMP table on Site1-cEdge01 shows that in VPN 1 the Next-Hop of Site2-cEdge01 route is now DC TLOC.

5. Similarly, Site2-cEdge01 shows that in VPN 1 the Next-Hop of Site1-cEdge01 route is now DC TLOC.

Let’s test this using the traceroute as well.

6. Go to Monitor > Devices > Site1-cEdge01 > Troubleshooting > Trace Route

Traceroute confirms that traffic from Site1 takes one-hop through DC site to reach Site2.

Traffic from Site2 to a host in Site1 confirms the hub-and-spoke topology as well.

Deactivate the Policy when not required.

REST API

Deactivate Policy.

Cisco Catalyst SD-WAN Manager Support for SD-Routing

Description:

This feature allows basic management of Cisco IOS XE devices that are operating in Autonomous (non-SD-WAN) mode through Cisco Catalyst SD-WAN Manager. These devices are referred as SD-Routing devices. A single NMS (Cisco SD-WAN Manager) can be used for both Cisco Catalyst SD-WAN and SD-Routing devices.

Onboarding SD-Routing Devices

SD-Routing devices can be onboarded using following ways:

-Automated Onboarding: Uses Dynamic Host Configuration Protocol (DHCP) and Cisco Plug and Play (PNP) to automatically onboard the device to Cisco Catalyst SD-WAN Manager.

-Bootstrap Onboarding: Uses bootstrap file either on the bootflash or on a USB and configures the device with the minimum configuration to reach Cisco Catalyst SD-WAN Manager.

-Manual Onboarding: Configures device manually using IOS-XE commands to onboard the device to Cisco Catalyst SD-WAN Manager.

https://www.cisco.com/c/en/us/td/docs/routers/cloud_edge/c8500/software-configuration-guide/c8500-software-config-guide/managing-the-device-using-vmanage.html

In this guide we’ll use Manual Onboarding procedure.

Manual Onboarding Configuration

1. From Cisco Catalyst SD-WAN Manager menu, go to Workflows > Quick Connect.

2. Click Get Started.

3. Click Next.

If the provisioning file (.csv or .viptela) is not uploaded from PnP to Cisco Catalyst SD-WAN Manager, you can use either Login to your Smart Account (Sync Smart Account) option or Upload file with serial numbers (.csv or .viptela) upload option to add the device to Cisco Catalyst SD-WAN Manager. If the device is already added to Cisco Catalyst SD-WAN Manager, select Skip for now option.

Note: - .csv file is applicable only for hardware devices, .viptela file is applicable for both hardware and software devices.

4. In this Lab, devices serial file has been already uploaded to Cisco Catalyst SD-WAN Manager, therefore choose Skip for now option, and click Next.

5. Select the device that you want to onboard and click Next.

6. In the Add and Review Configuration dialog box, enter the Site-ID, System-IP and Hostname.

• Click Export to download the file, fill the details and then Import it.

7. Click Apply and then click Next.

8. Check the details, then click Next.

9. Check the Summary, then click Onboard.

REST API Onboard SD-Routing Device

10. Activate the device using Chassis and Token.

• Login to CML and go to Site11-Edge01 console.

• Run the following command on the device console. request platform soft sd-routing activate chassis token

Chassis ID and Token number cab be obtained from Cisco Catalyst SD-WAN Manager. Go to Cisco Catalyst SD-WAN Manager > Configuration > Certificates

Note: This method is supported only on Cisco SD-WAN software devices (Cisco C8000V).

11. Verify the control connection status on the Edge device using these commands:

show sd-routing connections summary

Verify connectivity to another SD-Routing device.

• Ping Site12 from Site11

-Traceroute Site12 from Site11

Route Leaking

This feature enables the capability to leak routes between VRFs.

For reference: https://www.cisco.com/c/en/us/td/docs/routers/sdwan/configuration/routing/ios-xe-17/routing-book-xe/m-routing-leaking-for-service-sharing.html

Route Leaking between Global VRF and Service VPNs

This feature enables you to leak routes bidirectionally between Global VRF and Service VPNs. It allows bypassing hubs and provides migrated branches direct access to non-migrated branches.

Configure Route Leaking in Site2, so that Site2 LAN will have reachability to Site12 (non SD-WAN site) via Underlay.

Pre-Check

• Traceroute from Site2 VPN1 to Site12

Configuration:

1. From Cisco Catalyst SD-WAN Manager menu, choose Configuration > Templates.

2. Click Feature Templates.

3. Search for “VPN_1_Template_Route-Leak” template, click on the 3 dots and then click Edit.

4. Go to Route Leak section.

5. To leak routes from global VRF, click Add New Route Leak from Global VPN to Service VPN.

• In the Route Protocol Leak from Global to Service drop-down list, choose protocol as Static.

• Click Add.

6. To leak routes from the service VPNs to global VRF, click Add New Route Leak from Service VPN to Global VPN.

• In the Route Protocol Leak from Service to Global drop-down list, select Global and then choose protocol as connected.
• Click Add.

• Click Update.

7. Click Next.

8. Select the device and then click Config Diff to check the changes that will be pushed to the device.

9. Click Configure Devices.

10. Once the configuration is pushed to the device, the status will be displayed as Success.

REST API

Get all Feature Templates List.

Configure Route Leaking in VPN_1_Template_Route-Leak Template.

Push Template changes to the device.

Verification

Verify Route Leaking using following commands:

show ip route show ip route vrf

In the output, leaked routes are represented by a + sign next to the route leaked. Example: C+ denotes that a connected route is leaked into Global VRF.

In the output, leaked routes are represented by a + sign next to the route leaked. Example: S+ denotes that a static route is leaked into Service VPN.

Testing

Ping Site12 (non sd-wan) from Site2 VRF1

Traceroute Site12 (non sd-wan) from Site2 VRF1

Route Leaking between Inter-Service VPN

With this feature you can leak routes between the Service VPNs at the same Edge device.

Configuration:

Configure Route Leaking in Site2, so that VRF2 routes will be redistributed to VRF1.

1. From the Cisco Catalyst SD-WAN Manager menu, choose Configuration > Templates.

2. Click Feature Templates.

3. Search for “VPN_1_Template_Route-Leak” template, click on the 3 dots and then click Edit.

4. Click Route Leak field.

5. Click Route Leak between Service VPN.

• Click Add New Inter Service VPN Route Leak.
• In the Source VPN drop-down list, choose the service VPN from where you want to leak the routes.
• In the Route Protocol Leak to Current VPN drop-down list, select a route protocol to enable route leaking to the current VPN.
• Click Add.

6. Click Update.

7. Click Next.

8. Select the device and then click Config Diff to check the changes that will be pushed to the device.

9. Then click Configure Devices.

10. Once the configuration is pushed to the device, the status will be displayed as Success.

REST API

Configure Route Leak in VPN_1_Template_Route-Leak Template.

Push Template changes to the device.

Verification

Verify Route Leaking using following commands:

show ip route vrf

The below output shows that VRF2 route (100.0.0.0/32) is being redistributed to VRF1.

UX 2.0 Config Group Deployment

Note: Keep the running configuration of the target device handy/copied to a notepad before deploying the Configuration Group. Ensure the device is detached from device templates. (The target device in this example will be Site1-cEdge01).

1. Login to Cisco Catalyst SD-WAN Manager at select the main dashboard, Navigate to Configuration > Configuration Groups. Select Add Configuration Group.

2. This opens Workflow Library, from this menu choose Create Configuration Group

A “Welcome to Configuration Group Creation “ Banner appears, select Let’s Do it.

3. Name and Describe the Configuration Group. For the sake of this Lab Guide let’s name it as Site1_EC.

4. Following menu appears as shown below.

Select the Site Type as Single Router. Hop on to WAN Interfaces, as You select three transport types namely MPLS, Internet and LTE will appear, add the details of Interface name by clicking Show Advanced check box, delete the LTE transport. Select Static IP and provide the IP details of these transports.

Add WAN Routing, Select Static IPv4 from the drop-down menu and since we have two TLOC’s we will choose two Static IPv4 entries as shown below.

5. Select LAN & Service VPN Profile, add the Service VPN number followed by the number of interfaces under this VPN. Remove/Delete the additional Service VPN’s that is not required or present in your deployment.

6. Enable Remote Access if required else it is disabled by default, Click Next.

7. Review the Summary of the Configuration Group, edit anything if it seems incorrect.

8. Once conformed click on Create Configuration Group, user will success messages for all individual items. Once Configuration Group is created select on Associate Devices.

The above configuration group created is equivalent to the below API call: POST https://10.10.20.90/dataservice/v1/config-group

Add devices to the Configuration Group, select Next to below step

9. Select the target device from the list, Select the Site_1001 under Global, the target device Chassi Number can be seen including the hostname, System IP, Site ID, Serial number, select the device as seen below.

If You see no device in this list, it means there is none which have been detached using templates, for such the GET API looks as below.

Review the Summary for the device to be added. You can deploy the device now or later.

Click on Save. Following message appears.

10. Select Provision Devices and select Next. Select Sites to Deploy. Click Next.

11. Add and Review Device Configuration. Cisco Catalyst SD-WAN Manager autogenerated minimal configurations to make it easier.

12. Review the Summary of the device followed by Deploy

• Select Deploy.

• Deployment request for the device gets submitted, select View Deployment Status

• Success Status message is received as seen below.

• The above action against attaching the device shows up as below API

• Essentially view the existing Config Groups using the below API

GET https://10.10.20.90/dataservice/v1/config-group

Click on Configuration > Configuration Group the newly created CG for Site1 cEdge can be seen as below.

Creating Tags using UX2.0

Navigate to Configuration > Devices

Let’s Go ahead and assign Tags to devices at Site3, select Site3-cEdge01

POST https://10.10.20.90/dataservice/v1/tags/associate

On similar lines I have now created a tag named Europe1 as seen below

Let’s verify if the same can be seen on Cisco Catalyst SD-WAN Manager.

Global Site Topology using UX2.0

As seen we have onboarded Site1-cEdge01 to Config Groups, we can now view the Global Topology view for this device.

SITE_1001 Site1-cEdge01

Select the last icon in the Overview tab.

Select the Network Hierarchy in the left panel and select Site_1001

The device health can be seen on this dashlet for Site1-cEdge01, select the Global Topology icon as seen below.

Admin can see the Global Network Topology for this device, the WAN TLOC’s and Service VPN’s onboarded as seen below.

The above is basically a GET API at the backend, below is the API and its payload.

GET: https://10.10.20.90/dataservice/topology/monitor/site/1001

RBAC Using Resource Group

This use case will demonstrate how a user within a resource group can have restricted access to default permissions available in Cisco Catalyst SD-WAN Manager.

1. Go to the main dashboard, Navigate to Administration > Manage Users.

2. Select User Groups and create the custom role, in this case site1_service.

3. Select the features of interest that the created user wants to have read write access against, for testing purposes I have enabled the below features to this group named as site1_service. Other than monitoring, license management, the user should be limited to just service side of the config group.

Below is the API that is used to fetch the User groups on Cisco Catalyst SD-WAN Manager.

GET https://10.10.20.90/dataservice/admin/usergroup

Focus on the body of this GET call

The output has been truncated, the newly created user group site1_service can be seen with the defined feature read write access.

Define Resource Group, for this reference this site is name as America, specific to Site2 only.

Admin needs to manually change the resource group for the device templates and feature templates to America.

Next step is defining the user, Navigate to Users on the top and create a user. For this example, user added is Wen. For 17.12 we cannot use custom role and are limited to just the pre-defined default roles.

API Call to create a user:

POST: https://10.10.20.90/dataservice/admin/user Admin can now log out and login in via username Wen to view the level of access.

The new user Wen can see only the templates attached to America and Global. Also, this user can Edit the template assigned to Site2-cEdge01, while can only have View access to global templates.

License Management

Navigate to Administration > License Management

For this reference example, the mode used is offline. Navigate to Administration Settings and select License Reporting as Offline, below the PUT API call to achieve this function.

In either of the modes the admin needs to sync their SA/VA. Next step would be go ahead and sync the license file, below is the API call for the same task.

POST: https://10.10.20.90/dataservice/smartLicensing/syncLicenses

Admin needs to download their License Summary file from their SA/VA, this is a tar.gz file. While making this POST call, it requests for the above-mentioned summary file that needs to be uploaded to form-data under Body.

Select Devices, choose the desired one, and select Assign License/Subscription.

While assigning the license to Site3-cEdge01 below POST API call gets pushed, in the below body it shows the chassis ID for the device Site3-cEdge01, type of license which is prepaid, template name temp1, SA/VA details, since it is a prepaid license the MSLA flag is false, reg id of the license that gets assigned including the name.