An NSO Nano service as a CI/CD pipeline

Nano services are a great feature to help you build a solid CI/CD pipeline with
no or low code on top of Cisco NSO orchestration platform. Defined in YANG
format, Nano services can be easily extended with small Python modules for
specific tasks. This example service, named nano, demonstrates a pipeline with
manual work approval, pre and post-testing, SLA tracking, Webex notifications,
XML and Python template-based device configurations.

For a description of a use case and a typical workflow, please check
USECASE.md.

Provided code and configurations are intended for educational purposes only.

How to run this example

This repo represents a folder where you should instantiate a local NSO CDB with
the ncs-setup command. Combining with NSO Local Install, it will create an
environment to run and develop the example.

You need to complete a few steps:

  1. Install NSO in Local Install mode
  2. Clone the repo
  3. Create a Python virtual environment and install all dependencies
  4. Configure environment variables
  5. Compile packages and start NSO

The service is tested against NSO 5.5.2.6 for macOS (Intel), NEDs:
cisco-ios-cli-3.8, Python 3.9.5.

Install NSO

You need to have an NSO installed in Local Install mode to run this example with
mostly no modifications. For detailed instructions on how to get a copy of NSO
and do a local install, please check
DevNet documentation
and
NSO Installation Guide.

After a successful installation, please make sure you source the ncsrc file
from a local install folder, for example:

$ source /Users/username/nso/5.5.2.6/ncsrc

$ ncs --version 5.5.2.6

$ echo $PYTHONPATH /Users/username/nso/5.5.2.6/src/ncs/pyapi

$ echo $NCS_DIR /Users/username/nso/5.5.2.6

/Users/username/nso/5.5.2.6/ is a NSO local install folder.

Clone the repo

git clone https://githib.com/andreygrechin/nano
cd nano

Create a Python virtual environment and install all dependencies

Check a version of Python; we need at least 3.9.

$ python3 --version
Python 3.9.5

To install dependencies, run:

$ python3 -m venv .venv

$ source .venv/bin/activate

$ which python3 /Users/username/repos/nso/nano/.venv/bin/python3

$ pip3 install -r requirements.txt

For linting and other development activities, you may add additional
requirements from requirements-dev.txt.

To check if NSO Python API is available, try this:

python3 -c "import ncs"

Configure environment variables

The example uses Webex API to send notifications to an approver and report
Service Progress Monitoring (aka SLA of service instances) status. You can find
instructions on creating a Webex bot and getting all the required credentials
here.

You need to add obtained credentials via environment variables. Alternatively,
you may specify them as a part of a configuration of the service.

export WEBEX_BOT_TOKEN="YOUR-WEBEX-BOT-TOKEN-HERE"
export WEBEX_ROOM_ID="YOUR-WEBEX-ROOM-ID-HERE"

To check credentials, run:

curl --location --request POST 'https://api.ciscospark.com/v1/messages' 
--header 'Content-Type: application/json'
--header "Authorization: Bearer ${WEBEX_BOT_TOKEN}"
--data-raw '{ "roomId" : "'"${WEBEX_ROOM_ID}"'", "text" : "my test msg" }'

Compile packages and start NSO

make or make all will clean up underlying folders, compile YANG models, set
up required simulated devices, create an empty local NSO CDB, create local NSO
files in the repo folder, load configuration files, and start NSO.

To run CLI or WebUI, use make cli or make webui. A default password for the
user admin is admin.

After entering CLI, check if all packages are successfully loaded.

admin@ncs# show packages package oper-status
                                                                                                      PACKAGE
                        PROGRAM                                                                       META     FILE
                        CODE     JAVA           PYTHON         BAD NCS  PACKAGE  PACKAGE  CIRCULAR    DATA     LOAD   ERROR
NAME                UP  ERROR    UNINITIALIZED  UNINITIALIZED  VERSION  NAME     VERSION  DEPENDENCY  ERROR    ERROR  INFO

cisco-ios-cli-3.8 X - - - - - - - - - - example-nano-0.1.0 X - - - - - - - - - -

admin@ncs#

For a description of a use case and a typical workflow, please check
USECASE.md.

Clean up

After finishing your session, you may stop the NSO process and clean up folders
with make clean or just make stop to stop NSO.

Testing the NSO service

To optionally run lux tests, you need to install it with all dependencies.
Check the documentation
for details.

An example of a lux script is included in the
repo. It's pretty simple, but it does the job. Check it
here. You may run it from scratch with the
make all nsotest command.

Getting help

If you have questions, concerns, bug reports, etc., please create an issue
against this repository.

Getting involved

For contribution guidelines, please check
CONTRIBUTING.md.

References

  1. NetDevOps intro from Julio Gomez,
    based on NSO, Ansible and GitLab CI/CD pipelines.
  2. Nano Services – Another approach for Reactive Fastmap (RFM) services, demo.
  3. Building a Service, from Template to Reactive Fast Map to Nano services.

Use Case

An NSO Nano service as a CI/CD pipeline use case

Please check README.md for how to run a supportive example.

NSO has many features to do configuration and service management in a very reliable manner, including model-based network-wide transactions, rollbacks, input data validation, etc. Service developers can extend this functionality even further with the traditional CI/CD approach to add pipelines with testing and deployment steps. A great example is Julio Gomez's work, which is based on NSO, Ansible, and GitLab CI/CD pipelines.

However, NSO has a great embedded toolset to implement full CI/CD pipelines with the help of Nano services.

What are Nano services

Nano services are an evolution of Reactive FASTMAP and other design patterns like subscribers and kickers. Nano services will allow you to describe a high-level logic of the service and your pipeline with no or low code. You may start with just YANG models and XML config templates. Oh, some XPath is still there, but not too much. The main benefit of a Nano service is that you get a bullet-proof framework to divide your complicated service into small states with clear conditions to in and out between them. This design pattern allows you to focus on your service logic instead of controlling the consistency of configurations and rollbacks.

You may use Nano services in so many different ways. It could be a network or device service provisioning divided into small parts, having separate diffs, and executing in your defined order. For example, configure some QoS on one device before enabling VoIP functions on another.

Moreover, Nano services shine when integration with other systems or NSO services is desirable, especially when different parts of the service will be ready at different times in an async manner. For example, request a VM creation and stop executing a pipeline; continue when VM is ready.

Sample workflow

This repo contains an example of an NSO service, named nano, to demonstrate some possible features:

  • define states of the service in YANG format, including conditions, and callbacks to start provision of parts of the service or call NSO actions
  • use template-based device configuration with YANG and XML only, with adding Python for some parts if needed
  • continue execution of next states automatically if pre-conditions are meet sometime; for example, post-tests finished with positive results
  • pre and post-tests (NSO runs a command, pyATS parses the output)
  • control of approvals of service with NETCONF Access Control Model (NACM)
  • track and log execution progress of the states, including SLA tracking of created instances via Service Progress Monitoring policy of NSO
  • modify, delete, rollback to a particular state, and re-apply the service
  • Notify users via Webex bot

Network sophisticated configurations or core features of NSO aren't shown here since they deserve separate showcases. I tried to keep this example runnable by any user without excessive requirements to have a full-blown network sandbox and access to the actual version of production NEDs. A demo version of NSO, a Linux or macOS host, and Python is what you need.

Some ideas are worth considering to extend functionality further.

  • full lifecycle management of monitoring objects in external monitoring systems based on configured network services
  • management of resource allocations via external SoT systems, like IPAM and DCNM, or an internal native NSO Resource Manager package
  • closed-loop automation with the help of middleware software to translate webhooks from the monitoring system to NSO NB interfaces to change parameters of the service
  • extending testing functionality with frameworks like pyATS, scrapli, pytest
  • separating a Webex notification action or testing actions to separate packages for easy consumption in other services

A word of caution. NSO is capable of doing a lot of things. Even so, there is always a balance of doing it with internal tools or using an external feature-rich and purpose-built toolset. In many cases splitting functionality between NSO and other systems or frameworks may benefit your service in better stability, functionality, maintainability, and faster development.

Please note, that CI/CD for developing of NSO services itself is not covered in this use case.

Usage

Init the service

A Service Progress Monitoring policy should be created to be used by nano service instances. In real-life deployment, it's a one-time command, so it is implemented as a manual process.

To init a Service Progress Monitoring policy, run make cli. A default password for the user admin is admin.

User admin last logged in 2021-09-18T15:29:01.25052+00:00, to localhost, from 127.0.0.1 using cli-console
admin connected from 127.0.0.1 using console on localhost
admin@ncs# nano sla init-sla-policy
SPM policy will be created or updated with SLA timeouts, are you sure? [yes,no] yes
result true
msg OK
admin@ncs#
System message at 2021-09-18 18:30:48...
Commit performed by admin via tcp using nano_helper.spm.
admin@ncs# show running-config service-progress-monitoring
service-progress-monitoring policy nano-sla-policy
 violation-timeout 3600
 jeopardy-timeout  1800
 condition self-ready
  component-type self
   what       all
   plan-state ready
   status     reached
  !
 !
 action action-path /nano/sla/timeouts/timeout
 action always-call true
!
admin@ncs#

Optionally you may change SLA timeouts of this policy.

admin@ncs# c
admin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# nano sla timeouts ?
Possible completions:
  jeopardy    Jeopardy timeout, minutes
  timeout     Manually send SLA notifications
  violation   Violation timeout, minutes
admin@ncs(config)# nano sla timeouts jeopardy 1 violation 2
admin@ncs(config)# commit dry-run
cli {
    local-node {
        data  nano {
                  sla {
                      timeouts {
             +            jeopardy 1;
             +            violation 2;
                      }
                  }
              }
    }
}
admin@ncs(config)# commit
Commit complete.
admin@ncs(config)#
System message at 2021-09-18 18:33:54...
Commit performed by system via tcp using nano_helper.spm.
admin@ncs(config)# do show running-config service-progress-monitoring
service-progress-monitoring policy nano-sla-policy
 violation-timeout 120
 jeopardy-timeout  60
 condition self-ready
  component-type self
   what       all
   plan-state ready
   status     reached
  !
 !
 action action-path /nano/sla/timeouts/timeout
 action always-call true
!
admin@ncs(config)# exit
admin@ncs#

Configure the service

Network Configuration Access Control Model or NACM rules don't enforce a local connection; you need to log in through SSH. We will use a user engineer who has full access to the system and the service except for the approved flag. For this user, use password engineer.

$ make cli-en
ssh engineer@0.0.0.0 -p 2024 -o PreferredAuthentications=password -o PasswordAuthentication=yes -o PubkeyAuthentication=no
engineer@0.0.0.0's password:

engineer connected from 127.0.0.1 using ssh on localhost
engineer@ncs# c
engineer@ncs# config
Entering configuration mode terminal
engineer@ncs(config)# nano nano 123\
>
engineer@ncs# c
engineer@ncs# config
Entering configuration mode terminal
engineer@ncs(config)# nano nano 123
Value for 'device' [r0,r1]: r0
engineer@ncs(config-nano-123)# ?
Possible completions:
  check-sync           Check if device config is according to the service
  commit-queue
  deep-check-sync      Check if device config is according to the service
  device               Attached device
  get-modifications    Get the data this service created
  log
  name-server          Name server
  notify               Service notifications
  plan
  re-deploy            Run/Dry-run the service logic again
  reactive-re-deploy   Reactive re-deploy of service logic
  tests                Run tests
  touch                Mark the service as changed
  un-deploy            Undo the effects of the service
  ---
  commit               Commit current set of changes
  describe             Display transparent command information
  exit                 Exit from current mode
  help                 Provide help information
  no                   Negate a command or set its defaults
  pwd                  Display current mode path
  rload                Load configuration from an ASCII file or from terminal, relative to current location
  top                  Exit to top level and optionally run command
engineer@ncs(config-nano-123)# commit dry-run
cli {
    local-node {
        data  service-progress-monitoring {
             +    trigger nano-spm-123 {
             +        policy nano-sla-policy;
             +        component self;
             +        target /nano/nano[id='123'];
             +    }
              }
              nano {
             +    nano 123 {
             +        device r0;
             +    }
              }
    }
}
engineer@ncs(config-nano-123)# commit
Commit complete.
engineer@ncs(config-nano-123)#
System message at 2021-09-19 12:21:06...
Commit performed by engineer via system using cli.
engineer@ncs(config-nano-123)# end
engineer@ncs# show running-config nano
nano nano 123
 device   r0
 no approved
!
nano sla webex room-id %ENV{WEBEX_ROOM_ID}
nano sla webex bot-token %ENV{WEBEX_BOT_TOKEN}
engineer@ncs# show nano
nano nano 123
 plan-location /nano/nano[id='123']
                      BACK                                                                 POST ACTION
TYPE          NAME    TRACK  GOAL  STATE            STATUS       WHEN                 ref  STATUS
-----------------------------------------------------------------------------------------------------------
self          self    false  -     init             reached      2021-09-19T09:21:03  -    -
                                   ready            not-reached  -                    -    -
sla-com-type  sla     false  -     init             reached      2021-09-19T09:21:03  -    -
                                   trigger-created  reached      2021-09-19T09:21:03  -    -
                                   ready            reached      2021-09-19T09:21:03  -    -
cfg-com-type  config  false  -     init             reached      2021-09-19T09:21:03  -    -
                                   approver-notif   reached      2021-09-19T09:21:03  -    create-reached
                                   approved         not-reached  -                    -    -
                                   pre-test         not-reached  -                    -    -
                                   banner-cfg       not-reached  -                    -    -
                                   name-server-cfg  not-reached  -                    -    -
                                   post-test        not-reached  -                    -    -
                                   ready            not-reached  -                    -    -

                                                                         JEOPARDY                       VIOLATION           SUCCESS
NAME          POLICY           START TIME           JEOPARDY TIME        RESULT    VIOLATION TIME       RESULT     STATUS   TIME
-------------------------------------------------------------------------------------------------------------------------------------
nano-spm-123  nano-sla-policy  2021-09-19T12:21:03  2021-09-19T12:51:03  -         2021-09-19T13:21:03  -          running  -

engineer@ncs#

The Nano service plan in WebUI:

The Nano service plan

Note that the user cannot add the approved flag to the service configuration because of the statically defined NACM rules in the configs/nacm.xml file.

The sla component is responsible for creating an SPM trigger to track the SLA of our service instance.

Also, note that config component is stopped on approver-notif state. That means, we need an another user to login and change configuration of the service by adding approved flag.

Approve the service

approver user got a notification.

A notification for the approver

Log in to NSO with the user approver and password approved. Pay attention, that approver cannot change service's configuration; commands device and name-server are not visible or accessible.

$ make cli-app
ssh approver@0.0.0.0 -p 2024 -o PreferredAuthentications=password -o PasswordAuthentication=yes -o PubkeyAuthentication=no
approver@0.0.0.0's password:

approver connected from 127.0.0.1 using ssh on localhost
approver@ncs# c
approver@ncs# config
Entering configuration mode terminal
approver@ncs(config)# nano nano 123
approver@ncs(config-nano-123)# dev?
                               ^
% Invalid input detected at '^' marker.
approver@ncs(config-nano-123)# ?
Possible completions:
  approved             Approved flag
  check-sync           Check if device config is according to the service
  commit-queue
  deep-check-sync      Check if device config is according to the service
  get-modifications    Get the data this service created
  log
  notify               Service notifications
  plan
  re-deploy            Run/Dry-run the service logic again
  reactive-re-deploy   Reactive re-deploy of service logic
  tests                Run tests
  touch                Mark the service as changed
  un-deploy            Undo the effects of the service
  ---
  commit               Commit current set of changes
  describe             Display transparent command information
  exit                 Exit from current mode
  help                 Provide help information
  no                   Negate a command or set its defaults
  pwd                  Display current mode path
  rload                Load configuration from an ASCII file or from terminal, relative to current location
  top                  Exit to top level and optionally run command
approver@ncs(config-nano-123)# approved
approver@ncs(config-nano-123)# commit dry-run
cli {
    local-node {
        data  nano {
                  nano 123 {
             +        approved true;
                  }
              }
    }
}
approver@ncs(config-nano-123)# commit
Commit complete.
approver@ncs(config-nano-123)#
System message at 2021-09-19 12:26:51...
Commit performed by approver via ssh using cli.
approver@ncs(config-nano-123)#
System message at 2021-09-19 12:26:52...
Commit performed by approver via system using cli.
approver@ncs(config-nano-123)#
System message at 2021-09-19 12:26:52...
Commit performed by approver via system using cli.
approver@ncs(config-nano-123)#
System message at 2021-09-19 12:26:53...
Commit performed by approver via system using cli.
approver@ncs(config-nano-123)#
System message at 2021-09-19 12:26:53...
Commit performed by approver via system using cli.
approver@ncs(config-nano-123)#
System message at 2021-09-19 12:26:53...
Commit performed by approver via system using cli.
approver@ncs(config-nano-123)# end
approver@ncs#
approver@ncs# show running-config nano
nano nano 123
 device   r0
 approved
!
nano sla webex room-id %ENV{WEBEX_ROOM_ID}
nano sla webex bot-token %ENV{WEBEX_BOT_TOKEN}
approver@ncs(config)# exit
approver@ncs#

Once the approved flag is committed, a series of commits will finish all subsequent steps, including pre-test, change a banner and a name server, post-test. As a part of SPM policy, a notification will be send.

Success

In case of missed SLA, separate notifications will be sent by SPM policy trigger.

SLA violation notifications

The plan

Check the executed plan of our service instance and generated and delivered device configuration in NSO native format.

approver@ncs# show nano
nano nano 123
 modified devices [ r0 ]
 directly-modified devices [ r0 ]
 device-list   [ r0 ]
 plan-location /nano/nano[id='123']
                      BACK                                                             POST ACTION
TYPE          NAME    TRACK  GOAL  STATE            STATUS   WHEN                 ref  STATUS
-------------------------------------------------------------------------------------------------------
self          self    false  -     init             reached  2021-09-19T09:21:03  -    -
                                   ready            reached  2021-09-19T09:26:53  -    -
sla-com-type  sla     false  -     init             reached  2021-09-19T09:21:03  -    -
                                   trigger-created  reached  2021-09-19T09:21:03  -    -
                                   ready            reached  2021-09-19T09:21:03  -    -
cfg-com-type  config  false  -     init             reached  2021-09-19T09:21:03  -    -
                                   approver-notif   reached  2021-09-19T09:21:03  -    create-reached
                                   approved         reached  2021-09-19T09:26:51  -    -
                                   pre-test         reached  2021-09-19T09:26:51  -    create-reached
                                   banner-cfg       reached  2021-09-19T09:26:51  -    -
                                   name-server-cfg  reached  2021-09-19T09:26:52  -    -
                                   post-test        reached  2021-09-19T09:26:53  -    create-reached
                                   ready            reached  2021-09-19T09:26:53  -    -

                                                                         JEOPARDY                       VIOLATION
NAME          POLICY           START TIME           JEOPARDY TIME        RESULT    VIOLATION TIME       RESULT     STATUS      SUCCESS TIME
----------------------------------------------------------------------------------------------------------------------------------------------------
nano-spm-123  nano-sla-policy  2021-09-19T12:21:03  2021-09-19T12:51:03  passed    2021-09-19T13:21:03  passed     successful  2021-09-19T12:26:53

approver@ncs(config)# nano nano 123 get-modifications
cli {
    local-node {
        data  devices {
                   device r0 {
                       config {
                           ip {
                               domain {
              +                    list example.com {
              +                    }
                                   lookup-conf {
              +                        lookup true;
                                   }
              +                    name example.com;
                               }
                               name-server {
              +                    name-server-list 208.67.222.222 {
              +                    }
                               }
                           }
                           banner {
              +                motd "The message of the day";
                           }
                       }
                   }
               }
               service-progress-monitoring {
              +    trigger nano-spm-123 {
              +        policy nano-sla-policy;
              +        component self;
              +        target /nano/nano[id='123'];
              +    }
               }

    }
}
approver@ncs(config)# exit
approver@ncs#

Delete the service

To entirely delete the service instance, you need to remove the SLA policy firstly.

admin@ncs(config)# end
admin@ncs# c
admin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# nano nano 123
admin@ncs(config-nano-123)# plan ?
Possible completions:
  component   A component has a type and a list of states.
admin@ncs(config-nano-123)# plan component ?
Possible completions:
  cfg-com-type  self  sla-com-type
admin@ncs(config-nano-123)# plan component sla-com-type sla ?
Possible completions:
  force-back-track   Force a component to back-track
  state              A plan state represents a certain step or stage that a service needs to execute and/or reach.
admin@ncs(config-nano-123)# plan component sla-com-type sla force-back-track
result true
admin@ncs(config-nano-123)# no nano nano 123
admin@ncs(config)# commit dry-run
cli {
    local-node {
        data  devices {
                  device r0 {
                      config {
                          ip {
                              domain {
             -                    list example.com {
             -                    }
                                  lookup-conf {
             -                        lookup true;
                                  }
             -                    name example.com;
                              }
                              name-server {
             -                    name-server-list 208.67.222.222 {
             -                    }
                              }
                          }
                          banner {
             -                motd "The message of the day";
                          }
                      }
                  }
              }
              nano {
             -    nano 123 {
             -        device r0;
             -        approved true;
             -    }
              }
    }
}
admin@ncs(config)# commit
Commit complete.
admin@ncs(config)#
System message at 2021-09-19 12:41:20...
Commit performed by admin via system using cli.
admin@ncs(config)#
System message at 2021-09-19 12:41:20...
Commit performed by admin via system using cli.
admin@ncs(config)#
System message at 2021-09-19 12:41:20...
Commit performed by admin via system using cli.
admin@ncs(config)#
System message at 2021-09-19 12:41:20...
Commit performed by admin via system using cli.
admin@ncs(config)#

You may experiment a bit, revoke your approval, and see how NSO will back-track all the changes that cannot exist without the approve flag.

NSO learning labs are a great way to explore capabilities and NSO and start learning:

Another option is to install NSO in Local Install mode and learn in your own pace. For references check README.md.

An NSO Nano service as a CI/CD pipeline use case

Please check README.md for how to run a supportive example.

NSO has many features to do configuration and service management in a very reliable manner, including model-based network-wide transactions, rollbacks, input data validation, etc. Service developers can extend this functionality even further with the traditional CI/CD approach to add pipelines with testing and deployment steps. A great example is Julio Gomez's work, which is based on NSO, Ansible, and GitLab CI/CD pipelines.

However, NSO has a great embedded toolset to implement full CI/CD pipelines with the help of Nano services.

What are Nano services

Nano services are an evolution of Reactive FASTMAP and other design patterns like subscribers and kickers. Nano services will allow you to describe a high-level logic of the service and your pipeline with no or low code. You may start with just YANG models and XML config templates. Oh, some XPath is still there, but not too much. The main benefit of a Nano service is that you get a bullet-proof framework to divide your complicated service into small states with clear conditions to in and out between them. This design pattern allows you to focus on your service logic instead of controlling the consistency of configurations and rollbacks.

You may use Nano services in so many different ways. It could be a network or device service provisioning divided into small parts, having separate diffs, and executing in your defined order. For example, configure some QoS on one device before enabling VoIP functions on another.

Moreover, Nano services shine when integration with other systems or NSO services is desirable, especially when different parts of the service will be ready at different times in an async manner. For example, request a VM creation and stop executing a pipeline; continue when VM is ready.

Sample workflow

This repo contains an example of an NSO service, named nano, to demonstrate some possible features:

  • define states of the service in YANG format, including conditions, and callbacks to start provision of parts of the service or call NSO actions
  • use template-based device configuration with YANG and XML only, with adding Python for some parts if needed
  • continue execution of next states automatically if pre-conditions are meet sometime; for example, post-tests finished with positive results
  • pre and post-tests (NSO runs a command, pyATS parses the output)
  • control of approvals of service with NETCONF Access Control Model (NACM)
  • track and log execution progress of the states, including SLA tracking of created instances via Service Progress Monitoring policy of NSO
  • modify, delete, rollback to a particular state, and re-apply the service
  • Notify users via Webex bot

Network sophisticated configurations or core features of NSO aren't shown here since they deserve separate showcases. I tried to keep this example runnable by any user without excessive requirements to have a full-blown network sandbox and access to the actual version of production NEDs. A demo version of NSO, a Linux or macOS host, and Python is what you need.

Some ideas are worth considering to extend functionality further.

  • full lifecycle management of monitoring objects in external monitoring systems based on configured network services
  • management of resource allocations via external SoT systems, like IPAM and DCNM, or an internal native NSO Resource Manager package
  • closed-loop automation with the help of middleware software to translate webhooks from the monitoring system to NSO NB interfaces to change parameters of the service
  • extending testing functionality with frameworks like pyATS, scrapli, pytest
  • separating a Webex notification action or testing actions to separate packages for easy consumption in other services

A word of caution. NSO is capable of doing a lot of things. Even so, there is always a balance of doing it with internal tools or using an external feature-rich and purpose-built toolset. In many cases splitting functionality between NSO and other systems or frameworks may benefit your service in better stability, functionality, maintainability, and faster development.

Please note, that CI/CD for developing of NSO services itself is not covered in this use case.

Usage

Init the service

A Service Progress Monitoring policy should be created to be used by nano service instances. In real-life deployment, it's a one-time command, so it is implemented as a manual process.

To init a Service Progress Monitoring policy, run make cli. A default password for the user admin is admin.

User admin last logged in 2021-09-18T15:29:01.25052+00:00, to localhost, from 127.0.0.1 using cli-console
admin connected from 127.0.0.1 using console on localhost
admin@ncs# nano sla init-sla-policy
SPM policy will be created or updated with SLA timeouts, are you sure? [yes,no] yes
result true
msg OK
admin@ncs#
System message at 2021-09-18 18:30:48...
Commit performed by admin via tcp using nano_helper.spm.
admin@ncs# show running-config service-progress-monitoring
service-progress-monitoring policy nano-sla-policy
 violation-timeout 3600
 jeopardy-timeout  1800
 condition self-ready
  component-type self
   what       all
   plan-state ready
   status     reached
  !
 !
 action action-path /nano/sla/timeouts/timeout
 action always-call true
!
admin@ncs#

Optionally you may change SLA timeouts of this policy.

admin@ncs# c
admin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# nano sla timeouts ?
Possible completions:
  jeopardy    Jeopardy timeout, minutes
  timeout     Manually send SLA notifications
  violation   Violation timeout, minutes
admin@ncs(config)# nano sla timeouts jeopardy 1 violation 2
admin@ncs(config)# commit dry-run
cli {
    local-node {
        data  nano {
                  sla {
                      timeouts {
             +            jeopardy 1;
             +            violation 2;
                      }
                  }
              }
    }
}
admin@ncs(config)# commit
Commit complete.
admin@ncs(config)#
System message at 2021-09-18 18:33:54...
Commit performed by system via tcp using nano_helper.spm.
admin@ncs(config)# do show running-config service-progress-monitoring
service-progress-monitoring policy nano-sla-policy
 violation-timeout 120
 jeopardy-timeout  60
 condition self-ready
  component-type self
   what       all
   plan-state ready
   status     reached
  !
 !
 action action-path /nano/sla/timeouts/timeout
 action always-call true
!
admin@ncs(config)# exit
admin@ncs#

Configure the service

Network Configuration Access Control Model or NACM rules don't enforce a local connection; you need to log in through SSH. We will use a user engineer who has full access to the system and the service except for the approved flag. For this user, use password engineer.

$ make cli-en
ssh engineer@0.0.0.0 -p 2024 -o PreferredAuthentications=password -o PasswordAuthentication=yes -o PubkeyAuthentication=no
engineer@0.0.0.0's password:

engineer connected from 127.0.0.1 using ssh on localhost
engineer@ncs# c
engineer@ncs# config
Entering configuration mode terminal
engineer@ncs(config)# nano nano 123\
>
engineer@ncs# c
engineer@ncs# config
Entering configuration mode terminal
engineer@ncs(config)# nano nano 123
Value for 'device' [r0,r1]: r0
engineer@ncs(config-nano-123)# ?
Possible completions:
  check-sync           Check if device config is according to the service
  commit-queue
  deep-check-sync      Check if device config is according to the service
  device               Attached device
  get-modifications    Get the data this service created
  log
  name-server          Name server
  notify               Service notifications
  plan
  re-deploy            Run/Dry-run the service logic again
  reactive-re-deploy   Reactive re-deploy of service logic
  tests                Run tests
  touch                Mark the service as changed
  un-deploy            Undo the effects of the service
  ---
  commit               Commit current set of changes
  describe             Display transparent command information
  exit                 Exit from current mode
  help                 Provide help information
  no                   Negate a command or set its defaults
  pwd                  Display current mode path
  rload                Load configuration from an ASCII file or from terminal, relative to current location
  top                  Exit to top level and optionally run command
engineer@ncs(config-nano-123)# commit dry-run
cli {
    local-node {
        data  service-progress-monitoring {
             +    trigger nano-spm-123 {
             +        policy nano-sla-policy;
             +        component self;
             +        target /nano/nano[id='123'];
             +    }
              }
              nano {
             +    nano 123 {
             +        device r0;
             +    }
              }
    }
}
engineer@ncs(config-nano-123)# commit
Commit complete.
engineer@ncs(config-nano-123)#
System message at 2021-09-19 12:21:06...
Commit performed by engineer via system using cli.
engineer@ncs(config-nano-123)# end
engineer@ncs# show running-config nano
nano nano 123
 device   r0
 no approved
!
nano sla webex room-id %ENV{WEBEX_ROOM_ID}
nano sla webex bot-token %ENV{WEBEX_BOT_TOKEN}
engineer@ncs# show nano
nano nano 123
 plan-location /nano/nano[id='123']
                      BACK                                                                 POST ACTION
TYPE          NAME    TRACK  GOAL  STATE            STATUS       WHEN                 ref  STATUS
-----------------------------------------------------------------------------------------------------------
self          self    false  -     init             reached      2021-09-19T09:21:03  -    -
                                   ready            not-reached  -                    -    -
sla-com-type  sla     false  -     init             reached      2021-09-19T09:21:03  -    -
                                   trigger-created  reached      2021-09-19T09:21:03  -    -
                                   ready            reached      2021-09-19T09:21:03  -    -
cfg-com-type  config  false  -     init             reached      2021-09-19T09:21:03  -    -
                                   approver-notif   reached      2021-09-19T09:21:03  -    create-reached
                                   approved         not-reached  -                    -    -
                                   pre-test         not-reached  -                    -    -
                                   banner-cfg       not-reached  -                    -    -
                                   name-server-cfg  not-reached  -                    -    -
                                   post-test        not-reached  -                    -    -
                                   ready            not-reached  -                    -    -

                                                                         JEOPARDY                       VIOLATION           SUCCESS
NAME          POLICY           START TIME           JEOPARDY TIME        RESULT    VIOLATION TIME       RESULT     STATUS   TIME
-------------------------------------------------------------------------------------------------------------------------------------
nano-spm-123  nano-sla-policy  2021-09-19T12:21:03  2021-09-19T12:51:03  -         2021-09-19T13:21:03  -          running  -

engineer@ncs#

The Nano service plan in WebUI:

The Nano service plan

Note that the user cannot add the approved flag to the service configuration because of the statically defined NACM rules in the configs/nacm.xml file.

The sla component is responsible for creating an SPM trigger to track the SLA of our service instance.

Also, note that config component is stopped on approver-notif state. That means, we need an another user to login and change configuration of the service by adding approved flag.

Approve the service

approver user got a notification.

A notification for the approver

Log in to NSO with the user approver and password approved. Pay attention, that approver cannot change service's configuration; commands device and name-server are not visible or accessible.

$ make cli-app
ssh approver@0.0.0.0 -p 2024 -o PreferredAuthentications=password -o PasswordAuthentication=yes -o PubkeyAuthentication=no
approver@0.0.0.0's password:

approver connected from 127.0.0.1 using ssh on localhost
approver@ncs# c
approver@ncs# config
Entering configuration mode terminal
approver@ncs(config)# nano nano 123
approver@ncs(config-nano-123)# dev?
                               ^
% Invalid input detected at '^' marker.
approver@ncs(config-nano-123)# ?
Possible completions:
  approved             Approved flag
  check-sync           Check if device config is according to the service
  commit-queue
  deep-check-sync      Check if device config is according to the service
  get-modifications    Get the data this service created
  log
  notify               Service notifications
  plan
  re-deploy            Run/Dry-run the service logic again
  reactive-re-deploy   Reactive re-deploy of service logic
  tests                Run tests
  touch                Mark the service as changed
  un-deploy            Undo the effects of the service
  ---
  commit               Commit current set of changes
  describe             Display transparent command information
  exit                 Exit from current mode
  help                 Provide help information
  no                   Negate a command or set its defaults
  pwd                  Display current mode path
  rload                Load configuration from an ASCII file or from terminal, relative to current location
  top                  Exit to top level and optionally run command
approver@ncs(config-nano-123)# approved
approver@ncs(config-nano-123)# commit dry-run
cli {
    local-node {
        data  nano {
                  nano 123 {
             +        approved true;
                  }
              }
    }
}
approver@ncs(config-nano-123)# commit
Commit complete.
approver@ncs(config-nano-123)#
System message at 2021-09-19 12:26:51...
Commit performed by approver via ssh using cli.
approver@ncs(config-nano-123)#
System message at 2021-09-19 12:26:52...
Commit performed by approver via system using cli.
approver@ncs(config-nano-123)#
System message at 2021-09-19 12:26:52...
Commit performed by approver via system using cli.
approver@ncs(config-nano-123)#
System message at 2021-09-19 12:26:53...
Commit performed by approver via system using cli.
approver@ncs(config-nano-123)#
System message at 2021-09-19 12:26:53...
Commit performed by approver via system using cli.
approver@ncs(config-nano-123)#
System message at 2021-09-19 12:26:53...
Commit performed by approver via system using cli.
approver@ncs(config-nano-123)# end
approver@ncs#
approver@ncs# show running-config nano
nano nano 123
 device   r0
 approved
!
nano sla webex room-id %ENV{WEBEX_ROOM_ID}
nano sla webex bot-token %ENV{WEBEX_BOT_TOKEN}
approver@ncs(config)# exit
approver@ncs#

Once the approved flag is committed, a series of commits will finish all subsequent steps, including pre-test, change a banner and a name server, post-test. As a part of SPM policy, a notification will be send.

Success

In case of missed SLA, separate notifications will be sent by SPM policy trigger.

SLA violation notifications

The plan

Check the executed plan of our service instance and generated and delivered device configuration in NSO native format.

approver@ncs# show nano
nano nano 123
 modified devices [ r0 ]
 directly-modified devices [ r0 ]
 device-list   [ r0 ]
 plan-location /nano/nano[id='123']
                      BACK                                                             POST ACTION
TYPE          NAME    TRACK  GOAL  STATE            STATUS   WHEN                 ref  STATUS
-------------------------------------------------------------------------------------------------------
self          self    false  -     init             reached  2021-09-19T09:21:03  -    -
                                   ready            reached  2021-09-19T09:26:53  -    -
sla-com-type  sla     false  -     init             reached  2021-09-19T09:21:03  -    -
                                   trigger-created  reached  2021-09-19T09:21:03  -    -
                                   ready            reached  2021-09-19T09:21:03  -    -
cfg-com-type  config  false  -     init             reached  2021-09-19T09:21:03  -    -
                                   approver-notif   reached  2021-09-19T09:21:03  -    create-reached
                                   approved         reached  2021-09-19T09:26:51  -    -
                                   pre-test         reached  2021-09-19T09:26:51  -    create-reached
                                   banner-cfg       reached  2021-09-19T09:26:51  -    -
                                   name-server-cfg  reached  2021-09-19T09:26:52  -    -
                                   post-test        reached  2021-09-19T09:26:53  -    create-reached
                                   ready            reached  2021-09-19T09:26:53  -    -

                                                                         JEOPARDY                       VIOLATION
NAME          POLICY           START TIME           JEOPARDY TIME        RESULT    VIOLATION TIME       RESULT     STATUS      SUCCESS TIME
----------------------------------------------------------------------------------------------------------------------------------------------------
nano-spm-123  nano-sla-policy  2021-09-19T12:21:03  2021-09-19T12:51:03  passed    2021-09-19T13:21:03  passed     successful  2021-09-19T12:26:53

approver@ncs(config)# nano nano 123 get-modifications
cli {
    local-node {
        data  devices {
                   device r0 {
                       config {
                           ip {
                               domain {
              +                    list example.com {
              +                    }
                                   lookup-conf {
              +                        lookup true;
                                   }
              +                    name example.com;
                               }
                               name-server {
              +                    name-server-list 208.67.222.222 {
              +                    }
                               }
                           }
                           banner {
              +                motd "The message of the day";
                           }
                       }
                   }
               }
               service-progress-monitoring {
              +    trigger nano-spm-123 {
              +        policy nano-sla-policy;
              +        component self;
              +        target /nano/nano[id='123'];
              +    }
               }

    }
}
approver@ncs(config)# exit
approver@ncs#

Delete the service

To entirely delete the service instance, you need to remove the SLA policy firstly.

admin@ncs(config)# end
admin@ncs# c
admin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# nano nano 123
admin@ncs(config-nano-123)# plan ?
Possible completions:
  component   A component has a type and a list of states.
admin@ncs(config-nano-123)# plan component ?
Possible completions:
  cfg-com-type  self  sla-com-type
admin@ncs(config-nano-123)# plan component sla-com-type sla ?
Possible completions:
  force-back-track   Force a component to back-track
  state              A plan state represents a certain step or stage that a service needs to execute and/or reach.
admin@ncs(config-nano-123)# plan component sla-com-type sla force-back-track
result true
admin@ncs(config-nano-123)# no nano nano 123
admin@ncs(config)# commit dry-run
cli {
    local-node {
        data  devices {
                  device r0 {
                      config {
                          ip {
                              domain {
             -                    list example.com {
             -                    }
                                  lookup-conf {
             -                        lookup true;
                                  }
             -                    name example.com;
                              }
                              name-server {
             -                    name-server-list 208.67.222.222 {
             -                    }
                              }
                          }
                          banner {
             -                motd "The message of the day";
                          }
                      }
                  }
              }
              nano {
             -    nano 123 {
             -        device r0;
             -        approved true;
             -    }
              }
    }
}
admin@ncs(config)# commit
Commit complete.
admin@ncs(config)#
System message at 2021-09-19 12:41:20...
Commit performed by admin via system using cli.
admin@ncs(config)#
System message at 2021-09-19 12:41:20...
Commit performed by admin via system using cli.
admin@ncs(config)#
System message at 2021-09-19 12:41:20...
Commit performed by admin via system using cli.
admin@ncs(config)#
System message at 2021-09-19 12:41:20...
Commit performed by admin via system using cli.
admin@ncs(config)#

You may experiment a bit, revoke your approval, and see how NSO will back-track all the changes that cannot exist without the approve flag.

NSO learning labs are a great way to explore capabilities and NSO and start learning:

Another option is to install NSO in Local Install mode and learn in your own pace. For references check README.md.

View code on GitHub

Code Exchange Community

Get help, share code, and collaborate with other developers in the Code Exchange community.View Community
Disclaimer:
Cisco provides Code Exchange for convenience and informational purposes only, with no support of any kind. This page contains information and links from third-party websites that are governed by their own separate terms. Reference to a project or contributor on this page does not imply any affiliation with or endorsement by Cisco.