Building and Installing a NETCONF NED

The NETCONF NED Builder is a development tool that is built into NSO for building NETCONF NEDs. The process of creating and installing a NED consists of the following steps:

1. Make the YANG data models available to NSO
2. Build the NED (Network Equipment Driver) from the YANG data models
3. Install the NED
4. Make the NED available to NSO

Make the YANG Data Models Available to NSO

You will first need to create a NETCONF NED Builder project for the device test1a.

SSH NSO Window:

Copyncs-run$ ncs_cli -C -u admin
admin@ncs# devtools true
admin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# netconf-ned-builder project tailf-dhcpd 1.0 device test1a local-user admin vendor Tail-f
admin@ncs(config-project-tailf-dhcpd/1.0)# commit
admin@ncs(config-project-tailf-dhcpd/1.0)# exit
admin@ncs(config)# exit
admin@ncs# show netconf-ned-builder project tail-dhcpd
netconf-ned-builder project tailf-dhcpd 1.0
 download-cache-path ./state/netconf-ned-builder/cache/tailf-dhcpd-nc-1.0
 ned-directory-path  ./state/netconf-ned-builder/tailf-dhcpd-nc-1.0
admin@ncs#

The NETCONF NED Builder is a developer tool and needs to be enabled first through the devtools command to set it to true. The NETCONF NED Builder feature is not expected to be used by the end users of NSO.

The project argument for the netconf-ned-builder command requires both the project name and a version number for the NED being built. A reasonable version number to pick is the version number of the device OS. The name of the device for the NED being built should match the one that was configured in the previous section.

Copying Manually to the Cache Directory

If the device supports the NETCONF <get-schema> RPC call, you can skip to the next section of Using NETCONF. Otherwise, you can copy all available YANG modules of the device to the download-cache-path as shown in the previous section.

SSH NSO Window:

Copyncs-run $ ls ./state/netconf-ned-builder/cache/tailf-dhcp-nc-1.0/
`dhcpd@.yang`
`iana-crypt-hash@2014-08-06.yang`
`ietf-inet-types@2013-07-15.yang`
`ietf-netconf-acm@2018-02-14.yang`
`ietf-netconf-monitoring@2010-10-04.yang`
`ietf-netconf-notifications@2012-02-06.yang`
`ietf-netconf-partial-lock@2009-10-19.yang`
`ietf-netconf-with-defaults@2011-06-01.yang`
`ietf-netconf@2011-06-01.yang`
`ietf-restconf-monitoring@2017-01-26.yang`
`ietf-x509-cert-to-name@2014-12-10.yang`
`ietf-yang-library@2016-06-21.yang`
`ietf-yang-types@2013-07-15.yang`
`tailf-aaa@2018-09-12.yang`
`tailf-acm@2013-03-07.yang`
`tailf-cli-extensions@2019-05-16.yang`
`tailf-common-monitoring@2019-04-09.yang`
`tailf-common-query@2017-12-15.yang`
`tailf-common@2019-05-16.yang`
`tailf-confd-monitoring@2019-04-09.yang`
`tailf-kicker@2017-09-28.yang`
`tailf-meta-extensions@2017-03-08.yang`
`tailf-netconf-extensions@2019-04-09.yang`
`tailf-netconf-forward@.yang`
`tailf-netconf-inactive@.yang`
`tailf-netconf-monitoring@2019-03-28.yang`
`tailf-netconf-query@2017-01-06.yang`
`tailf-netconf-rollback@2018-01-12.yang`
`tailf-netconf-transactions@.yang`
`tailf-netconf-with-transaction-id@2018-11-23.yang`
`tailf-progress@2018-06-14.yang`
`tailf-rollback@2018-01-12.yang`
`tailf-tls@2019-06-13.yang`
`tailf-webui@2013-03-07.yang`
`tailf-xsd-types@2017-11-20.yang`

The YANG modules added need to be registered with the NETCONF NED builder in the same way as for YANG annotation files below.

Adding YANG annotation files

Sometimes you may want to annotate the YANG data models that were downloaded from the device. For example, when an encrypted string is stored on the device, the encrypted value that is stored on the device will differ from the value stored in NSO as the two initialization vectors differ.

Let’s say you have a YANG data model like this:

Copymodule my-yang-module {
  namespace "urn:my-yang-module";
  prefix m;
  …
  grouping my-grouping {
   container my-container {
      leaf my-encrypted-password {
         type tailf:aes-cfb-128-encrypted-string;
     }
    }
  }
}

You then create a YANG annotation module like this one:

Copymodule my-yang-module-ann {
  namespace "urn:my-yang-module-ann";
  prefix m-ann;
  import tailf-common {
    prefix tailf;
  }
  tailf:annotate-module " my-yang-module" {
    tailf:annotate-statement "grouping[name='my-grouping']" {
     tailf:annotate-statement "container[name='my-container']" {
        tailf:annotate-statement "leaf[name=' my-encrypted-password']" {
          tailf:ned-ignore-compare-config;
        }
      }
    }
  }
}

After downloading the YANG data models and before building the NED with the NED builder, you need to register the annotation module with the NSO NED builder using the NSO tool "ncs_cmd":

Copy$ cp /path/to/my-yang-module-ann.yang/$MY_NSO_PROJECT/state/netconf-ned-builder/cache/$MY_NED_NAME-nc-1.0/
$ ncs_cmd -c 'mcreate "/netconf-ned-builder/project{/$MY_NED_NAME 1.0}/module{my-yang-module-ann 2021-mm-dd}"; mset "/netconf-ned-builder/project{$MY_NED_NAME 1.0}/module{my-yang-module-ann 2021-mm-dd}/namespace" "urn:my-yang-module-ann"; mset "/netconf-ned-builder/project{$MY_NED_NAME 1.0}/module{my-yang-module-ann 2021-mm-dd}/location" "NETCONF"; mset "/netconf-ned-builder/project{$MY_NED_NAME 1.0}/module{my-yang-module-ann 2021-mm-dd}/status" "selected downloaded"'

Another option is to create a development NED and add the annotation module to the YANG model directory in the development NED before a manual build. See the make-development-ned command in the troubleshooting section below.

Using NETCONF

If the device supports requests, the device can be contacted directly for downloading the YANG data models. For a device using ConfD, the ietf-netconf-monitoring.fxs must be in the loadpath of your ConfD installation for to work. These YANG source files are what ConfD will return when the NETCONF RPC is issued to the device from NSO.

SSH NSO Window:

Copyncs-run$ ncs_cli -C -u admin
admin@ncs# devtools true
admin@ncs# netconf-ned-builder project tailf-dhcp 1.0 fetch-module-list
admin@ncs# show netconf-ned-builder project tailf-dhcp 1.0 module
module dhcpd ""
 namespace http://tail-f.com/ns/example/dhcpd
 location  [ NETCONF ]
module ietf-inet-types 2013-07-15
 namespace urn:ietf:params:xml:ns:yang:ietf-inet-types
 location  [ NETCONF ]
…
module tailf-common 2019-05-16
 namespace http://tail-f.com/yang/common
 location  [ NETCONF ]
 submodule tailf-cli-extensions 2019-05-16
  location [ NETCONF ]
 submodule tailf-meta-extensions 2017-03-08
  location [ NETCONF ]
…
module tailf-xsd-types 2017-11-20
 namespace http://tail-f.com/yang/xsd-types
 location  [ NETCONF ]
admin@ncs# netconf-ned-builder project tailf-dhcp 1.0 module dhcpd "" select
admin@ncs# show netconf-ned-builder project tailf-dhcp 1.0 module status
NAME             REVISION    STATUS               
--------------------------------------------------
dhcpd                        selected,downloaded  
ietf-inet-types  2013-07-15  selected,downloaded  
tailf-common     2019-05-16  selected,downloaded  
tailf-xsd-types  2017-11-20  selected,downloaded  

The fetch-module-list command will either look for existing YANG modules in the download-cache-path folder or issue and to fetch the YANG modules over NETCONF and put them in the download-cache-path folder. You can then view the list of downloaded modules and select the ones that you would like to include in the NETCONF NED. When you select a top-level module that has dependencies on other modules, the dependent modules are automatically selected. With the example above, I have chosen only the dhcpd module with an empty revision as the dhcpd module didn’t contain any revision date information. A typical YANG module should contain the revision date information. The dependent modules are automatically selected, such as the ones listed above for the dhcpd module including ietf-inet-types, tailf-common and tailf-xsd-types. If you would like to select all available modules, you can use the wild card for both fields. You can also use the deselect command to exclude modules from the build.

YANG Modules to Include in the NETCONF NED

It is recommended to select only a subset of all available YANG models on the device that are relevant for the use cases that you are interested in validating for NSO interoperability. It is important to note that you should be exposing only a single consistent view of the device datastore through the NETCONF NED. For example, if your device supports both the native proprietary modules along with standards-based modules (such as IETF or OpenConfig) in which the same data nodes exist in both, you should build the NETCONF NED for either just the native modules or just the standards- based modules. Including both will cause NSO to get out-of-sync with the device after a request when a common data node from only one of the two views is specified in the request.

Build the NED from the YANG Data Models

An NSO Network Element Driver (NED) is a package containing the device’s YANG data models. The NED must first be built, then installed in NSO, and finally the package must be loaded in order for NSO to communicate with the device via NETCONF using that device’s YANG data models.

After the files have been downloaded from the device, they must first be built before they can be used. The following example shows how to build a NED for the device test1a:

SSH NSO Window:

Copyadmin@ncs(config)# netconf-ned-builder project tailf-dhcpd 1.0 build-ned
admin@ncs(config)# exit
admin@ncs# show netconf-ned-builder project tailf-dhcpd 1.0 build-status
build-status success

Note:
Make sure no errors occur during compilation. If any errors appear in the compilation output, resolve the issues in you YANG data models and re-upload them before continuing.

As the compilation will halt if an error is found in a YANG data model, it can be helpful to first do a quick check of all YANG data models at once using a shell script plus the yanger tool. Example:

Copy$ cat check.sh
#!/bin/sh
for f in *.yang
do
    $NCS_DIR/bin/yanger -p yang $f
done

YANG data models that don’t compile due to YANG RFC compliance issues can either be updated in the cache folder directly or in the device and re-uploaded again through <get-schema> by removing them from the cache folder and repeating the previous process again to rebuild the NED. They can also be excluded from the build by not selecting them if they aren’t needed for your use case.

Troubleshoot NED building issues

To see any warnings after building the NED, they can be found in the build-warning leaf under the module list entry. It is always a good idea to clean up the build warnings in your YANG models. A command line example for showing the build warnings is as follows:

SSH NSO Window:

Copyadmin@ncs# show netconf-ned-builder project tailf-dhcpd 1.0 module build-warning

If the build failed in the previous step with errors, the build errors can be found in the build-error leaf under the module list entry. A command line example for showing the build errors is as follows:

SSH NSO Window:

Copyadmin@ncs# show netconf-ned-builder project tailf-dhcpd 1.0 module build-error

To see the full compiler output for debugging purposes, they can be found in the compiler-output leaf under the project list entry. The compiler-output leaf is hidden by the hide-group debug and may be accessed in the CLI using the unhide debug command if the hide-group is configured in ncs.conf at the top level as follows:

Copy<hide-group>
    <name>debug</name>
</hide-group>

A reload of NSO is needed for the configuration changes made to ncs.conf to take effect. You will run ncs –-reload in your Linux shell.

A command line example for showing the full compiler output is as follows:

SSH NSO Window:

Copyadmin@ncs# unhide debug
admin@ncs# show netconf-ned-builder project tailf-dhcp 1.0 compiler-output

The unhide debug command only needs to be invoked once per CLI session.

As an alternative to debugging the NED building issues inside of an NSO CLI session, the NED building environment can be exported. Following shows the step for exporting the NED and building it in your Linux shell.

SSH NSO Window:

Copyadmin@ncs(config)# netconf-ned-builder project tailf-dhcpd 1.0 make-development-ned in-directory /tmp
admin@ncs(config)# exit
$ cd /tmp/tailf-dhcp-nc-1.0/src
$ make clean all
'''

The YANG models for your device can be found in the yang directory under `/tmp/tailf-dhcp-nc-1.0/src`. Once all of the errors in the YANG models have been fixed, you can then go back to building the NETCONF NED using the NETCONF NED Builder and continue with the next step.

## Prepare the NED for Loading

After the YANG data model files have been compiled into the NED, the NED package must be exported before the YANG data models can be loaded into the NSO runtime. The following example shows how to export the NED that was just built for the device `test1a`:

SSH NSO Window:
```Terminal
ncs-run$ ncs_cli -C -u admin
admin@ncs# devtools true
admin@ncs# netconf-ned-builder project tailf-dhcp 1.0 export-ned to-directory /ncs-run/packages
tar-file /ncs-run/packages/ncs-5.2.0.3-tailf-dhcp-nc-1.0.tar.gz
admin@ncs# exit

Please note that for non-root users with write permission to the exported directory, you will need to configure your NSO user with the same uid (id -u) as your non-root Linux user by doing the following:

Copyadmin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# aaa authentication users user admin uid 100
admin@ncs(config)# commit

This example assumes 100 is the uid of the Linux user running NSO.

Load the Compiled YANG Files

Once the NED package has been copied to the packages folder of the NSO project, the NED package is now ready to be loaded into the NSO.

SSH NSO Window:

Copyadmin@ncs# packages reload

>>> System upgrade is starting.
>>> Sessions in configure mode must exit to operational mode.
>>> No configuration changes can be performed until upgrade has completed.
>>> System upgrade has completed successfully.
reload-result {
    package drned-xmnr
    result true
}
reload-result {
    package tailf-dhcpd-nc-1.0
    result true
}
admin@ncs#
System message at 2019-10-08 19:12:04...
    Subsystem stopped: ncs-dp-19222-drned-xmnr:drned_xmnr
admin@ncs#
System message at 2019-10-08 19:12:04...
    Subsystem started: ncs-dp-99309-drned-xmnr:drned_xmnr
admin@ncs#

After the package has been loaded, NSO will be able to manage devices matching your YANG data models.

Update the ned-id for the test1a Device

Now that the NETCONF NED has been built for the test1a device, the ned-id for test1a needs to be updated before the NED can be used to manage the device.

SSH NSO Window:

Copyadmin@ncs# config
admin@ncs(config)# devices device test1a device-type netconf ned-id
Possible completions:
  lsa-netconf  netconf  snmp  tailf-dhcpd-nc-1.0
admin@ncs(config)# devices device test1a device-type netconf ned-id tailf-dhcpd-nc-1.0
admin@ncs(config-device-test1a)# commit
Commit complete.
admin@ncs(config-device-test1a)#

How to Remove a NED from NSO

Installed NED packages can be removed from NSO by deleting them from the packages folder of the NSO project followed by deleting the device and the NETCONF NED project through ncs_cli. The following example shows how to uninstall a NED built for the device test1a:

SSH NSO Window:

Copyncs-run$ rm packages/ncs-5.2.0.3-tailf-dhcpd-nc-1.0.tar.gz
ncs-run$ ncs_cli -C -u admin
admin@ncs# devtools true
admin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# no devices device test1a
admin@ncs(config)# no netconf-ned-builder project tailf-dhcp 1.0  
admin@ncs(config)# commit
Commit complete.
admin@ncs(config-device-test1a)# exit
admin@ncs(config)# exit
admin@ncs# packages reload
Error: The following modules will be deleted by upgrade:
tailf-dhcpd-nc-1.0: dhcpd
tailf-dhcpd-nc-1.0: tailf-dhcpd-nc
tailf-dhcpd-nc-1.0: tailf-dhcpd-nc-1.0
If this is intended, proceed with 'force' parameter.
admin@ncs# packages reload force

>>> System upgrade is starting.
>>> Sessions in configure mode must exit to operational mode.
>>> No configuration changes can be performed until upgrade has completed.
>>> System upgrade has completed successfully.
reload-result {
    package drned-xmnr
    result true
}
admin@ncs#
System message at 2019-10-08 19:30:22...
    Subsystem stopped: ncs-dp-983-drned-xmnr:drned_xmnr
admin@ncs#
System message at 2019-10-08 19:30:23...
    Subsystem started: ncs-dp-1847-drned-xmnr:drned_xmnr