- About
- NSO 5.7 Getting Started Guide
- NSO 5.7 User Guide
- NSO Installation Guide
- NSO 5.7 Administration Guide
- NSO 5.7 Northbound APIs
- NSO 5.7 Development Guide
- Preface
- The Configuration Database and YANG
- Basic Automation with Python
- Creating a Service
- Applications in NSO
- The NSO Java VM
- The NSO Python VM
- Embedded Erlang applications
- The YANG Data Modeling Language
- Using CDB
- Java API Overview
- Python API Overview
- NSO Packages
- Package Development
- Developing NSO Services
- Templates
- NED Upgrades and Migration
- Developing Alarm Applications
- SNMP Notification Receiver
- The web server
- Kicker
- Scheduler
- Progress Trace
- Nano Services for Staged Provisioning
- Encryption Keys
- External Logging
- NSO 5.7 Web UI
- NSO CDM Migration Guide
- NSO Layered Service Architecture
- NSO 5.7 NED Development
- Preface
- Introduction
- Network Element Drivers (NEDs)
- Network Element Drivers (NEDs)
- Introduction
- The SNMP NED
- NED identification
- NED Version Scheme
- Dumb versus capable devices
- The CLI NED
- Generic NED
- Getting started with a CLI NED
- Getting started with a generic NED
- NED commands
- Statistics
- Multi NEDs for Statistics
- Making the NED handle default values properly
- Using set hooks in the NED
- Dry-run considerations
- Naming Conventions
- Revision Merge Functionality
- NETCONF NED Builder
- Writing a data model for a CLI NED
- Tail-f CLI NED Annotations
- NSO 5.7 Manual Pages
- SDK API Reference
- NSO on DevNet
- Get Support
NSO knows how to automatically communicate southbound to NETCONF and SNMP enabled devices. By supplying NSO with the YANG models of a NETCONF device, NSO knows the data models of the device, and through the NETCONF protocol knows exactly how to manipulate the device configuration. This can be used for a NETCONF device such as a Juniper router, any device that uses ConfD as management system, or any other device that runs a compliant NETCONF server. Similarly, by providing NSO with the MIBs for a device, NSO can automatically manage such a device.
Unfortunately, the majority of existing devices in current networks do not speak NETCONF and SNMP is usually mostly used to retrieve data from devices. By far the most common way to configure network devices is through the CLI. Management systems typically connect over SSH to the CLI of the device and issue series of CLI configuration commands. Some devices do not even have a CLI, and thus SNMP, or even worse, various proprietary protocols, are used to configure the device.
NSO can speak southbound not only to NETCONF-enabled devices, but through the NED architecture it can speak to an arbitrary management interface. This is of course not entirely automatic like with NETCONF, and depending on the type of interface the device has for configuration, this may involve some programming. SNMP devices can be managed automatically, by supplying NSO with the MIBs for the device, with some additional declarative annotations. Devices with a Cisco style CLI can be managed by writing YANG models describing the data in the CLI, and a relatively thin layer of Java code to handle the communication to the devices. Other types of devices require more coding.
The NSO architecture is described in the picture below, with a built-in NED for NETCONF, another built-in NED for SNMP, one NED for Cisco CLIs, and a generic NED for other protocols. The NED is the adaptation layer between the XML representation of the network configuration contained inside NSO and the wire protocol between NSO and managed devices. The NETCONF and SNMP NEDs are built in, the CLI NED is entirely model driven, whereas the generic NED requires a Java program to translate operations on the NSO XML tree into configuration operations towards the device. Depending on what means are used to configure the device, this may be more or less complicated.
![]() |
NSO NED architecture
NSO can use SNMP to configure a managed device, under certain circumstances. SNMP in general is not suitable for configuration, and it is important to understand why:
-
In SNMP, the size of a SET request, which is used to write to a device, is limited to what fits into one UDP packet. This means that a large configuration change must be split into many packets. Each such packet contains some parameters to set, and each such packet is applied on its own by the device. If one SET request out of many fails, there is no abort command to undo the already applied changes, meaning that rollback is very difficult.
-
The data modelling language used in SNMP, SMIv2, does not distinguish between configuration objects and other writable objects. This means that it is not possible to retrieve only the configuration from a device without explicit, exact knowledge of all objects in all MIBs supported by the device.
-
SNMP supports only two basic operations, read and write. There is no protocol support for creating or deleting data. Such operations must be modeled in the MIBs, explicitly.
-
SMIv2 has limited support for semantic constraints in the data model. This means that it is difficult to know if a certain configuration will apply cleanly on a device. If it doesn't, rollback is tricky, as explained above.
-
Because of all of the above, ordering of SET requests becomes very important. If a device refuses to create some object A before another B, an SNMP manager must make sure to create B before creating A. It is also common that objects cannot be modified without first making them disabled or inactive. There is no standard way to do this, so again, different data models do this in different ways.
Despite all this, if a device can be configured over SNMP, NSO can use its built-in multilingual SNMP manager to communicate with the device. However, in order to solve the problems mentioned above, the MIBs supported by the device need to be carefully annotated with some additional information that instruct NSO on how to write configuration data to the device. This additional information is described in detail below.
To add a device, the following steps need to be followed. They are described in more details in the following sections.
-
Collect (a subset of) the MIBs supported by the device.
-
Optionally annotate the MIBs with annotations to instruct NSO on how to talk to the device, for example ordering dependencies that are not explicitly modeled in the MIB. This step is not required.
-
Compile the MIBs and load them into NSO.
-
Configure NSO with the address and authentication parameter for the SNMP devices.
-
Optionally configure a named MIB group in NSO with the MIBs supported by the device, and configure the managed device in NSO to use this MIB group. If this step is not done, NSO assumes the device implements all MIBs known to NSO.
(See the Makefile
snmp-ned/basic/packages/ex-snmp-ned/src/Makefile
,
for an example of the below description.) Make sure that
you have all MIBs available, including import dependencies
and that they contain no errors.
The ncsc --ncs-compile-mib-bundle compiler
is used to compile MIBs and MIB annotation files into NSO load
files. Assuming a directory with input MIB files (and optional
MIB annotation files) exist, the following command compiles
all the MIBs in device-models
and writes
the output to ncs-device-model-dir
.
$ ncsc --ncs-compile-mib-bundle device-models \ --ncs-device-dir ./ncs-device-model-dir
The compilation steps performed by the ncsc --ncs-compile-mib-bundle are elaborated below::
-
Transform the MIBs into YANG according to the IETF standardized mapping (https://www.ietf.org/rfc/rfc6643.txt). The IETF defined mapping makes all MIB objects read-only over NETCONF.
-
Generate YANG deviations from the MIB, this basically makes SMIv2
read-write
objects YANGconfig true
as a YANG deviation. -
Include the optional MIB annotations.
-
Merge the read-only YANG from step 1 with the read-write deviation from step 2.
-
Compile the merged YANG files into NSO load format.
These steps are illustrated in the Figure below:
![]() |
SNMP NED Compile Steps
Finally make sure that the NSO configuration file points to the correct device model directory:
<device-model-dir>./ncs-device-model-dir</device-model-dir>
Each managed device is configured with a name, IP address and port (161 by default), and the SNMP version to use (v1, v2c, or v3).
admin@host# show running-config devices device r3
address 127.0.0.1
port 2503
device-type snmp version v3 snmp-authgroup my-authgroup
state admin-state unlocked
In order to minimize the necessary configuration, the
authentication group concept (see
the section called “Authentication Groups” in NSO 5.7 User Guide)
is used also for
SNMP. A configured managed device of type snmp
refers to an SNMP authgroup. An SNMP authgroup contains
community strings for SNMP v1 and v2c, and USM parameters for
SNMP v3.
admin@host# show running-config devices authgroups snmp-group my-authgroup
devices authgroups snmp-group my-authgroup
default-map community-name public
umap admin
usm remote-name admin
usm security-level auth-priv
usm auth md5 remote-password $4$wIo7Yd068FRwhYYI0d4IDw==
usm priv des remote-password $4$wIo7Yd068FRwhYYI0d4IDw==
!
!
In the example above, when NSO needs to speak to the device "r3", it sees that the device is of type snmp, and that SNMP v3 should be used, with authentication parameters from the SNMP authgroup "my-authgroup". This authgroup maps the local NSO user "admin" to the USM user "admin", with explicit remote passwords given. These passwords will be localized for each SNMP engine that NSO communicates with. While the passwords above are shown encrypted, when you enter them in the CLI you write them in clear-text. Note also that the remote engine id is not configured; NSO performs a discovery process to find it automatically.
No NSO user other than "admin" is mapped by the authgroup "my-authgroup" for SNMP v3.
With SNMP, there is no standardized, generic way for an SNMP manager to learn which MIBs an SNMP agent implements. By default, NSO assumes that an SNMP device implements all MIBs known to NSO, i.e., all MIBs that have been compiled with the ncsc --ncs-compile-mib-bundle command. This works just fine if all SNMP devices NSO manages are of the same type, and implement the same set of MIBs. But if NSO is configured to manage many different SNMP devices, some other mechanism is needed.
In NSO, this problem is solved by using MIB groups. A MIB group is a named collection of MIB module names. A managed SNMP device can refer to one or more MIB groups. For example, below two MIB groups are defined:
admin@ncs# show running-config devices mib-group
devices mib-group basic
mib-module [ BASIC-CONFIG-MIB BASIC-TC ]
!
devices mib-group snmp
mib-module [ SNMP* ]
!
The wildcard '*' can be used only at the end of a string; it is thus used to define a prefix of a MIB module name. So the string "SNMP*" matches all loaded standard SNMP modules, such as SNMPv2-MIB, SNMP-TARGET-MIB etc.
An SNMP device can then be configured to refer to one or more of the MIB groups:
admin@ncs# show running-config devices device r3 device-type snmp
devices device r3
device-type snmp version v3
device-type snmp snmp-authgroup default
device-type snmp mib-group [ basic snmp ]
!
Most annotations for MIB objects are used to instruct NSO on how to split a large transaction into suitable SNMP SET requests. This step is not necessary for a default integration. But when for example ordering dependencies in the MIB is discovered it is better to add this as annotations and let NSO handle the ordering rather then leaving it to the CLI user or Java programmer.
In some cases, NSO can automatically understand when rows in a table must be created or deleted before rows in some other table. Specifically, NSO understands that if a table B has an INDEX object in table A (i.e., B sparsely augments A), then rows in table B must be created after rows in table B, and vice versa for deletions. NSO also understand that if table B AUGMENTS table A, then a row in table A must be created before any column in B is modified.
However, in some MIBs, table dependencies cannot be detected
automatically. In this case, these tables must be annotated
with a sort-priority
. By default, all rows
have sort-priority 0. If table A has a lower sort-priority
than table B, then rows in table A are created before rows in
table B.
In some tables, existing rows cannot be modified unless the
row is inactivated. Once inactive, the row can be modified,
and then activated again. Unfortunately, there is no formal
way to declare this is SMIv2, so these tables must be
annotated with two statements;
ned-set-before-row-modification
and
ned-modification-dependent
. The
former is used to instruct NSO which column and which value is
used to inactivate a row, and the latter is used on each
column that requires the row to be inactivated before
modification.
ned-modification-dependent
can be used
in the same table as
ned-set-before-row-modification
, or in
a table that augments or sparsely augments the table with
ned-set-before-row-modification
.
By default, NSO treats a writable SMIv2 object as configuration, except if the object is of type RowStatus. Any writable object that does not represent configuration must be listed in a MIB annotation file when the MIB is compiled, with the "operational" modifier.
When NSO retrieves data from an SNMP device, e.g., when doing
a sync from-device
, it uses the GET-NEXT request
to scan the table for available rows. When doing the
GET-NEXT, NSO must ask for an accessible column. If the row
has a column of type RowStatus, NSO uses this column.
Otherwise, if the one of the INDEX objects are accessible, it
uses this object. Otherwise, if table has been annotated with
ned-accessible-column
, this column is
used. And, as a last resort, NSO does not indicate any column
in the first GET-NEXT request, and uses the column returned
from the device in subsequent requests. If the table has
"holes" for this column, i.e., the column is not instantiated
in all rows, NSO will not detect those rows.
NSO can automatically create and delete table rows for tables that use the RowStatus TEXTUAL-CONVENTION, defined in RFC 2580.
It is pretty common to mix configuration objects with non-configuration objects in MIBs. Specifically, it is quite common that rows are created automatically by the device, but then some columns in the row are treated as configuration data. In this case, the application programmer must tell NSO to sync from the device before attempting to modify the configuration columns, in order to let NSO learn which rows exist on the device.
Some SNMP agents require a certain order of row deletions and
creations. By default, the SNMP NED send all creates before
deletes. The annotation ned-delete-before-create
can be used on a table entry in order to send row deletions
before row creations, for that table.
Sometimes rows in some SNMP agents cannot be modified once
created. Such rows can be marked with the annotation
ned-recreate-when-modified
. This makes the SNMP
NED to first delete the row, and then immediately recreate it
with the new values.
A good starting point for understanding annotations is to look at
the example in examples.ncs/snmp-ned
directory.
The BASIC-CONFIG-MIB mib has a table where rows can be modified if
the bscActAdminState
is set to locked. In order to
have NSO do this automatically when modifying entries rather
then leaving it to users an annotation file can be created. See
the BASIC-CONFIG-MIB.miba
which contains the following:
## NCS Annotation module for BASIC-CONFIG-MIB bscActAdminState ned-set-before-row-modification = locked bscActFlow ned-modification-dependent
This tells NSO that before modifying the
bscActFlow
column set the
bscActAdminState
to locked and restore the
previous value after committing the set operation.
All MIB annotations for a particular MIB are written to a file
with the file suffix .miba
.
See
mib_annotations(5) in NSO 5.7 Manual Pages
for details.
Make sure that the MIB annotation file is put into the directory where all the MIB files are which is given as input to the ncsc --ncs-compile-mib-bundle command
NSO can manage SNMP devices within transactions, a transaction can span Cisco devices, NETCONF devices and SNMP devices. If a transaction fails NSO will generate the reverse operation to the SNMP device.
The basic features of the SNMP will be illustrated below by using the
examples.ncs/snmp-ned
example. First try to connect
to all SNMP devices:
admin@ncs# devices connect
connect-result {
device r1
result true
info (admin) Connected to r1 - 127.0.0.1:2501
}
connect-result {
device r2
result true
info (admin) Connected to r2 - 127.0.0.1:2502
}
connect-result {
device r3
result true
info (admin) Connected to r3 - 127.0.0.1:2503
}
When NSO executes the connect request for SNMP devices it performs a get-next request with 1.1 as var-bind. When working with the SNMP NED it is helpful to turn on the NED tracing:
$ ncs_cli -C -u admin
admin@ncs config
admin@ncs(config)# devices global-settings trace pretty trace-dir .
admin@ncs(config)# commit
Commit complete.
This creates a trace-file named ned-devicename.trace. The trace for the ncs connect action looks like:
$ more ned-r1.trace
get-next-request reqid=2
1.1
get-response reqid=2
1.3.6.1.2.1.1.1.0=Tail-f ConfD agent - 1
When looking at SNMP trace files it is useful to have the OBJECT-DESCRIPTOR rather than the OBJECT-IDENTIFIER. To do this, pipe the trace file to the smixlate tool:
$ more ned-r1.trace | smixlate $NCS_DIR/src/ncs/snmp/mibs/SNMPv2-MIB.mib
get-next-request reqid=2
1.1
get-response reqid=2
sysDescr.0=Tail-f ConfD agent - 1
You can access the data in the SNMP systems directly (read-only and read-write objects):
admin@ncs# show devices device live-status
ncs live-device r1
live-status SNMPv2-MIB system sysDescr "Tail-f ConfD agent - 1"
live-status SNMPv2-MIB system sysObjectID 1.3.6.1.4.1.24961
live-status SNMPv2-MIB system sysUpTime 596197
live-status SNMPv2-MIB system sysContact ""
live-status SNMPv2-MIB system sysName ""
...
NSO can synchronize all writable objects into CDB:
admin@ncs# devices sync-from
sync-result {
device r1
result true
...
admin@ncs# show running-config devices device r1 config r:SNMPv2-MIB
devices device r1
config
system
sysContact ""
sysName ""
sysLocation ""
!
snmp
snmpEnableAuthenTraps disabled;
!
All the standard features of NSO with transactions and roll-backs will work with SNMP devices. The sequence below shows how to enable authentication traps for all devices as one transaction. If any device fails, NSO will automatically rollback the others. At the end of the CLI sequence a manual rollback is shown:
admin@ncs# config
admin@ncs(config)# devices device r1-3 config r:SNMPv2-MIB snmp snmpEnableAuthenTraps enabled
admin@ncs(config)# commit
Commit complete.
admin@ncs(config)# top rollback configuration
admin@ncs(config)# commit dry-run outformat cli
cli devices { device r1 { config { r:SNMPv2-MIB { snmp { - snmpEnableAuthenTraps enabled; + snmpEnableAuthenTraps disabled; } } } } device r2 { config { r:SNMPv2-MIB { snmp { - snmpEnableAuthenTraps enabled; + snmpEnableAuthenTraps disabled; } } } } device r3 { config { r:SNMPv2-MIB { snmp { - snmpEnableAuthenTraps enabled; + snmpEnableAuthenTraps disabled; } } } } }
admin@ncs(config)# commit
Commit complete.
Each managed device in NSO has a device type, which informs NSO
how to communicate with the device. The device type is one of
netconf, snmp,
cli, or generic. For
the cli and generic device types, which are handled by Java NED
code, a special ned-id identifier is
needed. This identifier is a YANG identity, which must be
derived from one of the pre-defined identities in
tailf-ncs-ned.yang
:
module tailf-ncs-ned { namespace "http://tail-f.com/ns/ncs-ned"; prefix ned; import tailf-common { prefix tailf; } organization "Tail-f Systems"; description "This module defines the Tail-f NCS NED base identities. Copyright 2011-2021 Cisco Systems, Inc. All rights reserved. Permission is hereby granted to redistribute this file without modification."; revision 2021-09-02 { description "Released as part of NCS-5.6. Added identity 'generic-ned-notification-id'. Added idenity 'cli-ned-notification-id'."; } revision 2019-04-09 { description "Released as part of NCS-5.1. Added 'ned-id' as base to all protocol specific ned ids."; } revision 2016-11-24 { description "Released as part of NCS-4.3. Added base identity for NETCONF devices. Added identity lsa-netconf"; } revision 2011-06-01 { description "Released as part of NCS-1.6."; } identity ned-id { description "Base identity for Tail-f NEDs."; } identity netconf-ned-id { base ned-id; tailf:abstract; description "Base identity for NETCONF NEDs."; } identity generic-ned-id { base ned-id; tailf:abstract; description "Base identity for generic NEDs."; } identity cli-ned-id { base ned-id; tailf:abstract; description "Base identity for CLI NEDs."; } identity snmp-ned-id { base ned-id; tailf:abstract; description "Base identity for SNMP NEDs. Note that currently there is no way to actually set a ned-id for SNMP devices."; } identity rfc5277-id { base netconf-ned-id; tailf:abstract; description "Special internal id for the data model in RFC 5277."; } identity generic-ned-notification-id { base generic-ned-id; tailf:abstract; description "Special internal id for generic NEDs with notification capability."; } identity cli-ned-notification-id { base cli-ned-id; tailf:abstract; description "Special internal id for CLI NEDs with notification capability."; } identity netconf { base netconf-ned-id; description "Default identity for a netconf device."; } identity lsa-netconf { base netconf-ned-id; description "Base identity for LSA nodes."; } identity snmp { base snmp-ned-id; description "Default identity for an SNMP device."; } }
A YANG model for devices handled by NED code needs to extend the base identity and provide a new identity that can be configured.
import tailf-ncs-ned { prefix ned; } identity cisco-ios { base ned:cli-ned-id; }
The Java NED code registers the identity it handles with NSO.
Similar to how we import device models for NETCONF based devices,we use the
ncsc --ncs-compile-bundle
command to import YANG models
for NED handled devices.
Once we have imported such a YANG model into NSO, we can configure managed device in NSO to be handled by the appropriate NED handler (which is user Java code, more on that later)
admin@ncs# show running config devices device r1
address 127.0.0.1
port 2025
authgroup default
device-type cli ned-id cisco-ios
state admin-state unlocked
...
When NSO needs to communicate southbound towards a managed device which is not of type NETCONF, it will look for a NED that has registered with the name of the identity, in the case above, the string "ios".
Thus before NSO attempts to connect to a NED device, before it tries to sync,
or manipulate the configuration of the device, a user based Java NED code
must have registered with the NSO service manager indicating which Java
class is responsible for the NED with the string of the identity, in this
case the string "ios". This happens automatically when the NSO java VM gets
a instantiate-component
request for a NSO package component of
type ned
.
The component java class myNed needs to implement either of the interfaces NedGeneric or NedCli. Both interfaces require the NED class to implement the following:
// should return "cli" or "generic" String type(); // Which YANG modules are covered by the class String [] modules(); // Which identity is implemented by the class String identity();
The above three callbacks are used by the NSO Java VM to connect the NED
Java class with NSO. They are called at when the NSO Java VM receives the
instantiate-component request
.
The underlying NedMux will start a number of threads, and invoke the registered class with other data callbacks as transactions execute.
A NED has a version associated with it. A version consists of a sequence of numbers separated by dots ('.'). The first two number defines the major and minor version number, the third number defines the maintenance version number and any following numbers are patch release version numbers.
For instance, the 5.8.1 number indicates a maintenance release (1) on the minor release 5.8 and 5.8.1.1 indicates a patch release (1) on the maintenance release 5.8.1. Any incompatible YANG model change will require the major or minor version number to change, i.e. any 5.8.x version is to be backward compatible with the previous.
When a NED release is replaced with a later maintenance/patch release with the same major/minor version, NSO can do a simple data model upgrade to handle stored instance data in CDB. There is no risk that any data would be lost by this sort of upgrade.
On the other hand when a NED is replaced by a new major/minor release this becomes a NED migration. These are non trivial since the YANG model changes can result in loss of instance data if not handled correctly.
NSO differentiates between managed devices that can handle transactions and devices that can not. This discussion applies regardless of NED type, i.e., NETCONF, SNMP, CLI or Generic.
NEDs for devices that cannot handle abort, must indicate so in the reply of the newConnection() method indicating that the NED wants a reverse diff in case of abort. Thus NSO has two different ways to abort a transaction towards a NED, invoke the abort() method with or without a generated reverse diff.
For non transactional devices, we have no other way of trying out a proposed configuration change than to send the change to the device and see what happens.
The table below shows the 7 different data related callbacks that could or must be implemented by all NEDs. It also differentiates between 4 different types of devices and what the NED must do in each callback for the different types of devices.
Non transactional devices | Transactional devices | Transactional devices with confirmed commit | Fully capable NETCONF server |
---|---|---|---|
SNMP, Cisco IOS, NETCONF devices with startup+running | Devices that can abort, NETCONF devices without confirmed commit | Cisco XR type of devices | ConfD, Junos |
INITIALIZE The initialize phase is used to initialize an transaction. For instance if locking or other transaction preparations are necessary they should be performed here. This callback is not mandatory to implement if no NED specific transaction preparations are needed. | |||
initialize(). NED code shall make the device go into config mode (if applicable) and lock (if applicable). | initialize(). NED code shall start a transaction on the device. | initialize(). NED code shall do the equivalent of configure exclusive. | Built in, NSO will lock. |
UNINITIALIZE If the transaction is not completed and the NED has done INITIALIZE this method is called to undo the transaction preparations, that is restoring the NED to the state before INITIALIZE. This callback is not mandatory to implement if no NED specific preparations was performed in INITIALIZE. | |||
uninitialize(). NED code shall unlock (if applicable). | uninitialize(). NED code shall abort the transaction. | uninitialize(). NED code shall abort the transaction. | Built in, NSO will unlock. |
PREPARE In the prepare phase, the NEDs get exposed to all the changes that are destined for each managed device handled by each NED. It is the responsibility of the NED to determine the outcome here. If the NED replies successfully from the prepare phase, NSO assumes the device will be able to go through with the proposed configuration change. | |||
prepare(Data). NED code shall send all data to the device. | prepare(Data). NED code shall add Data to the transaction and validate. | prepare(Data). NED code shall add Data to the transaction and validate. | Built in, NSO will edit-config towards the candidate, validate and commit confirmed with a timeout. |
ABORT If any participants in the transaction reject the proposed changes, all NEDs will be invoked in the abort() method for each managed device the NED handles. It is the responsibility of the NED to make sure that whatever was done in the PREPARE phase is undone. For NEDs that indicate as reply in newConnection() that they want the reverse diff, they will get the reverse data as a parameter here. | |||
abort(ReverseData | null) Either do the equivalent of copy startup to running, or apply the ReverseData to the device. | abort(ReverseData | null). Abort the transaction | abort(ReverseData | null). Abort the transaction | Built in, discard-changes and close. |
COMMIT Once all NEDs that get invoked in commit(Timeout) reply ok, the transaction is permanently committed to the system. The NED may still reject the change in COMMIT. If any NED reject the COMMIT, all participants will be invoked in REVERT, NEDs that support confirmed commit with a timeout, Cisco XR, may choose to use the provided timeout to make REVERT easy to implement. | |||
commit(Timeout). Do nothing | commit(Timeout). Commit the transaction. | commit(Timeout). Execute commit confirmed [Timeout] on the device. | Built in, commit confirmed with the timeout. |
REVERT This state is reached if any NED reports failure in the COMMIT phase. Similar to the ABORT state, the reverse diff is supplied to the NED if the NED has asked for that. | |||
revert(ReverseData | null) Either do the equivalent of copy startup to running, or apply the ReverseData to the device. | revert(ReverseData | null) Either do the equivalent of copy startup to running, or apply the ReverseData to the device. | revert(ReverseData | null). discard-changes | Built in, discard-changes and close. |
PERSIST This state is reached at the end of a successful transaction. Here it's responsibility of the NED to make sure that if the device reboots, the changes are still there. | |||
persist() Either do the equivalent of copy running to startup or nothing. | persist() Either do the equivalent of copy running to startup or nothing. | persist(). confirm. | Built in, commit confirm. |
The following state diagram depicts the different states the NED code goes through in the life of a transaction.

NED transaction states
The CLI NED is magic, it is an entirely model driven way to CLI script towards all Cisco like devices. The basic idea is that the Cisco CLI engine found in ConfD, can be run in both directions.
-
A sequence of Cisco CLI commands can be turned into the equivalent manipulation of the internal XML tree that represents the configuration inside NSO/ConfD. This is the normal mode of operations of ConfD, run in Cisco mode.
A YANG model, annotated appropriately, will produce a Cisco CLI. The user can enter Cisco commands and ConfD will, using the annotated YANG model, parse the Cisco CLI commands and change the internal XML tree accordingly. Thus this is the CLI parser and interpreter. Model driven.
-
The reverse operation is also possible. Given two different XML trees, each representing a configuration state, in the ConfD case it represents the configuration of a single device, i.e. the device using ConfD as management framework, whereas in the NSO case, it represents the entire network configuration, we can generate the list of Cisco commands that would take us from one XML tree to another.
This technology is used by NSO to generate CLI commands southbound when we manage Cisco like devices.
It will become clear later in the examples how the CLI engine is run in forward and also reverse mode. The key point though, is that the Cisco CLI NED Java programmer doesn't have to understand and parse the structure of the CLI, this is entirely done by the NSO CLI engine.
In order to implement a CLI NED, the following components are required:
-
A YANG data model that describes the CLI. An important development tool here is ConfD, the Tail-f on-device management toolkit. For NSO to manage a CLI device, it needs a YANG file with exactly the right annotations to produce precisely the CLI of the managed device. In the NSO example collection we have a few examples of annotated YANG models that render different variants of Cisco CLI. See for example
$NCS_DIR/packages/neds/dell-ftos
and$NCS_DIR/packages/neds/cisco-nx
.Thus, to create an annotated YANG files for a device with a Cisco-like CLI, the work procedure is thus to run ConfD and write a YANG file which renders the correct CLI. This procedure is well described in the ConfD user guide documentation.
Furthermore, this YANG model must declare an identity with ned:cli-ned-id as base.
-
The next thing we need is a Java class that implements the NED. This is typically not a lot of code, and the existing example NED Java classes are easily extended and modified to fit other needs. The most important point of Java NED class code though is that the code can be oblivious of the actual CLI commands sent and received.
Java CLI NED code must implement the CliNed interface.
/* -*- Java -*- * * Copyright 2010 Tail-F Systems AB. All rights reserved. * * This software is the confidential and proprietary * information of Tail-F Systems AB. * * $Id$ * */ package com.tailf.ned; import java.io.InputStream; import java.util.Calendar; import javax.xml.XMLConstants; import javax.xml.parsers.DocumentBuilder; import javax.xml.parsers.DocumentBuilderFactory; import org.apache.logging.log4j.Logger; import org.apache.logging.log4j.LogManager; import org.w3c.dom.Document; import org.xml.sax.InputSource; import com.tailf.conf.ConfPath; import com.tailf.conf.ConfXMLParam; import com.tailf.maapi.Maapi; import com.tailf.maapi.MaapiSchemas.CSSchema; import com.tailf.ned.NedWorker.TransactionIdMode; /** * A NedConnection is the interface used by the NedMux for keeping * track of connections to different devices. One instance of each * type should be registered with the NedMux before the NedMux is * started. Specific sub-classes are defined for cli and generic * neds, see NedCli and NedGeneric. * * The life of a specific connection to a backend device is as * follows. * * 1 Initially an instance is created through the invocation of * the create method, and a connection to the backend * device is set up. * 2 a mix of prepare/abort/revert/commit/persist/show/getTransId/ * showStatsPath, etc * 3 possibly get invocations to the isAlive() method. This method * is involved when the connection is pooled. * 4 possibly get an invocation to the reconnect method, and start * over at 2 * 5 finally one of the close() methods will be involved. The * connection should close the connection to the device and * release all resources. * * If the connection is poolable then it may live in the connection * pool. The state of the connection is polled by the connection * pool manager using the isAlive() method. * */ abstract public class NedConnectionBase { private static Logger LOGGER = LogManager.getLogger( NedConnectionBase.class); private NedCapability[] capas; private NedCapability[] statscapas; private boolean wantRevertDiff; private TransactionIdMode transMode = TransactionIdMode.NONE; private ConfXMLParam[] platformData; private int connectionId; private long poolTimestamp = Long.MAX_VALUE; private boolean useStoredCapas = false; // Holds reference to the new style SSH connection protected SSHClient sshClient = null; protected void setPoolTimestamp(long timestamp) { this.poolTimestamp = timestamp; } public long getTimeInPool() { return Calendar.getInstance().getTimeInMillis() - poolTimestamp; } /** * This function is used to set the parameters of NedConnection for * a specific NED. * * @param capas * an array of capabilities for config data * @param statscapas * an array of capabilities for stats data * @param wantRevertDiff * Indicates if the NED should be provided with the edit operations * needed to undo the configuration changes done in * the prepare method when a transaction is aborted. * @param transMode * Indicates the mode of Transaction ID supported by the NED. * NONE if not supported. If supported, then getTransId() * should be implemented. Support for Transaction IDs is required * for check-sync action. */ public void setConnectionData(NedCapability[] capas, NedCapability[] statscapas, boolean wantRevertDiff, TransactionIdMode transMode) { this.capas = capas; this.statscapas = statscapas; this.wantRevertDiff = wantRevertDiff; this.transMode = transMode; } /** * This function is used to set the capabilities for a specific NED. * It has the same functionality as setConnectionData, but only for * config capabilities. This is useful when initializing a NED instance * without establishing connection to the device because other connection * parameters such as stats capabilities, reverse diff and * transaction id mode are irrelevant in this case * * @param capas * an array of capabilities for config data */ public void setCapabilities(NedCapability[] capas) { this.capas = capas; } /** * This function is used to set the same capabilities as stored in * CDB for a particular device. This method can only be used when * initializing a NED instance without establishing connection to * the device. */ public void useStoredCapabilities() { this.useStoredCapas = true; } /** * This function is used to set the platform operational data for * a specific NED. This is optional data that can be retrieved and * used for instance in service code. * * It is possible to augment NED specific data into the platform container * in the NCS device model. This method is then used to set both standard * and augmented data. * * The ConfXMLParam[] array is expected to start with the platform tag: * * The following is an example of how the platformData array would * be structured in an example with both the NCS standard name, * model and version leaves as well as a augmented inventory list * with three list elements: * <pre> * ConfXMLParam[] platformData = * new ConfXMLParam[] { * new ConfXMLParamStart("ncs", "platform"), * new ConfXMLParamValue("ncs", "name", * new ConfBuf("ios")), * new ConfXMLParamValue("ncs", "version", * new ConfBuf("15.0M")), * new ConfXMLParamValue("ncs", "model", * new ConfBuf("7200")), * * new ConfXMLParamStart("ginv", "inventory"), * new ConfXMLParamValue("ginv", "name", * new ConfBuf("lx-345")), * new ConfXMLParamValue("ginv", "value", * new ConfBuf("line-card")), * new ConfXMLParamStop("ginv", "inventory"), * new ConfXMLParamStart("ginv", "inventory"), * new ConfXMLParamValue("ginv", nameStr, * new ConfBuf("lx-1001")), * new ConfXMLParamValue("ginv", "value", * new ConfBuf("line-card")), * new ConfXMLParamStop("ginv", "inventory"), * new ConfXMLParamStart("ginv", "inventory"), * new ConfXMLParamValue("ginv", "name", * new ConfBuf("FA1209A4E389")), * new ConfXMLParamValue("ginv", "value", * new ConfBuf("licence")), * new ConfXMLParamStop("ginv", "inventory"), * * new ConfXMLParamStop("ncs", "platform") * }; * * setPlatformData(platformData); * * </pre> * * @param platformData * An ConfXMLParam array containing operational data to be set under * the platform container in the device model. Expected to start with * the platform tag. If the platform container is augmented with some user * specific model such data should also be part of this array to be set at * connection time. */ public void setPlatformData(ConfXMLParam[] platformData) { this.platformData = platformData; } public int connection_id() { return getConnectionId(); } protected void setConnectionId(int connectionId) { this.connectionId = connectionId; } public int getConnectionId() { return connectionId; } public NedCapability[] getCapas() { return capas; } public NedCapability[] getStatsCapas() { return statscapas; } public boolean getWantRevertDiff() { return wantRevertDiff; } public TransactionIdMode getTransactionIdMode() { return transMode; } public ConfXMLParam[] getSystemStateData() { return platformData; } public boolean getUseStoredCapas() { return useStoredCapas; } /** * The device_id is originally provided by NCS to properly identify * the device. It is the name used for the device by NCS in the * list of devices. */ abstract public String device_id(); /** * The type is one of "cli" and "generic". This information is sent to * NCS when the NedMux is started to let NCS know how to communicate * with each device. */ abstract public String type(); /** * Which YANG modules are covered by the class instance. This information * is defined by the setConnectionData() call and is sent to NCS after * initiating a new connection, or when re-establishing a connection. * The modules() method is not actually used. */ abstract public String [] modules(); /** * This should return the a unique (among registered NedConnection classes) * identity. It will be used by NCS when creating new connections to * control which of the registered NedConnection classes to use. */ public String identity() { // obsolete since NSO 4.7 return null; } /** * This indicates that the current set of operations should be * committed to the running configuration. When completed the * w.commitResponse() method should be invoked. Devices that does * not support commit() should invoke the w.commitResponse() method * without delay. On error invoke the w.error(NedCmd.COMMIT, * Error, Reason) method. * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. * @param timeout * If the commit operation does not complete within 'timeout' seconds * the operation should be aborted. */ abstract public void commit(NedWorker w, int timeout) throws Exception; /** * This method is invoked when the currently committed change set * should be made permanent. This corresponds to copying the * running configuration to the startup configuration, on a * running/startup device, or issuing the confirming commit operation * on a device that supports that. When completed the * w.persistResponse() should be invoked. On error invoke the * w.error(NedCmd.PERSIST,Error, Reason) method. * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. */ abstract public void persist(NedWorker w) throws Exception; /** * This method is invoked when the connection is terminated. It is * not invoked when placing the connection in the connection pool. * No response is required, but trace messages may be generated * during the close down. * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. */ abstract public void close(NedWorker w) throws Exception; /** * This method is invoked when a connection close is forced and no * NedWorker is involved. This typically occurs when a connection * is removed from the connection pool. No response or trace * messages can be sent during the operation. */ abstract public void close(); /** * @deprecated Use the method * {@link #isAlive(NedWorker)} * instead. * * If the {@link #isAlive(NedWorker)} * method is implemented in the NED, this method will not be * invoked. The NED must implement one of these methods. * */ @Deprecated public boolean isAlive() { return false; } /** * This method is invoked to check if a connection is still * alive. When a connection is stored in the connection pool * it will periodically be polled to see if it is alive. If * false is returned the connection will be closed using the * close() method invocation. * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. */ public boolean isAlive(NedWorker w) { return this.isAlive(); } protected void isSessionAlive(NedWorker w) throws Exception { boolean alive = false; try { alive = this.isAlive(w); } catch (Exception e) { alive = false; } w.isAliveResponse(alive); } /** * This method is invoked periodically to keep an connection * alive. If false is returned the connection will be closed using the * close() method invocation. * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. */ public boolean keepAlive(NedWorker w) { return true; } protected void keepSessionAlive(NedWorker w) throws Exception { boolean alive = true; try { alive = this.keepAlive(w); } catch (Exception e) { alive = false; } w.isAliveResponse(alive); } /** * This is for any optional commands on the device that are * not part of the yang files config data, but is modeled as * tailf:actions or rpcs in the device yang files. * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. * @param cmdName * Name of the command (path to action?) * @param params * */ abstract public void command(NedWorker w, String cmdName, ConfXMLParam[] params) throws Exception; /** * This method is deprecated. New implementations should implement * showStatsPath() and announce * http://tail-f.com/ns/ncs-ned/show-stats-path capability. * * When this method is invoked the NED should write the value * of the leaf, indicated by the path, to the provided transaction * handler (th) using MAAPI. The worker is free to * write other values as well, for example, if a larger chunk of * data is read from the backend device. NCS will then not * request those paths but read them directly from the transaction. * Different timeouts can be provided for different paths. Note * however, that all data under a subtree will be removed from the * cache once the timeout triggers. * * The only drawback of this approach is that there is no way * for the NED to indicate that a leaf does not exist other than * not providing the value when directly asked for its path. * Consequently, if a NED writes a larger set of data then the path * requested in the method, NCS may still ask for leafs in * that data-set if no value has been provided for them (due to * them not existing). * * This method will not invoked if the NED announced capability * http://tail-f.com/ns/ncs-ned/show-stats-path * * The method should indicate its return status by invoking * the method w.error() or w.showStatsResponse() * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. * @param th * a transaction handler that can be used in Maapi * @param path * a ConfPath indication which element is requested * */ @Deprecated public void showStats(NedWorker w, int th, ConfPath path) throws Exception { w.error(NedCmd.SHOW_STATS, "not implemented"); } /** * This method is deprecated. New implementations should implement * showStatsPath() and announce * http://tail-f.com/ns/ncs-ned/show-stats-path capability. * * When this method is invoked the NED should write the full * list to the transaction (th), but it is not required to write * any data for any of the the instances. It may, however, choose to write * more than the requested list, if that is convenient. Often * a larger chunk of data is provided by the backend and the NED * may then write that to the cache (the transaction th provided). * * Note that even if the data for a list is written during another * invocation, NCS will still invoke the showStatsList() method * to make sure that the list instances are written in full. * The NED may then just respond with a new timeout and does not * have to write the data again to the transaction (but it is * a good idea to make sure the data is still stored in the * cache (transaction th) using MAAPI. * * This method will not invoked if the NED announced capability * http://tail-f.com/ns/ncs-ned/show-stats-path * * The method should indicate its return status by invoking * the method w.error() or w.showStatsResponse() * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. * @param th * a transaction handler that can be used in Maapi * @param path * a ConfPath indication which list is requested * */ @Deprecated public void showStatsList(NedWorker w, int th, ConfPath path) throws Exception { w.error(NedCmd.SHOW_STATS_LIST, "not implemented"); } /** * When this method is invoked depending on the node type the NED should: * * If the path points to the list node or leaf-list node without * specifying the key, then the NED should populate the list keys. * The NED should also set the TTL on the list node or individual list * instances. It may choose to write more data into the list instances * in which case it may populate TTL values for this data as well. * * If the path indicates a list entry, presence container, empty leaf or * a leaf-list instance, then the NED should indicate the existence of * this node in the data tree and return the corresponding TTL value. It * may populate more data into the list instance or presence container in * which case it may populate TTL values for this data as well. * * If the path points to a leaf, then the NED should write the value of * the leaf and indicate its TTL. * * If the NED chooses to populate the entire subtree below the path and * has nothing more to fetch, it should indicate so in the TTL value. * The TTL value for this path will act as the default TTL. * * This method will only be invoked if the NED announced capability * http://tail-f.com/ns/ncs-ned/show-stats-path * * The abovementioned operations should be performed on the provided * transaction th. * * The method should indicate its return status by invoking * the method w.error() or w.showStatsPathResponse() * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. * @param th * a transaction handler that can be used in Maapi * @param path * a ConfPath indication which list is requested * */ public void showStatsPath(NedWorker w, int th, ConfPath path) throws Exception { w.error(NedCmd.SHOW_STATS_PATH, "not implemented"); } /** * When this method is invoked the NED should produce a transaction * id that must be changed if any changes has been made to the * configuration since the last time the transaction id was requested. * The transaction id can either be requested from the system, * or calculated by the callback, for example by calculating an * MD5 checksum of the configuration text. * * The method should indicate its return status by invoking * the method w.error() or w.getTransIdResponse() * * The method should be implemented if the NED claimed * a NedWorker.TransactionIdMode which is not NONE * in setConnectionData(). * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. */ abstract public void getTransId(NedWorker w) throws Exception; /** * Used for resuming a connection found in the connection pool. * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. */ abstract public void reconnect(NedWorker w) throws Exception; /** * Used for initializing an transaction. For instance if locking * or other transaction preparations are necessary, * they should be performed here. * Note, that this method has a proper implementation in the base class * and is therefore not necessary to override if no NED specific * transaction preparations are needed. * * The method should indicate its return status by invoking * the method w.error() or w.initializeResponse() * * If the NED has claimed a NedWorker.TransactionIdMode other than * not NONE a transaction id must be produced in the response same * as for the getTransId() call. * * @param w * The NedWorker instance currently responsible for driving the * communication * between NCS and the device. This NedWorker instance should be * used when communicating with NCS, ie for sending responses, * errors, and trace messages. It is also implements the NedTracer * API and can be used in, for example, the SSHSession as a tracer. */ public void initialize(NedWorker w) throws Exception { if (transMode == TransactionIdMode.NONE || w.isSuppressTransId()) { w.initializeResponse(""); } else { getTransId(w); } } /** * If the transaction is not completed and the NED has done initialize * this method is called to undo the transaction preparations. * That is restoring the NED to the state before initialize. * Note, that this method has a proper implementation in the base class * and is therefore not necessary to override if no NED specific operations * was performed in initialize. * * The method should indicate its return status by invoking * the method w.error() or w.uninitializeResponse() * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. */ public void uninitialize(NedWorker w) throws Exception { w.uninitializeResponse(); } /** * This method is invoked to create a notification subscription. * After the subscription has been created the NedWorker can * send notifiction messages, in the NETCONF notification format, * with the w.notification() method. * * The method should indicate its return status by invoking * the method w.error() or w.createSubscriptionResponse() * * @param w * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker * instance should be used when communicating with NCS, i.e, * for sending responses, errors, and trace messages. It is also * implements the NedTracer API and can be used in, for example, * the SSHSession as a tracer. * @param stream * The notification stream to establish the subscription on. * @param startTime * Trigger the replay feature and indicate that the replay should * start at the time specified. If null, this is not a replay * subscription. It is not valid to specify start times that are * later than the current time. If the time specified is * earlier than the log can support, the replay will begin with * the earliest available notification. This parameter is of * dateTime XML schema type and compliant to RFC 3339. * Implementations must support time zones. * @param filter * Indicates which subset of all possible events is of interest. * The format of this parameter is the same as that of the filter * parameter in the NETCONF protocol operations. If not present, * all events not precluded by other parameters will be sent. * @param filterType * Indicates the type of filter if it is used: * <ul> * <li> {@link com.tailf.ned.NedCmd#FILTER_NONE} * <li> {@link com.tailf.ned.NedCmd#FILTER_XPATH} * <li> {@link com.tailf.ned.NedCmd#FILTER_SUBTREE} * </ul> * * @see <a href="https://tools.ietf.org/html/rfc3339">RFC 3339</a> * @see <a href="https://tools.ietf.org/html/rfc5277">RFC 5277</a> * @see <a href="https://www.w3.org/TR/xmlschema-2/"> * XSD-TYPES: XML Schema Part 2: Datatypes Second Edition * </a> */ public void createSubscription(NedWorker w, String stream, String startTime, String filter, int filterType) throws Exception { w.error(NedCmd.CREATE_SUBSCRIPTION, "not implemented"); } static protected String retrieveIdentity(NedConnectionBase ned) throws NedException { InputStream stream = ned.getClass().getClassLoader(). getResourceAsStream("package-meta-data.xml"); if (stream == null) { // backward compatibility with old neds. String nedName = ned.getClass().getName(); LOGGER.warn("Ned '" + nedName + "' do not contain the package-meta-data.xml in any " + "of its private jar files"); return ned.identity(); } // The Xalan/Xerces libraries have their own impl for this factory // and it doesn't support disabling external DTD/schema. // So, we have to force usage of JDK's built-in implementation here DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance( "com.sun.org.apache.xerces.internal.jaxp." + "DocumentBuilderFactoryImpl", NedConnectionBase.class.getClassLoader()); dbf.setAttribute(XMLConstants.ACCESS_EXTERNAL_DTD, ""); // SQ Rule java:S2755 dbf.setAttribute(XMLConstants.ACCESS_EXTERNAL_SCHEMA, ""); // SQ Rule java:S2755 dbf.setNamespaceAware(true); DocumentBuilder db; try { db = dbf.newDocumentBuilder(); InputSource src = new InputSource(stream); Document doc = db.parse(src); org.w3c.dom.NodeList nlist = doc.getElementsByTagName("ned-id"); org.w3c.dom.Node node = nlist.item(0); String[] sarr1 = node.getTextContent().split(":"); String lprefix = sarr1[0].trim(); String nedid = sarr1[1].trim(); String attrname = "xmlns:"+lprefix; org.w3c.dom.Node nsnode = node.getAttributes().getNamedItem(attrname); String uri = nsnode.getNodeValue().trim(); CSSchema schema = Maapi.getSchemas().findCSSchema(uri); String prefix = schema.getPrefix(); String identity = prefix+":"+nedid; return identity; } catch (Exception e) { String nedName = ned.getClass().getName(); throw new NedException(NedErrorCode.CONNECT_BADKEY, "Could not retrieve ned-id from package-meta-data.xml " + "for ned '" + nedName + "'.", e); } } }
/* -*- Java -*- * * Copyright 2010 Tail-F Systems AB. All rights reserved. * * This software is the confidential and proprietary * information of Tail-F Systems AB. * * $Id$ * */ package com.tailf.ned; import java.net.InetAddress; import com.tailf.conf.ConfPath; /** * This class is used for connections between the NCS and CLI based * NEDs. A NedCli instance must be combined with a YANG data model * that models the data and CLI commands used for talking to the device. */ abstract public class NedCliBase extends NedConnectionBase { // mangle output, we're invoked during prepare phase // of NCS /** * Is invoked by NCS to take the configuration to a new state. The Ned may * choose to apply the changes directly to the device, preferably to a * candidate * configuration, but if the device lacks candidate support it may choose * to apply the changes directly to the running config (typically the case * on IOS like boxes). If the configuration changes are later aborted, or * reverted, the NCS will provide the necessary commands for restoring the * configuration to its previous state. The device should invoke the * w.prepareResponse() when the operation is completed. * * initialize (prepare transaction) * / \ * / uninitialize (undo preparations) * v * prepare (send data to device) * / \ * v v * abort | commit(send confirmed commit (ios would do noop)) * / \ * v v * revert | persist (send confirming commit) * * @param w * The NedWorker instance currently responsible for driving the * communication * between NCS and the device. This NedWorker instance should be * used when communicating with the NCS, ie for sending responses, * errors, and trace messages. It is also implements the NedTracer * API and can be used in, for example, the SSHSession as a tracer. * * @param data * is the CLI commands for transforming the configuration to * a new state. The commands are generated using the YANG data * model in combination with the tailf: extensions to guide the * mapping. */ abstract public void prepare(NedWorker w, String data) throws Exception; /** * Is invoked by NCS to tell the NED what actions it should take towards * the device if it should do a prepare. * * The NED should invoke the method * {@link com.tailf.ned.NedWorker#prepareDryResponse(String) * prepareDryResponse()} * when the operation is completed. If no changes needs to be done * just answer <code>prepareDryResponse(data)</code> * * If an error is detected answer this through a call to * {@link com.tailf.ned.NedWorker#error(int,String,String) error()} * in <code>NedWorker w</code>. * * @param w * The NedWorker instance currently responsible for driving the * communication * between NCS and the device. This NedWorker instance should be * used when communicating with the NCS, ie for sending responses, * errors, and trace messages. It is also implements the * {@link NedTracer} * API and can be used in, for example, the {@link SSHSession} * as a tracer. * * @param data * is the CLI commands for transforming the configuration to * a new state. The commands are generated using the YANG data * model in combination with the tailf: extensions to guide the * mapping. */ abstract public void prepareDry(NedWorker w, String data) throws Exception; /** * Is invoked by NCS to abort the configuration to the state before the * previous prepare() invocation. The NCS has calculated the commands needed * to reach that state from the current state. The instance may choose * to use there commands, or use some other mechanism to reach the same * state. When the operation is completed it should invoke the * w.abortResponse() method in the NedWorker. * * @param w * The NedWorker instance currently responsible for driving the * communication * between NCS and the device. This NedWorker instance should be * used when communicating with the NCS, ie for sending responses, * errors, and trace messages. It is also implements the NedTracer * API and can be used in, for example, the SSHSession as a tracer. * * @param data * is the commands for taking the config back to the previous * state. The commands are generated using the YANG data * model in combination with the tailf: extensions to guide the * mapping. */ abstract public void abort(NedWorker w, String data) throws Exception; /** * Is invoked by NCS to undo the changes introduced in the last commit * operation (communicated to the NED in the prepare method invocation). * The difference between abort() and revert() is that revert() is invoked * after commit() (but before persist), whereas abort() is invoked before * commit(). Once the configuration has been made persistent by persist() * it can no longer be restored to any previous (potentially saved) state. * When the revert operation has been completed the w.revertResponse() * method should be called. * * @param w * The NedWorker instance currently responsible for driving the * communication * between NCS and the device. This NedWorker instance should be * used when communicating with the NCS, ie for sending responses, * errors, and trace messages. It is also implements the NedTracer * API and can be used in, for example, the SSHSession as a tracer. * * @param data * is the commands for taking the config back to the previous * state. */ abstract public void revert(NedWorker w, String data) throws Exception; /** * Extract parts of the configuration and send it to NCS. The response * is sent by invoking the w.showCliResponse() method in the provided * NedWorker. * * @param w * The NedWorker instance currently responsible for driving the * communication * between NCS and the device. This NedWorker instance should be * used when communicating with the NCS, ie for sending responses, * errors, and trace messages. It is also implements the NedTracer * API and can be used in, for example, the SSHSession as a tracer. * * @param toptag * is the top level tag indicating which part of the config * should be extracted. */ abstract public void show(NedWorker w, String toptag) throws Exception; /** * @deprecated Use the method * {@link #showPartial(NedWorker,ConfPath[],String[])} * instead. * * Extract parts of the configuration and send it to NCS. The response * is sent by invoking the w.showCliResponse() method in the provided * NedWorker. * * @param w * The NedWorker instance currently responsible for driving the * communication * between NCS and the device. This NedWorker instance should be * used when communicating with the NCS, ie for sending responses, * errors, and trace messages. It is also implements the NedTracer * API and can be used in, for example, the SSHSession as a tracer. * * @param cmdpaths * are cmd paths to filter the various parts of the configuration tree * that should be extracted. */ @Deprecated public void showPartial(NedWorker w, String[] cmdpaths) throws Exception { w.error(NedCmd.SHOW_PARTIAL_CLI, "not implemented"); } /** * Extract parts of the configuration and send it to NCS. The response * is sent by invoking the w.showCliResponse() method in the provided * NedWorker. * * @param w * The NedWorker instance currently responsible for driving the * communication * between NCS and the device. This NedWorker instance should be * used when communicating with the NCS, ie for sending responses, * errors, and trace messages. It is also implements the NedTracer * API and can be used in, for example, the SSHSession as a tracer. * * @param paths * are paths to filter the various parts of the configuration tree * that should be extracted. */ public void showPartial(NedWorker w, ConfPath[] paths) throws Exception { w.error(NedCmd.SHOW_PARTIAL_CLI, "not implemented"); } /** * Extract parts of the configuration and send it to NCS. The response * is sent by invoking the w.showCliResponse() method in the provided * NedWorker. * * @param w * The NedWorker instance currently responsible for driving the * communication * between NCS and the device. This NedWorker instance should be * used when communicating with the NCS, ie for sending responses, * errors, and trace messages. It is also implements the NedTracer * API and can be used in, for example, the SSHSession as a tracer. * * @param paths * are paths to filter the various parts of the configuration tree * that should be extracted. * * @param cmdpaths * are cmd paths to filter the various parts of the configuration tree * that should be extracted. */ public void showPartial(NedWorker w, ConfPath[] paths, String[] cmdpaths) throws Exception { throw new NedWorker.NotImplementedException(); } /** * Extract parts of the configuration and send it to NCS. The response * is sent by invoking the w.showCliResponse() method in the provided * NedWorker. * * @param w * The NedWorker instance currently responsible for driving the * communication * between NCS and the device. This NedWorker instance should be * used when communicating with the NCS, ie for sending responses, * errors, and trace messages. It is also implements the NedTracer * API and can be used in, for example, the SSHSession as a tracer. * * @param toptag * is the top level tag indicating which part of the config * should be extracted. * * @param data * is the CLI commands in native format. */ public void showOffline(NedWorker w, String toptag, String data) throws Exception { w.error(NedCmd.SHOW_OFFLINE_CLI, "not implemented"); } /** * Used by the connection pool to find a matching connection. If * the current connection is has the same parameters it should return * true, otherwise false. * * @param deviceId name of device * @param ip address to connect to device * @param port port to connect to * @param proto ssh or telnet * @param ruser name of user to connect as * @param pass password to use when connecting * @param secpass secondary password to use when entering config mode, * set to empty string if not configured in the authgroup * @param trace indicates if raw trace messages should be generated or not * @param connectTimeout in milliseconds * @param readTimeout in milliseconds * @param writeTimeout in milliseconds */ abstract public boolean isConnection(String deviceId, InetAddress ip, int port, String proto, // ssh or telnet String ruser, String pass, String secpass, String keydir, boolean trace, int connectTimeout, // msecs int readTimeout, // msecs int writeTimeout // msecs ); /** * Establish a new connection to a device and send response to * NCS with information about the device. This information is set by * using the setConnectionData() method. * A new instance representing the new connection should * be returned. That instance will then be used for further communication * with the device. Different worker instances may be used for that * communication and the instance cannot assume that the worker used in * this invocation will be the same used for the invocations of the * prepare, abort, revert, persist, show, etc methods. * * @return the connection instance * @param deviceId name of device * @param ip address to connect to device * @param port port to connect to * @param proto ssh or telnet * @param ruser name of user to connect as * @param pass password to use when connecting * @param secpass secondary password to use when entering config mode, * set to empty string if not configured in the authgroup * @param publicKeyDir directory to read public keys. null if password is * given * @param trace indicates if raw trace messages should be generated or not * @param connectTimeout in milliseconds * @param readTimeout in milliseconds * @param writeTimeout in milliseconds * @param mux * @param w * The NedWorker instance currently responsible for driving the * communication * between NCS and the device. This NedWorker instance should be * used when communicating with the NCS, ie for sending responses, * errors, and trace messages. It is also implements the NedTracer * API and can be used in, for example, the SSHSession as a tracer. */ abstract public NedCliBase newConnection(String deviceId, InetAddress ip, int port, String proto, // ssh or telnet String ruser, String pass, String secpass, String publicKeyDir, boolean trace, int connectTimeout, // msecs int readTimeout, // msecs int writeTimeout, // msecs NedMux mux, NedWorker w ); /** * Make a new instance of Ned object without establishing a connection * towards the device. The NED should use previously stored information * about the device to initialize its state if needed or throw * NedWorker.NotEnoughDataException. It should then send the information * about the device using the setConnectionData() method. A new instance * representing the new connection should be returned. That instance can * only be used for invoking prepareDry() callback and not for actual * communication with the device. close() callback will be invoked * before destroying the NED instance, so the implementation of the close() * callback should handle cleanup both for instances created with * newConnection() and instanced created with initNoConnect() * * @return the NED instance * @param device_id name of device * @param mux * @param worker * The NedWorker instance currently responsible for driving the * communication between NCS and the device. This NedWorker instance * should be used when communicating with the NCS, ie for sending * responses, errors, and trace messages. */ public NedCliBase initNoConnect(String device_id, NedMux mux, NedWorker worker) throws NedWorker.NotEnoughDataException { throw new NedWorker.NotEnoughDataException(); } }
Thus the Java NED class has the following responsibilities.
-
It must implement the identification callbacks, i.e modules(), type(), and identity()
-
It must implement the connection related callback methods newConnection(), isConnection() and reconnect()
NSO will invoke the newConnection() when it requires a connection to a managed device. It is the responsibility of the newConnection() method to connect to the device, figure out exactly what type of device it is and return an array of NedCapability objects.
public class NedCapability { public String str; public String uri; public String module; public String features; public String revision; public String deviations; ....
This is very much in line with how a NETCONF connect works and how the NETCONF client and server exchange hello messages.
-
Finally the NED code must implement a series of data methods. For example the method void prepare(NedWorker w, String data) get a String object which is the set of Cisco CLI commands it shall send to the device.
In the other direction, when NSO wants to collect data from the device, it will invoke void show(NedWorker w, String toptag) for each tag found at the top of data model(s) loaded for that device. For example if the NED gets invoked with show(w, "interface") it's responsibility is to invoke the relevant show configuration command for "interface", i.e show running-config interface over the connection to the device, and then dumbly reply with all data the device replies with. NSO will parse the output data and feed it into its internal XML trees.
NSO can order the
showPartial()
to collect part of the data if the NED announces the capability http://tail-f.com/ns/ncs-ned/show-partial?path-format=FORMAT in which FORMAT is of the followings:-
key-path: support regular instance keypath format
-
top-tag: support top tags under the /devices/device/config tree
-
cmd-path-full: support Cisco's CLI edit path with instances
-
path-modes-only: support Cisco CLI mode path
-
cmd-path-modes-only-existing: same as
path-mode-only
but NSO only supplies the path mode of existing nodes.
-
As described in previous sections, the CLI NEDs are almost programming free. The NSO CLI engine takes care of parsing the stream of characters that come from "show running-config [toptag]" and also automatically produce the sequence of CLI commands required to take the system from one state to another.
A generic NED is required when we want to manage a device that neither speaks NETCONF or SNMP, nor can be modeled so that ConfD - loaded with those models - gets a CLI that looks almost/exactly like the CLI of the managed device. For example devices that have other proprietary CLIs, devices that can only be configured over other protocols such as REST, Corba, XML-RPC, SOAP, other proprietary XML solutions, etc.
In a manner similar to the CLI NED, the Generic NED needs to be able to connect to the device, return the capabilities, perform changes to the device and finally, grab the entire configuration of the device.
The interface that a Generic NED has to implement is very similar to the interface of a CLI NED. The main differences are:
-
When NSO has calculated a diff for a specific managed device, it will for CLI NEDS also calculate the exact set of CLI commands to send to the device, according to the YANG models loaded for the device. In the case of a generic NED, NSO will instead send an array of operations to perform towards the device in the form of DOM manipulations. The generic NED class will receive an array of NedEditOp objects. Each NedEditOp object contains:
-
The operation to perform, i.e CREATED, DELETED, VALUE_SET etc.
-
The keypath to the object in case.
-
An optional value
-
-
When NSO wants to sync the configuration from the device to NSO, the CLI NED only has to issue a series of "show running-config [toptag]" commands and reply with the output received from the device. A generic NED has to do more work. It is given a transaction handler, which it must attach to over the Maapi interface. Then the NED code must - by some means - retrieve the entire configuration and write into the supplied transaction, again using the Maapi interface.
Once the generic NED is implemented, all other functions in NSO work precisely in same manner as with NETCONF and CLI NED devices. NSO still has the capability to run network wide transactions. The caveat is that to abort a transaction towards a device that doesn't support transactions, we calculate the reverse diff and send it to the device, i.e. we automatically calculate the undo operations.
Another complication with generic NEDs is how the NED class shall authenticate towards the managed device. This depends entirely on the protocol between the NED class and the managed device. If SSH is used to a proprietary CLI, the existing authgroup structure in NSO can be used as is. However, if some other authentication data is needed, it is up the generic NED implementer to augment the authgroups in tailf-ncs.yang accordingly.
We must also configure a managed device, indicating that it's configuration is handled by a specific generic NED. Below we see that the NED with identity "xmlrpc" is handling this device.
admin@ncs# show running-config devices device x1
address 127.0.0.1
port 12023
authgroup default
device-type generic ned-id xmlrpc
state admin-state unlocked
...
The example examples.ncs/generic-ned/xmlrpc-device
in
the NSO examples collection implements a generic NED that speaks
XML-RPC to 3 HTTP servers. The HTTP servers run the apache XML-RPC server
code and the NED code manipulates the 3 HTTP servers using a number
of predefined XML RPC calls.
A good starting point when we wish to implement a new generic NED
is the ncs-make-package --generic-ned-skeleton ...
command, that be used to generate a skeleton package for a generic
NED.
$ ncs-make-package --generic-ned-skeleton abc --build
$ ncs-setup --ned-package abc --dest ncs
$ cd ncs
$ ncs -c ncs.conf
$ ncs_cli -C -u admin
admin@ncs# show packages package abc
packages package abc
package-version 1.0
description "Skeleton for a generic NED"
ncs-min-version [ 3.3 ]
component MyDevice
callback java-class-name [ com.example.abc.abcNed ]
ned generic ned-id abc
ned device vendor "Acme abc"
...
oper-status up
NSO ships with several CLI NED examples. A good starting point is
$NCS_DIR/packages/neds/cisco-ios
which contains
that allows NSO to control Cisco IOS/Catalyst routers.
Implementing a CLI NED is almost entirely a YANG model activity. The tool to use while developing the YANG model is ConfD. The task is to write a YANG model, that when run with ConfD, make ConfD produce a CLI that is as close as possible to the target device, in this case a Cisco IOS router.
The ConfD example found under
$CONFD_DIR/examples.confd/cli/c7200
doesn't cover the entire Cisco c7200 router. It only covers certain
aspects of the device. This is important, in order to have NSO manage
a device with a Cisco like CLI we do not have to model the entire device,
we only need to cover the commands that we intend to use. When the
show() callback issues its
"show running-config [toptag]" command, and the device replies with data
that is fed to NSO, NSO will ignore all command dump output that is not
covered by the loaded YANG models.
Thus, whichever Cisco like device we wish to manage, we must first have YANG models that cover all aspects of the device we wish to use from NSO. Tailf ships various YANG models covering different variants of Cisco routers and switches in the NSO example collection. Either of these is a good starting point. Once we have a YANG model, we load it into NSO, modify the example CLI NED class to return the NedCapability list of the device.
The NED code gets to see all data that goes from and to the device. If it's impossible or too hard to get the YANG model exactly right for all commands, a last resort is to let the NED code modify the data inline. Hopefully this shall never be necessary.
NSO can order the showPartial()
to collect
part of the data if the NED announces the capability
http://tail-f.com/ns/ncs-ned/show-partial?path-format=key-path
A generic NED always requires more work than a CLI NED. The generic NED needs to know how to map arrays of NedEditOp objects into the equivalent reconfiguration operations on the device. Depending on the protocol and configuration capabilities of the device, this may be arbitrarily difficult.
Regardless of the device, we must always write a YANG model that describes the device. The array of NedEditOp objects that the generic NED code gets exposed to is relative the YANG model that we have written for the device. Again, this model doesn't necessarily have to cover all aspects of the device.
Often a useful technique with generic NEDs can be to write a pyang plugin to generate code for the generic NED. Again, depending on the device it may be possible to generate Java code from a pyang plugin that covers most or all aspects of mapping an array of NedEditOp objects into the equivalent reconfiguration commands for the device.
Pyang is an extensible and open source YANG parser (written by Tail-f)
available
at http://www.yang-central.org
. pyang is also part of the NSO
release. A number of plugins are shipped in the NSO release, for example
$NCS_DIR/lib/pyang/pyang/plugins/tree.py
is a good
plugin to start with if we wish to
write our own plugin.
$NCS_DIR/examples.ncs/generic-ned/xmlrpc-device
is a
good example to start with if we wish to write a generic NED. It
manages a set of devices over the XML-RPC protocol.
In this example we have:
-
Defined a fictious YANG model for the device.
-
Implemented an XML-RPC server exporting a set of RPCs to manipulate that fictious data model. The XML-RPC server runs the apache
org.apache.xmlrpc.server.XmlRpcServer
Java package. -
Implemented a Generic NED which acts as an XML-RPC client speaking HTTP to the XML-RPC servers.
The example is self contained, and we can, using the NED code, manipulate these XML-RPC servers in a manner similar to all other managed devices.
$ cd $NCS_DIR/generic-ned/xmlrpc-device
$ make all start
$ ncs_cli -C -u admin
admin@ncs# devices sync-from
sync-result {
device r1
result true
}
sync-result {
device r2
result true
}
sync-result {
device r3
result true
}
admin@ncs# show running-config devices r1 config
ios:interface eth0
macaddr 84:2b:2b:9e:af:0a
ipv4-address 192.168.1.129
ipv4-mask 255.255.255.0
status Up
mtu 1500
alias 0
ipv4-address 192.168.1.130
ipv4-mask 255.255.255.0
!
alias 1
ipv4-address 192.168.1.131
ipv4-mask 255.255.255.0
!
speed 100
txqueuelen 1000
!
As it was mentioned earlier the NedEditOp objects are relative to the YANG model of the device, and they are to be translated into the equivalent reconfiguration operations on the device. Applying reconfiguration operations may only be valid in a certain order.
For Generic NEDs NSO provides a feature to ensure dependency rules being obeyed when generating a diff to commit. It controls the order of operations delivered in the NedEditOp array. The feature is activated by adding the following option to package-meta-data.xml:
<option> <name>ordered-diff</name> </option>
When the ordered-diff flag is set the NedEditOp objects follow YANG schema order and consider dependencies between leaf nodes. Dependencies can be defined using leafrefs and the tailf:cli-diff-after, tailf:cli-diff-create-after, tailf:cli-diff-modify-after, tailf:cli-diff-set-after, tailf:cli-diff-delete-after YANG extensions. Read more about the above YANG extensions in the Tail-f CLI YANG extensions man page.
A device we wish to manage using a NED usually has a not just configuration data that we wish to manipulate from NSO, but the device usually has a set of commands that do not relate to configuration.
The commands on the device we wish to be able to invoke from NSO must be modelled as actions. We model this as actions, and compile it using a special ncsc command to compile NED data models that do not directly relate to configuration data on the device.
The NSO example
$NCS_DIR/examples.ncs/generic-ned/xmlrpc-device
contains an example where the managed device, a fictious XML-RPC
device contains a YANG snippet :
container commands { tailf:action idle-timeout { tailf:actionpoint ncsinternal { tailf:internal; } input { leaf time { type int32; } } output { leaf result { type string; } } } }
When that action YANG is imported into NSO it ends up under the managed device. We can invoke the action on the device as :
admin@ncs# devices device r1 config ios:commands idle-timeout time 55
result OK
The NED code is obviously involved here. All NEDs must always implement:
void command(NedWorker w, String cmdName, ConfXMLParam[] params) throws NedException, IOException;
The command() method gets invoked in the NED, the code must then execute the command. The input parameters in the params parameter correspond to the data provided in the action. The command() method must reply with another array of ConfXMLParam objects.
public void command(NedWorker worker, String cmdname, ConfXMLParam[] p) throws NedException, IOException { session.setTracer(worker); if (cmdname.compareTo("idle-timeout") == 0) { worker.commandResponse(new ConfXMLParam[]{ new ConfXMLParamValue(new interfaces(), "result", new ConfBuf("OK")) }); }
The above code is fake, on a real device, the job of the command() method is to establish a connection to the device, invoke the command, parse the output and finally reply with an ConfXMLParam array.
The purpose of implementing NED commands is usually that we want to expose device commands to the programmatic APIs in the NSO DOM tree.
NED devices have runtime data, statistics. The first part in
being able to collect non-configuration data from a NED
device is to model the statistics data we wish to gather.
In normal YANG files, it is common to have the runtime data
nested inside the configuration data. In gathering runtime data for
NED devices we have chosen to separate configuration data and
runtime data. In the case of the archetypical CLI device, the
show running-config ...
and friends are used to
display the running configuration of the device whereas other different
show ...
commands are used to display
runtime data, for example show interfaces
,
show routes
. Different commands for different
types of routers/switches and in particular, different tabular
output format for different device types.
To expose runtime data from a NED controlled device, regardless of whether it's a CLI NED or a Generic NED, we need to do two things:
-
Write YANG models for the aspects of runtime data we wish to expose northbound in NSO.
-
Write Java NED code that is responsible for collecting that data.
The NSO NED for the Avaya 4k device contains a data model for some real statistics for the Avaya router and also the accompanying Java NED code. Let's start to take a look at the YANG model for the stats portion, we have:
module tailf-ned-avaya-4k-stats { namespace 'http://tail-f.com/ned/avaya-4k-stats'; prefix avaya4k-stats; import tailf-common { prefix tailf; } import ietf-inet-types { prefix inet; } import ietf-yang-types { prefix yang; } container stats { config false; container interface { list gigabitEthernet { key "num port"; tailf:cli-key-format "$1/$2"; leaf num { type uint16; } leaf port { type uint16; } leaf in-packets-per-second { type uint64; } leaf out-packets-per-second { type uint64; } leaf in-octets-per-second { type uint64; } leaf out-octets-per-second { type uint64; } leaf in-octets { type uint64; } leaf out-octets { type uint64; } leaf in-packets { type uint64; } leaf out-packets { type uint64; } } } } }
It's a config false; list of
counters per interface. We compile the NED stats module
with the --ncs-compile-module
flag
or with the
--ncs-compile-bundle
flag.
It's the same non-config module that contains
both runtime data as well as commands and rpcs.
$ ncsc --ncs-compile-module avaya4k-stats.yang \ --ncs-device-dir <dir>
The config false; data from a module
that has been compiled with the
--ncs-compile-module
flag will end
up mounted under /devices/device/live-status
tree. Thus running
the NED towards a real router we have:
admin@ncs# show devices device r1 live-status interfaces
live-status {
interface gigabitEthernet1/1 {
in-packets-per-second 234;
out-packets-per-second 177;
in-octets-per-second 4567;
out-octets-per-second 3561;
in-octets 12666;
out-octets 16888;
in-packets 7892;
out-packets 2892;
}
............
It is the responsibility of the NED code to populate the data in the live device tree. Whenever a northbound agent tries to read any data in the live device tree for a NED device, the NED code is invoked.
The NED code implements an interface called,
NedConnection
This interface contains:
void showStatsPath(NedWorker w, int th, ConfPath path) throws NedException, IOException;
This interface method is invoked by NSO in the NED.
The NED should indicate that it supports the new interface by
announcing http://tail-f.com/ns/ncs-ned/show-stats-path
capability. If the capability is not announced, then NSO will
try to use the deprecated showStats() and showStatsList() APIs. The
Java code must return what is requested, but it may also return more.
The Java code always needs to signal errors by invoking
NedWorker.error()
and success by invoking
NedWorker.showStatsPathResponse()
. The
latter function indicates what is returned, and also how
long it shall be cached inside NSO.
The reason for this design, is that it is common for
many show
commands to work on for example
an entire interface, or some other item in the managed device.
Say that the NSO operator (or maapi code) invokes:
admin@host> show status devices device r1 live-status \ interface gigabitEthernet1/1/1 out-octets out-octets 340;
requesting a single leaf, the NED Java code can decide to
execute any arbitrary show
command towards
the managed device, parse the output and populate as much
data as it wants. The Java code also decides how long time
NSO shall cache the data.
-
When the
showStatsPath()
is invoked, the NED should indicate the state/value of the node indicated by the path (i.e. if a leaf was requested, the NED should write the value of this leaf to the provided transaction handler (th) using MAAPI, or indicate its absence as described below; if a list entry or a presence container was requested then the NED should indicate presence or absence of the element, if the whole list is requested then the NED should populate the keys for this list). Often requesting such data from the actual device will give the NED more data than specifically requested, in which case the worker is free to write other values as well. The NED is not limited to populating the subtree indicated by the path, it may also write values outside this subtree. NSO will then not request those paths but read them directly from the transaction. Different timeouts can be provided for different paths.If a leaf does not have a value, or does not exist, the NED can indicate this by returning a TTL for the path to the leaf, without setting the value in the provided transaction. This has changed from earlier versions of NSO. The same applies for optional containers and list entries. If the NED populates the keys for a certain list (both when it is requested to do so or when it decided to do so because it has received this data from the device), it should set the TTL value for the list itself to indicate the time the set of keys should be considered up to date. It may choose to provide different TTL values for some or all list entries, but it is not required to do so.
Sometimes we wish to use a different protocol to collect statistics from the live tree than the protocol that is used to configure a managed device. There are many interesting use cases where this pattern applies. For example, if we wish to access SNMP data as statistics in the live tree on a Juniper router, or alternatively if we have a CLI NED to a Cisco type device, and wish to access statistics in the live tree over SNMP.
The solution is to configure additional protocols for the live tree. We can have an arbitrary number of NEDs associated to statistics data for an individual managed device.
The additional NEDs are configured under
/devices/device/live-status-protocol
.
In the configuration snippet below, we have configured two additional NEDs for statistics data.
devices { authgroups { snmp-group g1 { umap admin { community-name public; } } } mib-group m1 { mib-module [ SIMPLE-MIB ]; } device device0 { live-status-protocol x1 { port 4001; device-type { snmp { version v2c; snmp-authgroup g1; mib-group [ m1 ]; } } } live-status-protocol x2 { authgroup default; device-type { cli { ned-id xstats; } } } }
One important task when implementing a NED of any type is to make it mimic the devices handling of default values as close as possible. Network equipment can typically deal with default values in many different ways.
Some devices display default values on leafs even if they have not been explicitly set. Others use trimming, meaning that if a leaf is set to its default value it will be 'unset' and disappear from the devices configuration dump.
It is the responsibility of the NED to make the NSO aware of how the device handles default values. This is done by registering a special NED Capability entry with the NSO. Two modes are currently supported by the NSO: "trim" and "report-all".
This is the typical behavior of a Cisco IOS device. The simple YANG code snippet below illustrates the behavior. A container with a boolean leaf. Its default value is true.
container aaa { leaf enabled { default true; type boolean; } }
Try setting the leaf to true in NSO and commit. Then compare the configuration:
$ ncs_cli -C -u admin
admin@ncs# config
admin@ncs(config)# devices device a0 config aaa enabled true
admin@ncs(config)# commit
Commit complete.
admin@ncs(config)# top devices device a0 compare-config
diff
devices {
device a0 {
config {
aaa {
- enabled;
}
}
}
}
The result shows that the configurations differ. The reason is that the device does not display the value of the leaf 'enabled'. It has been trimmed since it has its default value. The NSO is now out of sync with the device.
To solve this issue, make the NED tell the NSO that the device is trimming default values. Register an extra NED Capability entry in the Java code.
NedCapability capas[] = new NedCapability[2]; capas[0] = new NedCapability( "", "urn:ios", "tailf-ned-cisco-ios", "", "2015-01-01", ""); capas[1] = new NedCapability( "urn:ietf:params:netconf:capability:" + "with-defaults:1.0?basic-mode=trim", // Set mode to trim "urn:ietf:params:netconf:capability:" + "with-defaults:1.0", "", "", "", "");
Now, try the same operation again:
$ ncs_cli -C -u admin
admin@ncs# config
admin@ncs(config)# devices device a0 config aaa enabled true
admin@ncs(config)# commit
Commit complete.
admin@ncs(config)# top devices device a0 compare-config
admin@ncs(config)#
The NSO is now in sync with the device.
Some devices display default values for leafs even if they have not been explicitly set. The simple YANG code below will be used to illustrate this behavior. A list containing a key and a leaf with a default value.
list interface { key id; leaf id { type string; } leaf treshold { default 20; type uint8; } }
Try creating a new list entry in NSO and commit. Then compare the configuration:
$ ncs_cli -C -u admin
admin@ncs# config
admin@ncs(config)# devices device a0 config interface myinterface
admin@ncs(config)# commit
admin@ncs(config)# top devices device a0 compare-config
diff
devices {
device a0 {
config {
interface myinterface {
+ treshold 20;
}
}
}
}
The result shows that the configurations differ. The NSO is out of sync. This is because the device displays the default value of the 'threshold' leaf even if it has not been explicitly set through the NSO.
To solve this issue, make the NED tell the NSO that the device is reporting all default values. Register an extra NED Capability entry in the Java code.
NedCapability capas[] = new NedCapability[2]; capas[0] = new NedCapability( "", "urn:abc", "tailf-ned-abc", "", "2015-01-01", ""); capas[1] = new NedCapability( "urn:ietf:params:netconf:capability:" + "with-defaults:1.0?basic-mode=report-all", // Set mode to report-all "urn:ietf:params:netconf:capability:" + "with-defaults:1.0", "", "", "", "");
Now, try the same operation again:
$ ncs_cli -C -u admin
admin@ncs# config
admin@ncs(config)# devices device a0 config interface myinterface
admin@ncs(config)# commit
Commit complete.
admin@ncs(config)# top devices device a0 compare-config
admin@ncs(config)#
The NSO is now in sync with the device.
When implementing a NED it sometimes happens that the device has a really tricky behavior regarding how different parts of the configuration are related to each other. This is typically so complex making it impossible to model it in YANG.
Examples of such are:
-
A device that alters unrelated configuration. For instance if a value of leaf A is changed through NSO the device will also automatically modify the value of leaf B.
leaf A { type uint8; } leaf B { type uint8; }
-
A device that creates additional configuration. For instance if a new entry in list A is created through NSO the device will also automatically create en entry in the sub list B.
list A { key a; leaf a { type uint8; } list B { key b; leaf b { type uint8; } } }
Both these cases will result in out of sync issues in the NSO.
One fairly straightforward way to solve this is by using set hooks in the NED. A set hook is a callback routine in Java that is mapped to something in the YANG model. This can for instance be a certain leaf or list in the model. The set hook can be configured to be called upon different operations. Typically this involves create, set or delete operations.
Assume a device that creates additional configuration as described above. The YANG code snippet below will be used to illustrate this.
list A { key a; leaf a { type string; } list B { key b; leaf b { type string; } } }
Try creating a new list entry in NSO and commit. Then compare the configuration:
$ ncs_cli -C -u admin
admin@ncs# config
admin@ncs(config)# devices device a0 config A mylist
admin@ncs(config)# commit
admin@ncs(config)# commit dry-run outformat cli
cli devices {
device a0 {
config {
+ A mylist {
+ }
}
}
}
Commit complete.
admin@ncs(config)# top devices device a0 compare-config
diff
devices {
device a0 {
config {
A mylist {
+ B default {
+ }
}
}
}
}
The device has automatically created the sub list 'default' when it created the list 'mylist'. The result is that NSO is now out of sync with the device.
The solution is to implement a set hook in the NED that makes the NSO mimic the device properly. In this case it shall create an entry named 'default' in the sub list B each time it creates an entry in list A.
The Java implementation of the set hook would look something like this:
public class XYZDp { @Resource(type=ResourceType.MAAPI, scope=Scope.INSTANCE) public Maapi mm; /* * Set hook. * Called when a new entry in the A list is created. * The callpoint to be mapped in the YANG model is * "list-a-create-hook" */ @DataCallback(callPoint="list-a-create-hook", callType=DataCBType.CREATE) public int createSublistBEntry(DpTrans trans, ConfObject[] keyPath) throws DpCallbackException { try { int tid = trans.getTransaction(); ConfPath cp = new ConfPath(keyPath); // Create a sublist entry <path>/B{"default"} cp.append("/B{default}"); mm.safeCreate(tid, cp); return Conf.REPLY_OK; } catch (Exception e) { throw new DpCallbackException("", e); } } // Init routine @TransCallback(callType=TransCBType.INIT) public void XYZDpInit(DpTrans trans) throws DpCallbackException { try { if (mm == null) { // Need a Maapi socket so that we can attach Socket s = new Socket("127.0.0.1", NcsMain.getInstance(). getNcsPort()); mm = new Maapi(s); } mm.attach(trans.getTransaction(),0, trans.getUserInfo().getUserId()); return; } catch (Exception e) { throw new DpCallbackException("Failed to attach", e); } } // Finish routine @TransCallback(callType=TransCBType.FINISH) public void XYZDpFinish(DpTrans trans) throws DpCallbackException { try { mm.detach(trans.getTransaction()); } catch (Exception e) { ; } } }
Finally the YANG model is extended with an extra annotation:
list A { tailf:callpoint list-a-create-hook { tailf:set-hook node; } key a; leaf a { type string; } list B { key b; leaf b { type string; } } }
Now, try the same operation again. Create a new list entry in NSO and commit. Then compare the configuration:
$ ncs_cli -C -u admin
admin@ncs# config
admin@ncs(config)# devices device a0 config A mylist
admin@ncs(config)# commit
admin@ncs(config)# commit dry-run outformat cli
cli devices {
device a0 {
config {
+ A mylist {
+ B default;
+ }
}
}
}
Commit complete.
admin@ncs(config)# top devices device a0 compare-config
admin@ncs(config)#
The NSO has now automatically created the "default" entry in sub list B the same way as the device does. The NSO will now be in sync with the device.
The possibility to do a dry-run on a transaction is a feature in NSO that allows to examine the changes to be pushed out to the managed devices in the network. The output can be produced in different formats, namely cli, xml and native. In order to produce dry-run in the native output format NSO needs to know the exact syntax used by the device, and the task of converting the commands or operations produced by the NSO into the device-specific output belongs the corresponding NED. This is the purpose of the prepareDry() callback in the NED interface.
In order to be able to invoke a callback an instance of the NED object needs to be created first. There are two ways to instantiate a NED:
-
newConnection()
callback that tells the NED to establish connection to the device which can later be used to perform any action such as show configuration, apply changes or view operational data as well as produce dry-run output. -
Optional
initNoConnect()
callback that tells the NED to create an instance that would not need to communicate with the device, and hence must not establish a connection or otherwise communicate with the device. This instance will only be used to calculate dry-run output. It is possible for a NED to reject the initNoConnect() request if it is not able to calculate the dry-run output without establishing a connection to the device, for example if a NED is capable of managing devices with different flavors of syntax and it is not known at the moment which syntax is used by this particular device.
The following state diagram displays NED states specific to dry-run scenario.

NED dry-run states
NED packages should follow some naming conventions.
A package is a directory where the package name is the same as
the directory name. At the toplevel of this directory a file called
package-meta-data.xml
must exist.
The package name in that file should follow
<vendor>-<ned_name>-<ned_version>
for example, cisco-iosxr-cli-7.29
.
A package may also be a tar archive with the same directory
layout. The tar archive can be either uncompressed with suffix
.tar
, or gzip-compressed with suffix
.tar.gz
or .tgz
.
The archive file should also follow some naming conventions, it should be named by
ncs-<ncs_version>-<vendor>-<ned_name>-<ned_version>.<suffix>
,
for example, ncs-5.4-cisco-iosxr-7.29.1.tar.gz
The NED name is expected to be two words (no dashes within the words)
separated by a dash, for example, cisco-iosxr
.
It may also include NED type at the end,
for example, cisco-iosxr_netconf
.
The YANG modeling language supports a notion of a module
revision
. It allows users to distinguish between
different versions of a module, so the module can evolve over
time. If you wish to use a new revision of a module for a
managed device, for example to access new features, you
generally need to create a new NED.
When a model evolves quickly and you have many devices that require the use of a lot of different revisions, you will need to maintain a high number of NEDs, which are mostly the same. This can become especially burdensome during NSO version upgrades, when all NEDs may need to be recompiled.
When a YANG module is only updated in a backwards compatible way (following the upgrade rules in RFC6020 or RFC7950), the NSO compiler, ncsc, allows you to pack multiple module revisions into the same package. This way, a single NED with multiple device model revisions can be used, instead of multiple NEDs. Based on the capabilities exchange, NSO will then use the correct revision for communication with each device.
However, there is a major downside to this approach. While the exact revision is known for each communication session with the managed device, the device model in NSO does not have that information. For that reason, the device model always uses the latest revision. When pushing configuration to a device that only supports an older revision, NSO silently drops the unsupported parts. This may have surprising results, as the NSO copy can contain configuration that is not really supported on the device. Use the no-revision-drop commit parameter when you want to make sure you are not committing config that is not supported by a device.
If you still wish to use this functionality, you can create a NED
package with the ncs-make-package --netconf-ned
command as you would otherwise. But the supplied source YANG
directory should contain YANG modules with different revisions.
The files should follow the
naming convention, as specified in the RFC6020.
Some versions of the compiler require you to use the
module-or-submodule-name
@revision-date
.yang--no-fail-on-warnings
option with the
ncs-make-package command or the build process
may fail.
The examples.ncs/development-guide/ned-upgrade/yang-revision
example shows how you can perform a YANG model upgrade. The original,
1.0 version of the router NED uses the
router@2020-02-27.yang
YANG model. First, it
is updated to the version 1.0.1
router@2020-09-18.yang
using a revision merge
approach. This is possible because the changes are backward-compatible.
In the second part of the example, the updates in
router@2022-01-25.yang
introduce breaking
changes, therefore the version is increased to 1.1 and a different
ned-id is assigned to the NED. In this case, you can't use revision
merge and the usual NED migration procedure is required.