Implementing Services

In NSO, the term service has a special meaning and represents an automation construct that orchestrates create, modify, and delete of a service instance into the resulting native commands to devices in the network. In its simplest form, a service takes some input parameters and maps them to device-specific configurations. It is a recipe, or a set of instructions.

Much like you can bake many cakes using a single cake recipe, you can create many service instances using the same service. But unlike cakes, having the recipe produce exactly the same output, is not very useful. That is why service instances define a set of input parameters, which the service uses to customize the produced configuration.

A network engineer on the CLI, or an API call from a northbound system, provides the values for input parameters when requesting a new service instance, and NSO uses the “service recipe”, called a service mapping, to configure the network.

Figure 7. A high-level view of services in NSO

A similar process takes place when deleting the service instance or modifying the input parameters. The main task of a service is therefore: from a given set of input parameters, calculate the minimal set of device operations to achieve the desired service change. Here, it is very important that the service supports any change; create, delete, and update of any service parameter.

Device configuration is usually the primary goal of a service. However, there may be other supporting functions that are expected from the service, such as service-specific actions. The complete service application, implementing all the service functionality, is packaged in an NSO service package.

The following definitions are used throughout this section:

Developing a service that transforms a service instance request to the relevant device configurations is done differently in NSO than in most other tools on the market. As a service developer, you create a mapping from a YANG service model to the corresponding device YANG model.

This is a declarative, model-to-model mapping. Irrespective of the underlying device type and its native device interface, the mapping is towards a YANG device model and not the native CLI (or any other protocol/API). As you write the service mapping, you do not have to worry about the syntax of different CLI commands or in which order these commands are sent to the device. It is all taken care of by the NSO device manager and device NEDs. Implementing a service in NSO is reduced to transforming the input data structure, described in YANG, to device data structures, also described in YANG.

Who writes the models?

A service application then has two primary artifacts: a YANG service model and a mapping definition to the device YANG, as illustrated in the following figure.

Figure 8. Service model and mapping

To reiterate:

This approach may seem somewhat unorthodox at first but allows NSO to streamline and greatly simplify how you implement services.

A common problem for traditional automation systems is that a set of instructions needs to be defined for every possible service instance change. Take for example a VPN service. During a service life cycle you want to:

The possible run-time changes for an existing service instance are numerous. If a developer must define instructions for every possible change, such as a script or a workflow, the task is daunting, error-prone, and never-ending.

NSO reduces this problem to a single data-mapping definition for the "create" scenario. At run-time, NSO renders the minimum resulting change for any possible change in the service instance. It achieves this with the FASTMAP algorithm.

Another challenge in traditional systems is that a lot of code goes into managing error scenarios. The NSO built-in transaction manager takes that burden away from the developer of the service application by providing automatic rollback of incomplete changes.

Another benefit of this approach is that NSO can automatically generate the northbound APIs and database schema from the YANG models, enabling a true DevOps way of working with service models. A new service model can be defined as part of a package and loaded into NSO. An existing service model can be modified and the package upgraded, and all northbound APIs and User Interfaces are automatically regenerated to reflect the new or updated models.

To demonstrate the simplicity a pure model-to-model service mapping affords, let us consider the most basic approach to providing the mapping: the service XML template. The XML template is an XML-encoded file that tells NSO what configuration to generate when someone requests a new service instance.

The first thing you need is the relevant device configuration (or configurations if multiple devices are involved). Suppose you must configure 192.0.2.1 as a DNS server on the target device. Using the NSO CLI, you first enter the device configuration, then add the DNS server. For a Cisco IOS-based device:

admin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# devices device c1 config
admin@ncs(config-config)# ip name-server 192.0.2.1
admin@ncs(config-config)# top
admin@ncs(config)#

Note here that the configuration is not yet committed and you can use the commit dry-run outformat xml command to produce the configuration in the XML format. This format is an excellent starting point for creating an XML template.

admin@ncs(config)# commit dry-run outformat xml

result-xml {
    local-node {
        data <devices xmlns="http://tail-f.com/ns/ncs">
               <device>
                 <name>c1</name>
                 <config>
                   <ip xmlns="urn:ios">
                     <name-server>192.0.2.1</name-server>
                   </ip>
                 </config>
               </device>
             </devices>
    }
}

The interesting portion is the part between <devices> and </devices> tags.

Another way to get the XML output is to list the existing device configuration in NSO by piping it through the display xml filter:

admin@ncs# show running-config devices device c1 config ip name-server | display xml

<config xmlns="http://tail-f.com/ns/config/1.0">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>c1</name>
      <config>
        <ip xmlns="urn:ios">
          <name-server>192.0.2.1</name-server>
        </ip>
      </config>
    </device>
  </devices>
</config>

If there is a lot of data, it is easy to save the output to a file using the save pipe in the CLI, instead of copying and pasting it by hand:

admin@ncs# show running-config devices device c1 config ip name-server | display xml\
 | save dns-template.xml

The last command saves the configuration for a device in the dns-template.xml file using XML format. To use it in a service, you need a service package.

You create an empty, skeleton service with the ncs-make-package command, such as:

ncs-make-package --build --no-test --service-skeleton template dns

The command generates the minimal files necessary for a service package, here named dns. One of the files is dns/templates/dns-template.xml, which is where the configuration in the XML format goes.

<config-template xmlns="http://tail-f.com/ns/config/1.0"
                 servicepoint="dns">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <!-- ... more statements here ... -->
  </devices>
</config-template>

If you look closely, there is one significant difference from the show running-config output: the template uses the config-template XML root tag, instead of config. This tag also has the servicepoint attribute. Other than that, you can use the XML formatted configuration from the CLI as-is.

Bringing the two XML documents together, gives the final dns/templates/dns-template.xml XML template:

<config-template xmlns="http://tail-f.com/ns/config/1.0"
                 servicepoint="dns">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>c1</name>
      <config>
        <ip xmlns="urn:ios">
          <name-server>192.0.2.1</name-server>
        </ip>
      </config>
    </device>
  </devices>
</config-template>

The service is now ready to use in NSO. Start the examples.ncs/implement-a-service/dns-v1 example to set up a live NSO system with such a service and inspect how it works. Try configuring two different instances of the dns service.

$ cd $NCS_DIR/examples.ncs/implement-a-service/dns-v1
$ make demo

The problem with this service is that it always does the same thing because it always generates exactly the same configuration. It would be much better if the service could configure different devices. The updated version, v1.1, uses a slightly modified template:

<config-template xmlns="http://tail-f.com/ns/config/1.0"
                 servicepoint="dns">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>{/name}</name>
      <config>
        <ip xmlns="urn:ios">
          <name-server>192.0.2.1</name-server>
        </ip>
      </config>
    </device>
  </devices>
</config-template>

The changed part is <name>{/name}</name>, which now uses the {/name} code instead of a hard-coded c1 value. The curly braces indicate that NSO should evaluate the enclosed expression and use the resulting value in its place. The /name expression is an XPath expression, referencing the service YANG model. In the model, name is the name you give each service instance. In this case, the instance name doubles for identifying the target device.

admin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# dns c2
admin@ncs(config-dns-c2)# commit dry-run

cli {
    local-node {
        data  devices {
                  device c2 {
                      config {
                          ip {
             +                name-server 192.0.2.1;
                          }
                      }
                  }
              }
             +dns c2 {
             +}
    }
}

In the output, the instance name used was c2 and that is why the service performs DNS configuration for the c2 device.

The template actually allows a decent amount of programmability through XPath and special XML processing instructions. For example:

<config-template xmlns="http://tail-f.com/ns/config/1.0"
                 servicepoint="dns">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>{/name}</name>
      <config>
        <ip xmlns="urn:ios">
          <?if {starts-with(/name, 'c1')}?>
            <name-server>192.0.2.1</name-server>
          <?else?>
            <name-server>192.0.2.2</name-server>
          <?end?>
        </ip>
      </config>
    </device>
  </devices>
</config-template>

In the preceding printout, the XPath starts-with() function is used to check if the device name starts with a specific prefix. Then one set of configuration items is used, and a different one otherwise. For additional available instructions and the complete set of template features, see Templates.

However, most provisioning tasks require some kind of input to be useful. Fortunately, you can define any number of input parameters in the service model that you can then reference from the template; either to use directly in the configuration or as something to base provisioning decisions on.

The YANG service model specifies the input parameters a service in NSO takes. For a specific service model think of the parameters that a northbound system sends to NSO or the parameters that a network engineer needs to enter in the NSO CLI.

Even a service as simple as the DNS configuration service usually needs some parameters, such as the target device. The service model gives each parameter a name and defines validation rules, ensuring the client-provided values fit what the service expects.

Suppose you want to add a parameter for the target device to the simple DNS configuration service. You need to construct an appropriate service model, adding a YANG leaf to capture this input.

The service model is located in the src/yang/servicename.yang file in the package. It typically resembles the following structure:

  list servicename {
    key name;

    uses ncs:service-data;
    ncs:servicepoint "servicename";

    leaf name {
      type string;
    }

    // ... other statements ...
  }

The list named after the package (servicename in the example) is the interesting part.

The uses ncs:service-data and ncs:servicepoint statements differentiate this list from any standard YANG list and make it a service. Each list item in NSO represents a service instance of this type.

The uses ncs:service-data part allows the system to store internal state and provide common service actions, such as re-deploy and get-modifications for each service instance.

The ncs:servicepoint identifies which part of the system is responsible for the service mapping. For a template-only service, it is the XML template which uses the same service point value in the config-template element.

The name leaf serves as the key of the list and is primarily used to distinguish service instances from each other.

The remaining statements describe the functionality and input parameters that are specific to this service. This is where you add the new leaf for the target device parameter of the dns service:

  list dns {
    key name;

    uses ncs:service-data;
    ncs:servicepoint "dns";

    leaf name {
      type string;
    }

    leaf target-device {
      type string;
    }
  }

Use the examples.ncs/implement-a-service/dns-v2 example to explore how this model works and try to discover what deficiencies it may have.

$ cd $NCS_DIR/examples.ncs/implement-a-service/dns-v2
$ make demo

In its current form, the model allows you to specify any value for target-device, including none at all! Obviously, this is not good as it breaks the provisioning of the service. But even more importantly, not validating the input may allow someone to use the service in the way you have not intended and perhaps bring down the network.

You can guard against invalid input with the help of additional YANG statements. For example:

    leaf target-device {
      mandatory true;
      type string {
        length "2";
        pattern "c[0-2]";
      }
    }

Now this parameter is mandatory for every service instance and must be one of the string literals: c0, c1, or c2. This format is defined by the regular expression in the pattern statement. In this particular case, the length restriction is redundant but demonstrates how you can combine multiple restrictions. You can even add multiple pattern statements to handle more complex cases.

What if you wanted to make the DNS server address configurable too? You can add another leaf to the service model:

    leaf dns-server-ip {
      type inet:ipv4-address {
        pattern "192\\.0\\.2\\..*";
      }
    }

There are three notable things about this leaf:

YANG is very powerful and allows you to model all kinds of values and restrictions on the data. In addition to the ones defined in the YANG language (RFC 7950, section 9), predefined types describing common networking concepts, such as those from the inet namespace (RFC 6991), are available to you out of the box. It is much easier to validate the inputs when so many options are supported.

The one missing piece for the service is the XML template. You can take Example 6, “Static DNS configuration template” as a base and tweak it to reference the defined inputs.

Using the code {XYZ} or {/XYZ} in the template, instructs NSO to look for the value in the service instance data, in the node with the name XYZ. So, you can refer to the target-device input parameter as defined in YANG with the {/target-device} code in the XML template.

The final, improved version of the DNS service template that takes into account the new model, is:

<config-template xmlns="http://tail-f.com/ns/config/1.0"
                 servicepoint="dns">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>{/target-device}</name>
      <config>
        <ip xmlns="urn:ios">
          <?if {/dns-server-ip}?>
            <!-- If dns-server-ip is set, use that. -->
            <name-server>{/dns-server-ip}</name-server>
          <?else?>
            <!-- Otherwise, use the default one. -->
            <name-server>192.0.2.1</name-server>
          <?end?>
        </ip>
      </config>
    </device>
  </devices>
</config-template>

The following figure captures the relationship between the YANG model and the XML template that ultimately produces the desired device configuration.

Figure 9. XML template and model relationship

The complete service is available in the examples.ncs/implement-a-service/dns-v2.1 example. Feel free to investigate on your own how it differs from the initial, no-validation service.

$ cd $NCS_DIR/examples.ncs/implement-a-service/dns-v2.1
$ make demo

When the service is simple, constructing the YANG model and creating the service mapping (the XML template) is straightforward. Since the two components are mostly independent, you can start your service design with either one.

If you write the YANG model first, you can load it as a service package into NSO (without having any mapping defined) and iterate on it. This way, you can try the model, which is the interface to the service, with network engineers or northbound systems before investing the time to create the mapping. This model-first approach is also sometimes called top-down.

The alternative is to create the mapping first. Especially for developers new to NSO, the template-first, or bottom-up, approach is often easier to implement. With this approach, you templatize the configuration and extract the required service parameters from the template.

Experienced NSO developers naturally combine the two approaches, without much thinking. However, if you have trouble modeling your service at first, consider following the template-first approach demonstrated here.

For the following example, suppose you want the service to configure IP addressing on an ethernet interface. You know what configuration is required to do this manually for a particular ethernet interface. For a Cisco IOS-based device you would use the commands, such as:

admin@ncs# config
Entering configuration mode terminal
admin@ncs(config)# devices device c1 config
admin@ncs(config-config)# interface GigabitEthernet 0/0
admin@ncs(config-if)# ip address 192.168.5.1 255.255.255.0

To transform this configuration into a reusable service, complete the following steps:

Start by generating the configuration in the XML format, making use of the display xml filter. Note that the XML output will not necessarily be a one-to-one mapping of the CLI commands; the XML reflects the device YANG model which can be more complex but the commands on the CLI can hide some of this complexity.

The transformation to a template also requires you to change the root tag, which produces the resulting XML template:

<config-template xmlns="http://tail-f.com/ns/config/1.0"
                 servicepoint="iface-servicepoint">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>c1</name>
      <config>
        <interface xmlns="urn:ios">
          <GigabitEthernet>
            <name>0/0</name>
            <ip>
              <address>
                <primary>
                  <address>192.168.5.1</address>
                  <mask>255.255.255.0</mask>
                </primary>
              </address>
            </ip>
          </GigabitEthernet>
        </interface>
      </config>
    </device>
  </devices>
</config-template>

However, this template has all the values hard-coded and only configures one specific interface on one specific device.

Now you must replace all the dynamic parts that vary from service instance to service instance with references to the relevant parameters. In this case, it is data specific to each device: which interface and which IP address to use.

Suppose you pick the following names for the variable parameters:

Generally, you can make up any name for a parameter but it is best to follow the same rules that apply for naming variables in programming languages, such as making the name descriptive but not excessively verbose. It is customary to use hyphen (minus sign) to concatenate words and use all-lowercase (“kebab-case”), which is the convention used in the YANG language standards.

Figure 10. Making a configuration template

The corresponding template then becomes:

<config-template xmlns="http://tail-f.com/ns/config/1.0"
                 servicepoint="iface-servicepoint">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>{/device}</name>
      <config>
        <interface xmlns="urn:ios">
          <GigabitEthernet>
            <name>{/interface}</name>
            <ip>
              <address>
                <primary>
                  <address>{/ip-address}</address>
                  <mask>255.255.255.0</mask>
                </primary>
              </address>
            </ip>
          </GigabitEthernet>
        </interface>
      </config>
    </device>
  </devices>
</config-template>

Having completed the template, you can add all the parameters, three in this case, to the service model.

Figure 11. Extracting service model from template in a bottom-up approach

The partially-completed model is now:

  list iface {
    key name;

    uses ncs:service-data;
    ncs:servicepoint "iface-servicepoint";

    leaf name {
      type string;
    }

    leaf device { ... }

    leaf interface { ... }

    leaf ip-address { ... }
  }

Missing are the data type and other validation statements. At this point, you could fill out the model with generic type string statements, akin to the name leaf. This is a useful technique to test out the service in early development. But here you can complete the model directly, as it contains only three parameters.

You can use a leafref type leaf to refer to a device by its name in the NSO. This type uses dynamic lookup at the specified path to enumerate the available values. For the device leaf, it lists every value for a device name that NSO knows about. If there are two devices managed by NSO, named rtr-sjc-01 and rtr-sto-01, either “rtr-sjc-01” or “rtr-sto-01” are valid values for such a leaf. This is a common way to refer to devices in NSO services.

    leaf device {
      mandatory true;
      type leafref {
        path "/ncs:devices/ncs:device/ncs:name";
      }
    }

In a similar fashion restrict the valid values of other two parameters.

    leaf interface {
      mandatory true;
      type string {
        pattern "[0-9]/[0-9]+";
      }
    }

    leaf ip-address {
      mandatory true;
      type inet:ipv4-address;
    }
  }

You would typically create the service package skeleton with the ncs-make-package command and update the model in the .yang file. The model in the skeleton might have some additional example leafs that you do not need and should remove to finalize the model. That gives you the final, full service model:

  list iface {
    key name;

    uses ncs:service-data;
    ncs:servicepoint "iface-servicepoint";

    leaf name {
      type string;
    }

    leaf device {
      mandatory true;
      type leafref {
        path "/ncs:devices/ncs:device/ncs:name";
      }
    }

    leaf interface {
      mandatory true;
      type string {
        pattern "[0-9]/[0-9]+";
      }
    }

    leaf ip-address {
      mandatory true;
      type inet:ipv4-address;
    }
  }

The examples.ncs/implement-a-service/iface-v1 example contains the complete YANG module with this service model in the packages/iface-v1/src/yang/iface.yang file, as well as the corresponding service template in packages/iface-v1/templates/iface-template.xml.

The YANG model and the mapping (the XML template) are the two main components required to implement a service in NSO. The hidden part of the system that makes such an approach feasible is called FASTMAP.

FASTMAP covers the complete service life cycle: creating, changing and deleting the service. It requires the minimal amount of code for mapping from a service model to a device model.

FASTMAP is based on generating changes from an initial create operation. When the service instance is created the reverse of the resulting device configuration is stored together with the service instance. If an NSO user later changes the service instance, NSO first applies (in an isolated transaction) the reverse diff of the service, effectively undoing the previous create operation. Then it runs the logic to create the service again, and finally performs a diff against the current configuration. Only the result of the diff is then sent to the affected devices.

If the service instance is deleted, NSO applies the reverse diff of the service, effectively removing all configuration changes the service did on the devices.

Figure 12. FASTMAP Create a Service

Assume we have a service model that defines a service with attributes X, Y, and Z. The mapping logic calculates that attributes A, B, and C must be set on the devices. When the service is instantiated, the previous values of the corresponding device attributes A, B, and C are stored with the service instance in the CDB. This allows NSO to bring the network back to the state before the service was instantiated.

Now let us see what happens if one service attribute is changed. Perhaps the service attribute Z is changed. NSO will execute the mapping as if the service was created from scratch. The resulting device configurations are then compared with the actual configuration and the minimal diff is sent to the devices. Note that this is managed automatically, there is no code to handle the specific "change Z" operation.

Figure 13. FASTMAP Change a Service

When a user deletes a service instance, NSO retrieves the stored device configuration from the moment before the service was created and reverts to it.

Figure 14. FASTMAP Delete a Service

For a complex service you may realize that the input parameters for a service are not sufficient to render the device configuration. Perhaps the northbound system only provides a subset of the required parameters. For example, the other system wants NSO to pick an IP address and does not pass it as an input parameter. Then additional logic or API calls may be necessary but XML templates provide no such functionality on their own.

The solution is to augment XML templates with custom code. Or, more accurately, create custom provisioning code that leverages XML templates. Alternatively, you can also implement the mapping logic completely in the code and not use templates at all. The latter, forgoing the templates altogether, is less common, since templates have a number of beneficial properties.

Templates separate the way parameters are applied, which depends on the type of target device, from calculating the parameter values. For example, you would use the same code to find the IP address to apply on a device, but the actual configuration might differ whether it is a Cisco IOS (XE) device, an IOS XR, or another vendor entirely.

Moreover, if you use templates, NSO can automatically validate the templates being compatible with the used NEDs, which allows you to sidestep whole groups of bugs.

NSO offers multiple programming languages to implement the code. The --service-skeleton option of the ncs-make-package command influences the selection of the programming language and if the generated code should contain sample calls for applying an XML template.

Suppose you want to extend the template-based ethernet interface addressing service to also allow specifying the netmask. You would like to do this in the more modern, CIDR-based single number format, such as is used in the 192.168.5.1/24 format (the /24 after the address). However, the generated device configuration takes the netmask in the dot-decimal format, such as 255.255.255.0, so the service needs to perform some translation. And that requires custom service code.

Such a service will ultimately contain three parts: the service YANG model, translation code, and the XML template. The model and the template serve the same purpose as before, while custom code provides fine-grained control over how templates are applied and the data available to them.

Figure 15. Code and template service compared to template-only service

Since the service is based on the previous interface addressing service, you can save yourself a lot of work by starting with the existing YANG model and XML template.

The service YANG model needs an additional cidr-netmask leaf to hold the user-provided netmask value:

  list iface {
    key name;

    uses ncs:service-data;
    ncs:servicepoint "iface-servicepoint";

    leaf name {
      type string;
    }

    leaf device {
      mandatory true;
      type leafref {
        path "/ncs:devices/ncs:device/ncs:name";
      }
    }

    leaf interface {
      mandatory true;
      type string {
        pattern "[0-9]/[0-9]+";
      }
    }

    leaf ip-address {
      mandatory true;
      type inet:ipv4-address;
    }

    leaf cidr-netmask {
      default 24;
      type uint8 {
        range "0..32";
      }
    }
  }

This leaf stores a small number (of uint8 type), with values between 0 and 32. It also specifies a default of 24, which is used when the client does not supply a value for this parameter.

The previous XML template also requires only minor tweaks. A small but important change is the removal of the servicepoint attribute on the top element. Since it is gone, NSO does not apply the template directly for each service instance. Instead, your custom code registers itself on this servicepoint and is responsible for applying the template.

The reason for it being this way, is that the code will supply the value for the additional variable, here called NETMASK. This is the other change that is necessary in the template: referencing the NETMASK variable for the netmask value:

<config-template xmlns="http://tail-f.com/ns/config/1.0">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>{/device}</name>
      <config>
        <interface xmlns="urn:ios">
          <GigabitEthernet>
            <name>{/interface}</name>
            <ip>
              <address>
                <primary>
                  <address>{/ip-address}</address>
                  <mask>{$NETMASK}</mask>
                </primary>
              </address>
            </ip>
          </GigabitEthernet>
        </interface>
      </config>
    </device>
  </devices>
</config-template>

Unlike references to other parameters, NETMASK does not represent a data path but a variable. It must start with a dollar character (“$”) to distinguish it from a path. As shown here, variables are often written in all-uppercase, making it easier to quickly tell whether something is a variable or a data path.

Variables get their values from different sources but the most common one is the service code. You implement the service code using a programming language, such as Java or Python.

The following two procedures create an equivalent service that acts identically from a user's perspective. They only differ in the language used; they use the same logic and the same concepts. Still, the final code differs quite a bit due to the nature of each programming language. Generally, you should pick one language and stick with it. If you are unsure which one to pick, you may find Python slightly easier to understand because it is less verbose.

ncs-make-package --no-test --service-skeleton python-and-template iface

    def cb_create(self, tctx, root, service, proplist):
        cidr_mask = service.cidr_netmask

        quad_mask = ipaddress.IPv4Network((0, cidr_mask)).netmask

        vars = ncs.template.Variables()
        vars.add('NETMASK', quad_mask)
        template = ncs.template.Template(service)
        template.apply('iface-template', vars)

    def cb_create(self, tctx, root, service, proplist):
        cidr_mask = service.cidr_netmask

        quad_mask = ipaddress.IPv4Network((0, cidr_mask)).netmask

        vars = ncs.template.Variables()
        vars.add('NETMASK', quad_mask)
        template = ncs.template.Template(service)
        template.apply('iface-template', vars)

ncs-make-package --no-test --service-skeleton java-and-template iface

    public Properties create(ServiceContext context,
                             NavuNode service,
                             NavuNode ncsRoot,
                             Properties opaque)
                             throws ConfException {

        String cidr_mask_str = service.leaf("cidr-netmask").valueAsString();
        int cidr_mask = Integer.parseInt(cidr_mask_str);

        long tmp_mask = 0xffffffffL << (32 - cidr_mask);
        String quad_mask =
            ((tmp_mask >> 24) & 0xff) + "." +
            ((tmp_mask >> 16) & 0xff) + "." +
            ((tmp_mask >> 8) & 0xff) + "." +
            ((tmp_mask >> 0) & 0xff);

        Template myTemplate = new Template(context, "iface-template");
        TemplateVariables myVars = new TemplateVariables();
        myVars.putQuoted("NETMASK", quad_mask);
        myTemplate.apply(service, myVars);

    public Properties create(ServiceContext context,
                             NavuNode service,
                             NavuNode ncsRoot,
                             Properties opaque)
                             throws ConfException {

        try {
            String cidr_mask_str = service.leaf("cidr-netmask").valueAsString();
            int cidr_mask = Integer.parseInt(cidr_mask_str);

            long tmp_mask = 0xffffffffL << (32 - cidr_mask);
            String quad_mask = ((tmp_mask >> 24) & 0xff) +
                "." + ((tmp_mask >> 16) & 0xff) +
                "." + ((tmp_mask >> 8) & 0xff) +
                "." + ((tmp_mask) & 0xff);

            Template myTemplate = new Template(context, "iface-template");
            TemplateVariables myVars = new TemplateVariables();
            myVars.putQuoted("NETMASK", quad_mask);
            myTemplate.apply(service, myVars);
        } catch (Exception e) {
            throw new DpCallbackException(e.getMessage(), e);
        }
        return opaque;
    }

A service instance may require configuration on more than just a single device. In fact, it is quite common for a service to configure multiple devices.

Figure 16. Service provisioning multiple devices

There are a few ways in which you can achieve this for your services:

The generally recommended approach is to use either code with templates or templates with foreach loops. They are explicit and also work well when you configure devices of different types. Using only code extends less well to the latter case, as it requires additional logic and checks for each device type.

Automatic, implicit loops in templates are harder to understand, since the syntax looks like the one for normal leafs. A common example is a device definition as a leaf-list in the service YANG model, such as:

    leaf-list device {
      type leafref {
        path "/ncs:devices/ncs:device/ncs:name";
      }
    }

Because it is a leaf-list, the following template applies to all the selected devices, using an implicit loop:

<config-template xmlns="http://tail-f.com/ns/config/1.0"
                 servicepoint="servicename">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>{/device}</name>
      <config>
        <!-- ... -->
     </config>
    </device>
  </devices>
</config-template>

Is performs the same as the one, which loops through the devices explicitly:

<config-template xmlns="http://tail-f.com/ns/config/1.0"
                 servicepoint="servicename">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <?foreach {/device}?>
      <device>
        <name>{.}</name>
        <config>
          <!-- ... -->
      </config>
      </device>
    <?end?>
  </devices>
</config-template>

Being explicit, the latter is usually much easier to understand and maintain for most developers. The examples.ncs/implement-a-service/dns-v3 demonstrates this syntax in the XML template.

Supporting Different Device Types

Applying the same template works fine as long as you have a uniform network with similar devices. What if two different devices can provide the same service but require different configuration? Should you create two different services in NSO? No. Services allow you to abstract and hide the device specifics through device-independent service model, while still allowing customization of device configuration per device type.

Figure 17. Service provisioning multiple device types

One way to do this is to apply a different XML template from service code, depending on the device type. However, the same is also possible through XML templates alone.

When NSO applies configuration elements in the template, it checks the XML namespaces that are used. If the target device does not support a particular namespace, NSO simply skips that part of the template. Consequently, you can put configuration for different device types in the same XML template and only the relevant parts will be applied.

Consider the following example:

<config-template xmlns="http://tail-f.com/ns/config/1.0">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>{/device}</name>
      <config>
        <!-- Part for device with the cisco-ios NED -->
        <interface xmlns="urn:ios">
          <GigabitEthernet>
            <name>{/interface}</name>
            <!-- ... -->
          </GigabitEthernet>
        </interface>

        <!-- Part for device with the router-nc NED -->
        <sys xmlns="http://example.com/router">
          <interfaces>
            <interface>
              <name>{/interface}</name>
              <!-- ... -->
            </interface>
          </interfaces>
        </sys>
     </config>
    </device>
  </devices>
</config-template>

Due to the xmlns="urn:ios" attribute, the first part of the template (the interface GigabitEthernet) will only apply to Cisco IOS-based device. While the second part (the sys interfaces interface) will only apply to the netsim-based router-nc-type devices, as defined by the xmlns attribute on the sys element.

In case you need to further limit what configuration applies where and namespace-based filtering is too broad, you can also use the if-ned-id XML processing instruction. Each NED package in NSO defines a unique ned-id, which distinguishes between different device types (and possibly firmware versions). Based on the configured ned-id of the device, you can apply different parts of the XML template. For example:

<config-template xmlns="http://tail-f.com/ns/config/1.0">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>{/device}</name>
      <config>
        <?if-ned-id cisco-ios-cli-3.0:cisco-ios-cli-3.0?>
        <interface xmlns="urn:ios">
          <GigabitEthernet>
            <name>{/interface}</name>
            <!-- ... -->
          </GigabitEthernet>
        </interface>
        <?end?>
      </config>
    </device>
  </devices>
</config-template>

The preceding template applies configuration for the interface only if the selected device uses the cisco-ios-cli-3.0 ned-id. You can find the full code as part of the examples.ncs/implement-a-service/iface-v3 example.

In the previous sections we have looked at service mapping when the input parameters are enough to generate the corresponding device configurations. In many situations this is not the case. The service mapping logic may need to reach out to other data in order to generate the device configuration. This is common in the following scenarios:

It is important to design the service model considering the above requirements: what is input and what is available from other sources. In the latter case, in terms of implementation, an important distinction is made between accessing the existing data and allocating new resources. You must take special care for resource allocation, such as VLAN or IP address assignment, as discussed later on. For now, let us focus on using pre-existing shared data.

One example of such use is to define QoS policies "on the side." Only a reference to an existing QoS policy is supplied as input. This is a much better approach than giving all QoS parameters to every service instance. But note that, if you modify the QoS definitions the services are referring to, this will not immediately change the existing deployed service instances. In order to have the service implement the changed policies, you need to perform a re-deploy of the service.

A simpler example is a modified DNS configuration service that allows selecting from a predefined set of DNS servers, instead of supplying the DNS server directly as a service parameter. The main benefit in this case is that clients have no need to be aware of the actual DNS servers (and their IPs). In addition, this approach simplifies the management for the network operator, as all the servers are kept in a single place.

What is required to implement such as service? There are two parts. The first is the model and data that defines the available DNS server options, which are shared (used) across all the DNS service instances. The second is a modification to the service inputs and mapping logic to use this data.

For the first part, you must create a data model. If the shared data is specific to one service type, such as the DNS configuration, you can define it alongside the service instance model, in the service package. But sometimes this data may be shared between multiple types of service. Then it makes more sense to create a separate package for the shared data models.

In this case, define a new top-level container in the service's YANG file as:

  container dns-options {
    list dns-option {
      key name;

      leaf name {
        type string;
      }

      leaf-list servers {
        type inet:ipv4-address;
      }
    }
  }

Note that the container is defined outside the service list because this data is not specific to individual service instances:

  container dns-options {
    // ...
  }

  list dns {
    key name;

    uses ncs:service-data;
    ncs:servicepoint "dns";

    // ...
  }

The dns-options container includes a list of dns-option items. Each item defines a set of DNS servers (leaf-list) and a name for this set.

Once the shared data model is compiled and loaded into NSO, you can define the available DNS server sets:

admin@ncs(config)# dns-options dns-option lon servers 192.0.2.3
admin@ncs(config-dns-option-lon)# top
admin@ncs(config)# dns-options dns-option sto servers 192.0.2.3
admin@ncs(config-dns-option-sto)# top
admin@ncs(config)# dns-options dns-option sjc servers [ 192.0.2.5 192.0.2.6 ]
admin@ncs(config-dns-option-sjc)# commit

You must also update the service instance model to allow clients to pick one of these DNS servers:

  list dns {
    key name;

    uses ncs:service-data;
    ncs:servicepoint "dns";

    leaf name {
      type string;
    }

    leaf target-device {
      type string;
    }

    // Replace the old, explicit IP with a reference to shared data
    // leaf dns-server-ip {
    //   type inet:ip-address {
    //     pattern "192\.0.\.2\..*";
    //   }
    // }
    leaf dns-servers {
      mandatory true;
      type leafref {
        path "/dns-options/dns-option/name";
      }
    }
  }

Different ways exist to model the service input for dns-servers. The first option you might think about might be using a string type and a pattern to limit the inputs to one of lon, sto, or sjc. Another option would be to use a YANG enum type. But both of these have the drawback that you need to change the YANG model if you add or remove available dns-option items.

Using a leafref allows NSO to validate inputs for this leaf by comparing them to the values, returned by the path XPath expression. So, whenever you update the /dns-options/dns-option items, the change is automatically reflected in the valid dns-server values.

At the same time, you must also update the mapping to take advantage of this service input parameter. The service XML template is very similar to the previous one. The main difference is the the way in which the DNS addresses are read from the CDB, using the special deref() XPath function:

<config-template xmlns="http://tail-f.com/ns/config/1.0"
                 servicepoint="dns">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>{/target-device}</name>
      <config>
        <ip xmlns="urn:ios">
          <name-server>{deref(/dns-servers)/../servers}</name-server>
        </ip>
      </config>
    </device>
  </devices>
</config-template>

The deref() function “jumps” to the item selected by the leafref. Here, leafref's path points to /dns-options/dns-option/name, so this is where deref(/dns-servers) ends: at the name leaf of the selected dns-option item.

The following code, which performs the same thing but in a more verbose way, further illustrates how the DNS server value is obtained:

        <ip xmlns="urn:ios">
          <?set dns_option = {/dns-servers}?>   <!-- Set $dns_option to e.g. 'lon' -->
          <?set-root-node {/}?>                 <!-- Make '/' point to datastore root,
                                                     instead of service instance   -->
          <name-server>{/dns-options/dns-option[name=$dns_option]/servers}</name-server>
        </ip>

The complete service is available in the examples.ncs/implement-a-service/dns-v3 example.

NSO provides some service actions out of the box, such as re-deploy or check-sync. You can also add others. A typical use case is to implement some kind of a self-test action that tries to verify the service is operational. The latter could use ping or similar network commands, as well as verify device operational data, such as routing table entries.

This action supplements the built-in check-sync or deep-check-sync action, which checks for the required device configuration.

For example, a DNS configuration service might perform a domain lookup to verify the Domain Name System is working correctly. Likewise, an interface configuration service could ping an IP address or check interface status.

The action consists of the YANG model for action inputs and outputs, as well as the action code that is executed when a client invokes the action.

Typically, such actions are defined per service instance, so you model them under the service list:

  list iface {
    key name;

    uses ncs:service-data;
    ncs:servicepoint "iface-servicepoint";

    leaf name { /* ... */ }
    leaf device { /* ... */ }
    leaf interface { /* ... */ }
    // ... other statements omitted ...

    action test-enabled {
      tailf:actionpoint iface-test-enabled;
      output {
        leaf status {
          type enumeration {
            enum up;
            enum down;
            enum unknown;
          }
        }
      }
    }
  }

The action needs no special inputs; because it is defined on the service instance, it can find the relevant interface to query. The output has a single leaf, called status, that uses an enumeration type for explicitly defining all the possible values it can take (up, down, or unknown).

Note that using the action statement requires you to also use the yang-version 1.1 statement in the YANG module header (see the section called “Actions”).

class IfaceActions(Action):
    @Action.action
    def cb_action(self, uinfo, name, kp, input, output, trans):
        ...

        root = ncs.maagic.get_root(trans)
        service = ncs.maagic.cd(root, kp)

class IfaceActions(Action):
    @Action.action
    def cb_action(self, uinfo, name, kp, input, output, trans):
        root = ncs.maagic.get_root(trans)
        service = ncs.maagic.cd(root, kp)

        device = root.devices.device[service.device]

        status = 'unknown'    # Replace with your own code that checks
                              # e.g. operational status of the interface

        output.status = status

class Main(ncs.application.Application):
    def setup(self):
        ...
        self.register_action('iface-test-enabled', IfaceActions)

    @ActionCallback(callPoint="iface-test-enabled",
                    callType=ActionCBType.ACTION)
    public ConfXMLParam[] test_enabled(DpActionTrans trans, ConfTag name,
                                       ConfObject[] kp, ConfXMLParam[] params)
    throws DpCallbackException {
        // ...
    }

            NavuContext context = new NavuContext(maapi);
            NavuContainer service =
                (NavuContainer)KeyPath2NavuNode.getNode(kp, context);

    @ActionCallback(callPoint="iface-test-enabled",
                    callType=ActionCBType.ACTION)
    public ConfXMLParam[] test_enabled(DpActionTrans trans, ConfTag name,
                                       ConfObject[] kp, ConfXMLParam[] params)
    throws DpCallbackException {
        int port = NcsMain.getInstance().getNcsPort();

        // Ensure socket gets closed on errors, also ending any ongoing
        // session and transaction
        try (Socket socket = new Socket("localhost", port)) {
            Maapi maapi = new Maapi(socket);
            maapi.startUserSession("admin", InetAddress.getByName("localhost"),
                "system", new String[] {}, MaapiUserSessionFlag.PROTO_TCP);

            NavuContext context = new NavuContext(maapi);
            context.startRunningTrans(Conf.MODE_READ);

            NavuContainer root = new NavuContainer(context);
            NavuContainer service =
                (NavuContainer)KeyPath2NavuNode.getNode(kp, context);

            String status = "unknown";    // Replace with your own code that
                                          // checks e.g. operational status of
                                          // the interface

            String nsPrefix = name.getPrefix();
            return new ConfXMLParam[] {
                new ConfXMLParamValue(nsPrefix, "status", new ConfBuf(status)),
            };
        } catch (Exception e) {
            throw new DpCallbackException(name.toString() + " action failed",
                e);
        }
    }

In addition to device configuration, services may also provide operational status or statistics. This is operational data, modeled with config false statements in YANG, and cannot be directly set by clients. Instead, clients can only read this data, for example to check service health.

What kind of data a service exposes depends heavily on what the service does. Perhaps the interface configuration service needs to provide information on whether a network interface was enabled and operational at the time of the last check (because such a check could be expensive).

Taking iface service as a base, consider how you can extend the instance model with another operational leaf to hold the interface status data as of the last check.

  list iface {
    key name;

    uses ncs:service-data;
    ncs:servicepoint "iface-servicepoint";

    // ... other statements omitted ...

    action test-enabled {
      tailf:actionpoint iface-test-enabled;
      output {
        leaf status {
          type enumeration {
            enum up;
            enum down;
            enum unknown;
          }
        }
      }
    }

    leaf last-test-result {
      config false;
      type enumeration {
            enum up;
            enum down;
            enum unknown;
      }
    }
  }

The new leaf last-test-result is designed to store the same data as the test-enabled action returns. Importantly, it also contains a config false substatement, making it operational data.

When faced with duplication of type definitions, as seen in the preceding code, the best practice is to consolidate the definition in a single place and avoid potential discrepancies in the future. You can use a typedef statement to define a custom YANG data type.

  typedef iface-status-type {
    type enumeration {
          enum up;
          enum down;
          enum unknown;
    }
  }

Once defined, you can use the new type as you would any other YANG type. For example:

    leaf last-test-status {
      config false;
      type iface-status-type;
    }

    action test-enabled {
      tailf:actionpoint iface-test-enabled;
      output {
        leaf status {
          type iface-status-type;
      }
    }

Users can then view operational data with the help of the show command. The data is also available through other NB interfaces, such as NETCONF and RESTCONF.

admin@ncs# show iface test-instance1 last-test-status
iface test-instance1 last-test-status up

But where does the operational data come from? The service application code provides this data. In this example, the last-test-status leaf captures the result of the enabled check, which is implemented as a custom action. So, here it is the action code that sets the leaf's value.

This approach works well when operational data is updated based on some event, such as a received notification or a user action, and NSO is used to cache its value.

For cases, where this is insufficient, NSO also allows producing operational data on demand, each time a client requests it, through the Data Provider API. See the section called “DP API” for this alternative approach.

with contextlib.closing(socket.socket()) as s:
    _ncs.cdb.connect(s, _ncs.cdb.DATA_SOCKET, ip='127.0.0.1', port=_ncs.PORT)
    _ncs.cdb.start_session(s, _ncs.cdb.OPERATIONAL)
    _ncs.cdb.set_elem(s, 'up', '/iface{test-instance1}/last-test-status')

with ncs.maapi.single_write_trans('admin', 'python', db=ncs.OPERATIONAL) as t:
    root = ncs.maagic.get_root(t)
    root.iface['test-instance1'].last_test_status = 'up'
    t.apply()

    def cb_action(self, uinfo, name, kp, input, output, trans):
        with ncs.maapi.single_write_trans('admin', 'python',
                                          db=ncs.OPERATIONAL) as t:
            root = ncs.maagic.get_root(t)
            service = ncs.maagic.cd(root, kp)

            # ...
            service.last_test_status = status
            t.apply()

        output.status = status

    leaf last-test-status {
      config false;
      type iface-status-type;
      tailf:cdb-oper {
        tailf:persistent true;
      }
    }

class ServiceApp(Application):
    def setup(self):
        ...
        self.register_fun(init_oper_data, lambda _: None)


def init_oper_data(state):
    state.log.info('Populating operational data')
    with ncs.maapi.single_write_trans('admin', 'python',
                                      db=ncs.OPERATIONAL) as t:
        root = ncs.maagic.get_root(t)
        # ...
        t.apply()

    return state

int port = NcsMain.getInstance().getNcsPort();

// Ensure socket gets closed on errors, also ending any ongoing session/lock
try (Socket socket = new Socket("localhost", port)) {
    Cdb cdb = new Cdb("IfaceServiceOperWrite", socket);
    CdbSession session = cdb.startSession(CdbDBType.CDB_OPERATIONAL);

    String status = "up";
    ConfPath path = new ConfPath("/iface{%s}/last-test-status",
        "test-instance1");
    session.setElem(ConfEnumeration.getEnumByLabel(path, status), path);

    session.endSession();
}

int port = NcsMain.getInstance().getNcsPort();

// Ensure socket gets closed on errors, also ending any ongoing
// session and transaction
try (Socket socket = new Socket("localhost", port)) {
    Maapi maapi = new Maapi(socket);
    maapi.startUserSession("admin", InetAddress.getByName("localhost"),
        "system", new String[] {}, MaapiUserSessionFlag.PROTO_TCP);

    NavuContext context = new NavuContext(maapi);
    context.startOperationalTrans(Conf.MODE_READ_WRITE);

    NavuContainer root = new NavuContainer(context);
    NavuContainer service =
        (NavuContainer)KeyPath2NavuNode.getNode(kp, context);

    // ...
    service.leaf("last-test-status").set(status);
    context.applyClearTrans();
}

    leaf last-check-status {
      config false;
      type iface-status-type;
      tailf:cdb-oper {
        tailf:persistent true;
      }
    }

A FASTMAP service cannot perform explicit function calls with side effects. The only action a service is allowed to take is to modify the configuration of the current transaction. For example, a service may not invoke an action to generate authentication key files or start a virtual machine. All such actions must occur before the service is created and provided as input parameters. This restriction is because the FASTMAP code may be executed as part of a commit dry-run, or the commit may fail, in which case the side effects would have to be undone.

Nano services use a technique called reactive FASTMAP (RFM) and provide a framework to safely execute actions with side effects by implementing the service as several smaller (nano) steps or stages. Reactive FASTMAP can also be implemented directly using the CDB subscribers, but nano services offer a more streamlined and robust approach for staged provisioning.

The services discussed previously in this chapter were modeled to give all required parameters to the service instance. The mapping logic code could immediately do its work. Sometimes this is not possible. Two examples that require staged provisioning where a nano service step executing an action is the best practice solution:

The basic concepts of nano services are covered in detail by Nano Services for Staged Provisioning . The example in examples.ncs/development-guide/nano-services/netsim-sshkey implements SSH public key authentication setup using a nano service. The nano service uses the following steps in a plan that produces the generated, distributed, and configured states:

Upon deletion of the service instance, NSO restores the configuration. The only delete step in the plan is the generated state side-effect action that deletes the key files. The example is described in more detail by Developing and Deploying a Nano Service in Getting Started

The basic-vrouter, netsim-vrouter, and mpls-vpn-vrouter examples in the examples.ncs/development-guide/nano-services directory start, configure, and stop virtual devices. In addition, the mpls-vpn-vrouter example manages Layer3 VPNs in a service provider MPLS network consisting of physical and virtual devices. Using a Network Function Virtualization (NFV) setup, the L3VPN nano service instructs a VM manager nano service to start a virtual device in a multi-step process consisting of the following:

See the mpls-vpn-vrouter example for details on how the l3vpn.yang YANG model l3vpn-plan pe-created state and vm-manager.yang vm-plan for more information. vm-manager plan states with a nano-callback have their callbacks implemented by the escstart.java escstart class. Nano services are documented by Nano Services for Staged Provisioning .

Service troubleshooting is an inevitable part of any NSO development process and eventually a part of their operational tasks as well. By their nature, NSO services are composed primarily out of user-defined code, models, and templates. This gives you plenty of opportunities to make unintended mistakes in mapping code, use incorrect indentations, create invalid configuration templates, and much more. Not only that, they also rely on southbound communication with devices of many different versions and vendors, which presents you with yet another domain that can cause issues in your NSO services.

This is why it is important to have a systematic approach when debugging and troubleshooting your services: