Service Development Using Java

We will first look at the following preparatory steps:

These steps are no different from defining services using templates. Next is to start playing with the Java Environment:

We will start by setting up a run-time environment that includes simulated Cisco IOS devices and configuration data for NSO. Make sure you have sourced the ncsrc file. Create a new directory that will contain the files for this example, such as:

$ mkdir ~/vlan-service
$ cd ~/vlan-service

Now lets create a simulated environment with 3 IOS devices and a NSO that is ready to run with this simulated network:

$ ncs-netsim create-network $NCS_DIR/packages/neds/cisco-ios 3 c
$ ncs-setup --netsim-dir ./netsim/ --dest ./

Start the simulator and NSO:

$ ncs-netsim start
DEVICE c0 OK STARTED
DEVICE c1 OK STARTED
DEVICE c2 OK STARTED
$ ncs

Use the Cisco CLI towards one of the devices:

$ ncs-netsim cli-i c0
admin connected from 127.0.0.1 using console on ncs
c0> enable
c0# configure
Enter configuration commands, one per line. End with CNTL/Z.
c0(config)# show full-configuration
no service pad
no ip domain-lookup
no ip http server
no ip http secure-server
ip routing
ip source-route
ip vrf my-forward
bgp next-hop Loopback 1
!
...

Use the NSO CLI to get the configuration:

$ ncs_cli -C -u admin

admin connected from 127.0.0.1 using console on ncs
admin@ncs# devices sync-from
sync-result {
    device c0
    result true
}
sync-result {
    device c1
    result true
}
sync-result {
    device c2
    result true
}
admin@ncs# config
Entering configuration mode terminal

admin@ncs(config)# show full-configuration devices device c0 config
devices device c0
 config
  no ios:service pad
  ios:ip vrf my-forward
   bgp next-hop Loopback 1
  !
  ios:ip community-list 1 permit
  ios:ip community-list 2 deny
  ios:ip community-list standard s permit
  no ios:ip domain-lookup
  no ios:ip http server
  no ios:ip http secure-server
  ios:ip routing
...

Finally, set VLAN information manually on a device to prepare for the mapping later.

admin@ncs(config)# devices device c0 config ios:vlan 1234 
admin@ncs(config)# devices device c0 config ios:interface
                   FastEthernet 1/0 switchport mode trunk 
admin@ncs(config-if)# switchport trunk allowed vlan 1234 
admin@ncs(config-if)# top 

admin@ncs(config)# show configuration 
devices device c0
 config
  ios:vlan 1234
  !
  ios:interface FastEthernet1/0
   switchport mode trunk
   switchport trunk allowed vlan 1234
  exit
 !
!

admin@ncs(config)# commit 

In the run-time directory you created:

$ ls -F1
README.ncs
README.netsim
logs/
ncs-cdb/
ncs.conf
netsim/
packages/
scripts/
state/

Note the packages directory, cd to it:

$ cd packages
$ ls -l
total 8
cisco-ios -> .../packages/neds/cisco-ios

Currently there is only one package, the Cisco IOS NED. We will now create a new package that will contain the VLAN service.

$ ncs-make-package --service-skeleton java vlan
$ ls
cisco-ios vlan

This creates a package with the following structure:

Figure 28. Package Structure

During the rest of this section we will work with the vlan/src/yang/vlan.yang and vlan/src/java/src/com/example/vlan/vlanRFS.java files.

So, if a user wants to create a new VLAN in the network what should the parameters be? Edit the vlan/src/yang/vlan.yang according to below:

  augment /ncs:services {
    list vlan {
      key name;

      uses ncs:service-data;
      ncs:servicepoint "vlan-servicepoint";
      leaf name {
        type string;
      }

      leaf vlan-id {
        type uint32 {
          range "1..4096";
        }
      }

      list device-if {
        key "device-name";
          leaf device-name {
            type leafref {
              path "/ncs:devices/ncs:device/ncs:name";
            }
          }
          leaf interface {
            type string;
          }
      }
    }
  }

This simple VLAN service model says:

The vlan service list is augmented into the services tree in NSO. This specifies the path to reach vlans in the CLI, REST etc. There is no requirements on where the service shall be added into ncs, if you want vlans to be at the top-level, just remove the augments statement.

Make sure you keep the lines generated by the ncs-make-package:

uses ncs:service-data;
ncs:servicepoint "vlan-servicepoint";

The two lines tell NSO that this is a service. The first line expands to a YANG structure that is shared amongst all services. The second line connects the service to the Java callback.

To build this service model cd to packages/vlan/src and type make (assumes you have the prerequisite make build system installed).

$ cd packages/vlan/src/
$ make

We can now test the service model by requesting NSO to reload all packages:

$ ncs_cli -C -U admin
admin@ncs# packages reload
>>> System upgrade is starting.
>>> Sessions in configure mode must exit to operational mode.
>>> No configuration changes can be performed until upgrade has completed.
>>> System upgrade has completed successfully.
result Done

You can also stop and start NSO, but then you have to pass the option --with-package-reload when starting NSO. This is important, NSO does not by default take any changes in packages into account when restarting. When packages are reloaded the state/packages-in-use is updated.

Now, create a VLAN service, (nothing will happen since we have not defined any mapping).

admin@ncs(config)# services vlan net-0 vlan-id 1234 device-if c0 interface 1/0
admin@ncs(config-device-if-c0)# top
admin@ncs(config)# commit

Now let us move on and connect that to some device configuration using Java mapping. Note well that Java mapping is not needed, templates are more straight-forward and recommended but we use this as a "Hello World" introduction to Java service programming in NSO. Also at the end we will show how to combine Java and templates. Templates are used to define a vendor independent way of mapping service attributes to device configuration and Java is used as a thin layer before the templates to do logic, call-outs to external systems etc.

The default configuration of the Java VM is:

admin@ncs(config)# show full-configuration java-vm | details
java-vm stdout-capture enabled
java-vm stdout-capture file ./logs/ncs-java-vm.log
java-vm connect-time           60
java-vm initialization-time    60
java-vm synchronization-timeout-action log-stop
java-vm jmx jndi-address 127.0.0.1
java-vm jmx jndi-port 9902
java-vm jmx jmx-address 127.0.0.1
java-vm jmx jmx-port 9901

By default, ncs will start the Java VM invoking the command $NCS_DIR/bin/ncs-start-java-vm That script will invoke

$ java com.tailf.ncs.NcsJVMLauncher

The class NcsJVMLauncher contains the main() method. The started Java VM will automatically retrieve and deploy all Java code for the packages defined in the load-path of the ncs.conf file. No other specification than the package-meta-data.xml for each package is needed.

The verbosity of Java error messages can be controlled by:

admin@ncs(config)# java-vm exception-error-message verbosity
Possible completions:
  standard  trace  verbose

For more detail on the Java VM settings see The NSO Java VM.

The service model and the corresponding Java callback is bound by the service point name. Look at the service model in packages/vlan/src/yang:

Figure 29. VLAN Service model service-point

The corresponding generated Java skeleton, (one print hello world statement added):

Figure 30. Java Service Create Callback

Modify the generated code to include the print "Hello World!" statement in the same way. Re-build the package:

$ cd packages/vlan/src/
$ make

Whenever a package has changed we need to tell NSO to reload the package. There are three ways:

When that is done we can create a service (or modify an existing) and the callback will be triggered:

admin@ncs(config)# vlan net-0 vlan-id 888
admin@ncs(config-vlan-net-0)# commit

Now, have a look in the logs/ncs-java-vm.log:

$ tail ncs-java-vm.log
...
<INFO> 03-Mar-2014::16:55:23.705 NcsMain JVM-Launcher: \
       - REDEPLOY PACKAGE COLLECTION  --> OK
<INFO> 03-Mar-2014::16:55:23.705 NcsMain JVM-Launcher: \
       - REDEPLOY ["vlan"] --> DONE
<INFO> 03-Mar-2014::16:55:23.706 NcsMain JVM-Launcher: \
       - DONE COMMAND --> REDEPLOY_PACKAGE
<INFO> 03-Mar-2014::16:55:23.706 NcsMain JVM-Launcher: \
       - READ SOCKET =>
Hello World!

Tailing the ncs-java-vm.log is one way of developing. You can also start and stop the Java VM explicitly and see the trace in the shell. To do this, tell NSO not to start the VM by adding the following snippet to ncs.conf:

<java-vm>
    <auto-start>false</auto-start>
</java-vm>

Then, after restarting NSO or reloading the configuration, from the shell prompt:

$ ncs-start-java-vm
.....
.. all stdout from JVM

So modifying or creating a VLAN service will now have the "Hello World!" string show up in the shell. You can modify the package and reload/redeploy and see the output.

To use a GUI-based IDE Eclipse, first generate environment for Eclipse:

$ ncs-setup --eclipse-setup

This will generate two files, .classpath and .project. If we add this directory to eclipse as a "File->New->Java Project", uncheck the "Use the default location" and enter the directory where the .classpath and .project have been generated. We are immediately ready to run this code in eclipse.

Figure 31. Creating the project in Eclipse

All we need to do is to choose the main() routine in the NcsJVMLauncher class. The eclipse debugger works now as usual, and we can at will start and stop the Java code.

One caveat here which is worth mentioning is that there are a few timeouts between NSO and the Java code that will trigger when we sit in the debugger. While developing with the Eclipse debugger and breakpoints we typically want to disable all these timeouts. First we have 3 timeouts in ncs.conf that matter. Set the three values of /ncs-config/japi/new-session-timeout /ncs-config/japi/query-timeout /ncs-config/japi/connect-timeout to a large value. See man page ncs.conf(5) for a detailed description on what those values are. If these timeouts are triggered, NSO will close all sockets to the Java VM and all bets are off.

$ cp $NCS_DIR/etc/ncs/ncs.conf .

Edit the file and enter the following XML entry just after the Webui entry.

<japi>
    <new-session-timeout>PT1000S</new-session-timeout>
    <query-timeout>PT1000S</query-timeout>
    <connect-timeout>PT1000S</connect-timeout>
</japi>

Now restart ncs, and from now on start it as

$ ncs -c ./ncs.conf

You can verify that the Java VM is not running by checking the package status:

admin@ncs# show packages package vlan
packages package vlan
 package-version 1.0
 description     "Skeleton for a resource facing service - RFS"
 ncs-min-version 3.0
 directory       ./state/packages-in-use/1/vlan
 component RFSSkeleton
  callback java-class-name [ com.example.vlan.vlanRFS ]
 oper-status java-uninitialized

Create a new project and start the launcher main in Eclipse:

Figure 32. Starting the NSO JVM from Eclipse

You can start and stop the Java VM from Eclipse. Note well that this is not needed since the change cycle is: modify the Java code, make in the src directory and then reload the package. All while NSO and the JVM is running.

Change the VLAN service and see the console output in Eclipse:

Figure 33. Console output in Eclipse

Another option is to have Eclipse connect to the running VM. Start the VM manually with the -d option.

$ ncs-start-java-vm -d
Listening for transport dt_socket at address: 9000
NCS JVM STARTING
...

Then you can setup Eclipse to connect to the NSO Java VM:

Figure 34. Connecting to NSO Java VM Remote with Eclipse

In order for Eclipse to show the NSO code when debugging add the NSO Source Jars, (add external Jar in Eclipse):

Figure 35. Adding the NSO source Jars

Navigate to the service create for the VLAN service and add a breakpoint:

Figure 36. Setting a break-point in Eclipse

Commit a change of a VLAN service instance and Eclipse will stop at the breakpoint:

Figure 37. Service Create breakpoint

The mapping strategy using only Java is illustrated in the following figure.

Figure 43. Flat mapping with Java

This strategy has some drawbacks:

Java and templates can be combined:

Figure 44. Two layered mapping using feature templates

In this model the Java layer focuses on required logic, but it never touches concrete device models from various vendors. The vendor specific details are abstracted away using feature templates. The templates take variables as input from the service logic, and the templates in turn transform these into concrete device configuration. Introduction of a new device type does not affect the Java mapping.

This approach has several benefits:

Note that the logic layer does not understand the device types, the templates will dynamically apply the correct leg of the template depending on which device is touched.

From an abstraction point of view we want a template that takes the following variables:

So the mapping logic can just pass these variables to the feature template and it will apply it to a multi-vendor network.

Create a template as described before.

This results in a feature template like below:

<!-- Feature Parameters -->
<!-- $DEVICE -->
<!-- $VLAN_ID -->
<!-- $INTF_NAME -->

<config-template xmlns="http://tail-f.com/ns/config/1.0"
                 servicepoint="vlan">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device>
      <name>{$DEVICE}</name>
      <config>
        <vlan xmlns="urn:ios" tags="merge">
          <vlan-list>
            <id>{$VLAN_ID}</id>
          </vlan-list>
        </vlan>
        <interface xmlns="urn:ios" tags="merge">
          <FastEthernet tags="nocreate">
            <name>{$INTF_NAME}</name>
            <switchport>
              <trunk>
                <allowed>
                  <vlan tags="merge">
                    <vlans>{$VLAN_ID}</vlans>
                  </vlan>
                </allowed>
              </trunk>
            </switchport>
          </FastEthernet>
        </interface>
      </config>
    </device>
  </devices>
</config-template>

This template only maps to Cisco IOS devices (the xmlns="urn:ios" namespace), but you can add "legs" for other device types at any point in time and reload the package.

The Java mapping logic for applying the template is shown below:

Figure 45. Mapping logic using template

Note that the Java code has no clue about the underlying device type, it just passes the feature variables to the template. At run-time you can update the template with mapping to other device types. The Java-code stays untouched, if you modify an existing VLAN service instance to refer to the new device type the commit will generate the corresponding configuration for that device.

The smart reader will complain, "why do we have the Java layer at all?", this could have been done as a pure template solution. That is true, but now this simple Java layer gives room for arbitrary complex service logic before applying the template.

The steps to build the solution described in this section are:

Sometimes the input parameters are enough to generate the corresponding device configurations. But in many cases this is not enough. The service mapping logic may need to reach out to other data in order to generate the device configuration. This is common in the following scenarios:

It is important to design the service model to consider the above examples: what is input? what is available from other sources? This example illustrates how to define QoS policies "on the side". A reference to an existing QoS policy is passed as input. This is a much better principle than giving all QoS parameters to every service instance. Note well that if you modify the QoS definitions that services are referring to, this will not change the existing services. In order to have the service to read the changed policies you need to perform a re-deploy on the service.

This example also uses a list that maps every CE to a PE. This list needs to be populated before any service is created. The service model only has the CE as input parameter, and the service mapping code performs a lookup in this list to get the PE. If the underlying topology changes a service re-deploy will adopt the service to the changed CE-PE links. See more on topology below.

NSO has a package to manage resources like VLAN and IP addresses as a pool within NSO. In this way the resources are managed within the transaction. The mapping code could also reach out externally to get resources. Nano services are recommended for this.

Using topology information in the instantiation of a NSO service is a common approach, but also an area with many misconceptions. Just like a service in NSO takes a black-box view of the configuration needed for that service in the network NSO treats topologies in the same way. It is of course common that you need to reference topology information in the service but it is highly desirable to have a decoupled and self-sufficient service that only uses the part of the topology that is interesting/needed for the specific service should be used.

Other parts of the topology could either be handled by other services or just let the network state sort it out, it does not necessarily relate to configuration the network. A routing protocol will for example handle the IP path through the network.

It is highly desirable to not introduce unneeded dependencies towards network topologies in your service.

To illustrate this, lets look at a Layer 3 MPLS VPN service. A logical overview of an MPLS VPN with three endpoints could look something like this. CE routers connecting to PE routers, that are connected to an MPLS core network. In the MPLS core network there are a number of P routers.

Figure 47. Simple MPLS VPN Topology

In the service model you only want to configure the CE devices to use as endpoints. In this case topology information could be used to sort out what PE router each CE router is connected to. However, what type of topology do you need? Lets look at a more detailed picture of what the L1 and L2 topology could look like for one side of the picture above.

Figure 48. L1-L2 Topology

In pretty much all networks there is an access network between the CE and PE router. In the picture above the CE routers are connected to local Ethernet switches connected to a local Ethernet access network, connected through optical equipment. The local Ethernet access network is connected to a regional Ethernet access network, connected to the PE router. Most likely the physical connections between the devices in this picture has been simplified, in the real world redundant cabling would be used. The example above is of course only one example of how an access network could look like and it is very likely that a service provider have different access technologies. For example Ethernet, ATM, or a DSL based access network.

Depending on how you design the L3VPN service, the physical cabling or the exact traffic path taken in the layer 2 Ethernet access network might not be that interesting, just like we don't make any assumptions or care about how traffic is transported over the MPLS core network. In both these cases we trust the underlying protocols handling state in the network, spanning tree in the Ethernet access network, and routing protocols like BGP in the MPLS cloud. Instead in this case it could make more sense to have a separate NSO service for the access network, both so it can be reused for both for example L3VPN's and L2VPN's but also to not tightly couple to the access network with the L3VPN service since it can be different (Ethernet or ATM etc.).

Looking at the topology again from the L3VPN service perspective, if services assume that the access network is already provisioned or taken care of by another service, it could look like this.

Figure 49. Black-box topology

The information needed to sort out what PE router a CE router is connected to as well as configuring both CE and PE routers is:

This section describes the creation of a MPLS L3VPN service in a multi vendor environment applying the concepts described above. The example discussed can be found in examples.ncs/service-provider/mpls-vpn. The example network consists of Cisco ASR 9k and Juniper core routers (P and PE) and Cisco IOS based CE routers.

The goal with the NSO service is to setup a MPLS Layer3 VPN on a number of CE router endpoints using BGP as the CE-PE routing protocol. Connectivity between the CE and PE routers is done through a Layer2 Ethernet access network, which is out of scope for this service. In a real world scenario the access network could for example be handled by another service.

In the example network we can also assume that the MPLS core network already exists and is configured.

Figure 50. The MPLS VPN Example

  interface GigabitEthernet0/1.77
   description Link to PE / pe0 - GigabitEthernet0/0/0/3
   encapsulation dot1Q 77
   ip address 192.168.1.5 255.255.255.252
   service-policy output volvo
  !
  policy-map volvo
   class class-default
    shape average 6000000
   !
  !
 interface GigabitEthernet0/11
   description volvo local network
   ip address 10.7.7.1 255.255.255.0
  exit
  router bgp 65101
   neighbor 192.168.1.6 remote-as 100
   neighbor 192.168.1.6 activate
   network 10.7.7.0
  !

  vrf volvo
   address-family ipv4 unicast
    import route-target
     65101:1
    exit
    export route-target
     65101:1
    exit
   exit
  exit
  policy-map volvo-ce1
   class class-default
    shape average 6000000 bps
   !
   end-policy-map
  !
  interface GigabitEthernet 0/0/0/3.77
   description Link to CE / ce1 - GigabitEthernet0/1
   ipv4 address 192.168.1.6 255.255.255.252
   service-policy output volvo-ce1
   vrf         volvo
   encapsulation dot1q 77
  exit
  router bgp 100
   vrf volvo
    rd 65101:1
    address-family ipv4 unicast
    exit
    neighbor 192.168.1.5
     remote-as 65101
     address-family ipv4 unicast
      as-override
     exit
    exit
   exit
  exit

container vpn {

  list l3vpn {
    tailf:info "Layer3 VPN";

    uses ncs:service-data;
    ncs:servicepoint l3vpn-servicepoint;

    key name;
    leaf name {
      tailf:info "Unique service id";
      type string;
    }
    leaf as-number {
      tailf:info "MPLS VPN AS number.";
      mandatory true;
      type uint32;
    }

    list endpoint {
      key id;
      leaf id {
        tailf:info "Endpoint identifier";
        type string;
      }
      leaf ce-device {
         mandatory true;
         type leafref {
           path "/ncs:devices/ncs:device/ncs:name";
         }
      }
      leaf ce-interface {
        mandatory true;
        type string;
      }
      leaf ip-network {
        tailf:info “private IP network”;
        mandatory true;
        type inet:ip-prefix;
      }
      leaf bandwidth {
        tailf:info "Bandwidth in bps";
        mandatory true;
        type uint32;
      }
    }
  }
}

container topology {
  list connection {
    key name;
    leaf name {
      type string;
    }
    container endpoint-1 {
      tailf:cli-compact-syntax;
      uses connection-grouping;
    }
    container endpoint-2 {
      tailf:cli-compact-syntax;
      uses connection-grouping;
    }
    leaf link-vlan {
      type uint32;
    }
  }
}

grouping connection-grouping {
  leaf device {
    type leafref {
      path "/ncs:devices/ncs:device/ncs:name";
    }
  }
  leaf interface {
    type string;
  }
  leaf ip-address {
    type tailf:ipv4-address-and-prefix-length;
  }
}

Since we need to lookup which PE routers to configure using the topology model in the mapping logic it is not possible to use a declarative configuration template based mapping. Using Java and configuration templates together is the right approach.

The Java logic lets you set a list of parameters that can be consumed by the configuration templates. One huge benefit of this approach is that all the parameters set in the Java code is completely vendor agnostic. When writing the code there is no need for knowledge of what kind of devices or vendors that exists in the network, thus creating an abstraction of vendor specific configuration. This also means that in to create the configuration template there is no need to have knowledge of the service logic in the Java code. The configuration template can instead be created and maintained by subject matter experts, the network engineers.

With this service mapping approach it makes sense to modularize the service mapping by creating configuration templates on a per feature level, creating an abstraction for a feature in the network. In this example means we will create the following templates:

This is both to make services easier to maintain and create but also to create components that are reusable from different services. This can of course be even more detailed with templates with for example BGP or interface configuration if needed.

Since the configuration templates are decoupled from the service logic it is also possible to create and add additional templates in a running NSO system. You can for example add a CE router from a new vendor to the layer3 VPN service by only creating a new configuration template, using the set of parameters from the service logic, to a running NSO system without changing anything in the other logical layers.

Figure 51. The MPLS VPN Example

READ topology
FOR EACH endpoint
        USING topology
DERIVE connected-pe-router
                READ ce-pe-connection
        SET pe-parameters
        SET ce-parameters
        APPLY TEMPLATE l3vpn-ce
        APPLY TEMPLATE l3vpn-pe

Template peTemplate = new Template(context, "l3vpn-pe");
        Template ceTemplate = new Template(context,"l3vpn-ce");
        NavuList endpoints = service.list("endpoint");
        NavuContainer topology = ncsRoot.getParent().
                container("http://com/example/l3vpn").
                container("topology");

 for(NavuContainer endpoint : endpoints.elements()) {
            try {
                String ceName =  endpoint.leaf("ce-device").valueAsString();
                // Get the PE connection for this endpoint router
                NavuContainer conn =
                    getConnection(topology,
                                  endpoint.leaf("ce-device").valueAsString());
                NavuContainer peEndpoint = getConnectedEndpoint(
                                                conn,ceName);
                NavuContainer ceEndpoint = getMyEndpoint(
                                                conn,ceName);

TemplateVariables vpnVar = new TemplateVariables();
vpnVar.putQuoted("PE",peEndpoint.leaf("device").valueAsString());
vpnVar.putQuoted("CE",endpoint.leaf("ce-device").valueAsString());
vpnVar.putQuoted("VLAN_ID", vlan.valueAsString());
vpnVar.putQuoted("LINK_PE_ADR",
getIPAddress(peEndpoint.leaf("ip-address").valueAsString()));
vpnVar.putQuoted("LINK_CE_ADR",
                getIPAddress(ceEndpoint. leaf("ip-address").valueAsString()));
vpnVar.putQuoted("LINK_MASK",
                getNetMask(ceEndpoint. leaf("ip-address").valueAsString()));
vpnVar.putQuoted("LINK_PREFIX",
                getIPPrefix(ceEndpoint.leaf("ip-address").valueAsString()));

peTemplate.apply(service, vpnVar);
ceTemplate.apply(service, vpnVar);

admin@ncs# devices device ce1 sync-from
admin@ncs# show running-config devices device ce1 config \
        ios:router bgp | display xml

<config xmlns="http://tail-f.com/ns/config/1.0">
  <devices xmlns="http://tail-f.com/ns/ncs">
  <device>
    <name>ce1</name>
      <config>
         <router xmlns="urn:ios">
          <bgp>
            <as-no>65101</as-no>
            <neighbor>
              <id>192.168.1.6</id>
              <remote-as>100</remote-as>
              <activate/>
            </neighbor>
            <network>
              <number>10.7.7.0</number>
            </network>
          </bgp>
        </router>
      </config>
  </device>
  </devices>
</config>

<config-template xmlns="http://tail-f.com/ns/config/1.0">
  <devices xmlns="http://tail-f.com/ns/ncs">
    <device tags="nocreate">
      <name>{$CE}</name>
      <config>
       <router xmlns="urn:ios" tags="merge">
          <bgp>
            <as-no>{/as-number}</as-no>
            <neighbor>
              <id>{$LINK_PE_ADR}</id>
              <remote-as>100</remote-as>
              <activate/>
            </neighbor>
            <network>
              <number>{$LOCAL_CE_NET}</number>
            </network>
          </bgp>
        </router>
      </config>
    </device>
  </devices>
</config-template>