Writing Custom KPIs

The following sections explain how to create custom KPIs for use with Cisco Crosswork Health Insights.

What are KPIs?

A Cisco Crosswork Health Insights Key Performance Indicator (KPI) is a programming construct that captures network device and health metrics. Crosswork Health Insights provides stock KPIs that you can start using right away, with minimal configuration (see Health Insights Stock KPIs).

The power of the Health Insights application comes from the ability to create custom KPIs, and to define custom alerts that the KPI will raise when it detects anomalies. You can then remediate detected anomalies manually, or by linking the KPI to one of the Crosswork Change Automation application's Playbooks and then triggering the Playbook when the alert is raised.

This guide will help you get an overview of the stock Crosswork Health Insights KPIs and how to create a custom KPI from a stock KPI.

Importance of KPIs

A KPI is a powerful, modern tool which collects data based on defined parameters, reports it, and alerts users when anomalies are detected. KPIs use lightweight, reliable and secure Model-Driven Telemetry (MDT) and SNMP polling to gather network and device data. The gathered data is then evaluated based on operator-configured settings in Crosswork.

When KPI data indicates a deviation outside established parameters, Crosswork alerts the operator. The response to the alert can be entirely manual, or semi-automated by linking the KPI to a Crosswork Change Automation Playbook, which the operator can then decide to trigger. Linking a KPI to a Playbook also allows the Playbook to get the execution variable values it needs directly from the incoming KPI alert, and indirectly from the configured KPI.

KPI Elements

A KPI contains the elements shown in the following table. For an example of these elements and their values in a stock KPI, see Inside the CPU Threshold KPI.

The following figure shows the first five of these elements as they appear in the KPI file (see KPI Files).

KPI Types

No-Alert State

KPIs in this category are solely for reporting purposes, such as with KPIs that report installed software versions or monitor device uptime. They collect data and report it based on the definitions in the Parameters element. These alerts are used for data collection and visualization.

Rate Change

KPIs in this category report data and generate alerts whenever the TICKscript detects an abnormal rate of change (rising or falling) in measured values. This is a derivative-based template. It can be used in scenarios such as detecting interface state changes, interface errors, and so on.

Standard Deviation

KPIs in this category report data and generate alerts based on statistical thresholding. These alerts are cleared based on the definitions in the Parameters element.

These KPIs generate alerts based on a formula, which is the general definition of Standard Deviation (defined as the square root of Variance, where Variance is the average of the squared differences from the Mean, and the Mean is the average of the reported values). They report an alert whenever the data is a given degree of standard deviation from the norm, which usually means there is a sudden spike or drop in the data. There is also an activation threshold, which sets the minimum value the data must present for an alert to be generated, so only spikes or drops that reach a certain raw value are significant. If the alert activation threshold is -1, it is always ignored. Data points are collected based on the value of the Cadence element.

Two-Level Threshold

The KPIs defined under this category report data and generate alerts whenever the values breach or cross a defined set of standards. Alerts are cleared when performance returns to the user defined acceptable level based on the definitions in the Parameters element.

Two levels of alerts can be generated, depending on user-defined values in the Parameters element, including labels for each threshold. Users customizing this type of KPI in the Crosswork Health Insights application user interface can pick labels from a dropdown menu (the choices are MINOR, MAJOR, WARNING, and CRITICAL.). When the data is above the Level 2 threshold, the KPI sends alerts using the Level 2 label. When it is between Level 2 and Level 1, it sends alerts using the Level 1 label. When the data is below Level 1, it will be CLEAR.

Crosswork Health Insights threshold KPIs employ dampening and hysteresis, as illustrated in the figure below. Alerts are generated only when the value is above a threshold for a time interval and are cleared when the value is below a threshold for a defined time interval (known as the clear time). Users can define the clear time by setting the amount of time the value needs to be below the level 1 threshold.

Inside the CPU Threshold KPI

The following table describes KPI elements “under the hood” of the CPU Threshold KPI and gives examples of the kinds of values these elements contain. KPI execution output has a nested structure, with (for example) many possible “Parameter” elements defined within the “Parameters” element. The table also indicates whether users can modify these elements’ values at runtime, using the Cisco Crosswork Health Insights user interface. Although many are not modifiable at runtime, you can modify many of them if you choose to create a custom KPI using a stock KPI as your template. See the topic Create Custom KPIs From Stock KPIs, where we demonstrate how to change the numerical thresholds for the CPU Threshold KPI.

Create Custom KPIs From Stock KPIs

In addition to creating your own KPIs (see Create a New KPI in the Cisco Crosswork Change Automation and Health Insights User Guide, you can use the Health Insights application and tools of your choice to export a stock KPI, customize it, then import and run it like any other KPI. The steps below use the KPI discussed in Inside the CPU Threshold KPI as an example, demonstrating how you can modify this stock KPI's level 1 and level 2 threshold values to match your needs, then make additional modifications that will allow you to import it back into Crossworks Health Insights.

1. Log in to an installed instance of Cisco Crosswork Change Automation and Health Insights and navigate to **Health Insights > Manage KPIs**.
2. Select the "CPU Threshold" KPI. Then click **Export** to download it as a gzip archive.
3. Un-tar the downloaded gzip archive file using the tool of your choice. For example, using the command line: tar -xvzf kpis-export-1234567890.tar.gz. In this example, the tar command will unzip the archive to a new export folder, with three subfolders: dashboards, kpis, and ticks.
4. Navigate to the ticks folder and open the pulse\_cpu\_threshold\_template.tick script using the ASCII editor of your choice.
5. Make the following changes to the ASCII tick script file:
   1. Find and change the default floating-point values for the two threshold variables level2\_threshold and level1\_threshold. For example:
      • var level1\_threshold = 60.0
      • var level1\_threshold = 40.0.
   2. Find and change the values of the ID variables kpi\_id and alert\_id. For example:
      • var kpi\_id = 'my\_pulse\_cpu\_threshold'
      • var alert\_id = 'my\_pulse\_cpu\_threshold'
   3. When you are finished, save the changed tick script using a new name. For example: my\_pulse\_cpu\_threshold\_template.tick.
6. Navigate to the kpis folder and open the pulse\_cpu\_threshold\_kpi.json file using the JSON editor of your choice.
7. Make the following changes to the KPI JSON file:
   1. Find and change the kpi\_id and script\_id variables in the JSON file so that they point to the KPI and script names you just modified. For example:
      • "kpi\_id": "my\_pulse\_cpu\_threshold"
      • "script\_id":"my\_pulse\_cpu\_threshold\_template.tick"
   2. When you are finished, save the modified JSON file with a new name. For example: my\_pulse\_cpu\_threshold\_kpi.json.
8. Navigate to the dashboards folder and open the Pulse-cpu-threshold-raw.json and Pulse-cpu-threshold-summary.json files using the JSON editor of your choice.
9. Make the following changes in both of the dashboard JSON files:
   • Find every occurrence of the old kpi\_id value and replace it with the new kpi\_id value you created. In this example, you would find every occurrence of pulse\_cpu\_threshold and replace it with my\_pulse\_cpu\_threshold.
   • When you are finished, save the modified dashboard JSON files using the same file names.
10. Compress the modified KPI files into a single gzip archive with a new file name. For example, using the command line: tar -cvzf kpis-my-cpu-threshold.tar.gz export/.
11. Log in to Cisco Crosswork Change Automation and Health Insights and navigate to **Health Insights \> Manage KPIs**. Then click **Import** to upload the new KPI you created using the stock KPI as a template.

Health Insights Stock KPIs

The table below lists the stock Health Insights KPIs supplied with Cisco Crosswork Change Automation and Health Insights.

Alerting types in the table that you can select when you create a new KPI using the Health Insights user interface are:

• No Alert : The KPI gathers, tracks and reports performance data without triggering alerts.
• Standard Deviation : The KPI detects spikes or drops in measured values and alerts when these values deviate some number of standard deviations away from their normal values.
• Two-Level Threshold : The KPI detects abnormal measured values using two custom thresholds and the ability to provide dampening intervals on the thresholds.
• Rate Change : The KPI detects abnormal rates of change in measured values to detect rising or falling values.

Additional alerting types that you can use when you export and use stock KPIs to create KPIs with custom parameters are:

• Standard Deviation of Rate Change : The KPI alerts on standard deviations of the rate of change.
• Low Single Threshold : The KPI alerts on a single threshold when the value falls below that threshold.
• Direct Alarm Forwarding : The KPI uses the alarm from the device directly, as a Health Insights KPI alert.
• Major/Minor/Low/High Thresholds : The KPI alerts on Major high, Minor high, Minor low, and Major low values.
• Line State Changes : The KPI alerts on shutdowns and flapping in line states.