Bulk API

Bulk API calls allow you to issue a single API request to collect information about multiple vEdge devices in the overlay network.
The information is returned in batches.

You can perform two types of bulk API operations:

• State — These operations return status information about the devices, such as the number and state of OMP and BFD sessions.
• Statistics — These operations return statistics from the devices, such as the number of transmitted and received data packets.

State Bulk API Operations

State bulk API operations collect state and status information for all devices in the overlay network.

To collect state information in a bulk API operation, issue a call to the following URL:

GET https://{vManage-ip-address}/dataservice/data/device/state/data-type?count=number

• data-type is the parameter to collect state information about. It can be one of the following. The parameters are case-sensitive.

After the?, specify the following parameter:

• count—(Mandatory) Number of devices to query. This number corresponds to the number of records returned in the batch. The number can be from 1 through 10000.

A state bulk API call returns data and batch (page) information in the following format:

{
  "header": {
    "generatedOn": timestamp,
    "viewKeys": {
      "uniqueKey": [
        "vdevice-dataKey"
      ],
      "preferenceKey": "output-format"
    },
    "columns": [],
    "fields": []
  },
  "data": [],
  "pageInfo": {
    "startId": "batch:start-device-identifier",
    "endId": "batch:end-device-identifier",
    "moreEntries": (false | true),
    "count": number
  }
}

The output of a bulk API state call contains the following sections:

• header—The system provides a timestamp marking when the data collection occurred, a unique operation identifier, and formatting options for the collected data.
• data—State data itself.
• pageInfo—Information about the batch (page) itself:
• startId—Batch number and the identifier of the first device in the batch. The startId and endId for a single call have the same batch number. For each call, the batch number is unique.
• endId—Batch number and the identifier of the last device in the batch.
• moreEntries—Flag that indicates whether more data is available for given state type. If no more data is available, this field returns false. If more data is available, this field returns true.
• count—Number of records returned in the batch.

If moreEntries is true, retrieve the next batch of data by issuing a call with the following URL:

https://{vManage-ip-address}/dataservice/data/device/state/data-type?startId=batch:start-device-identifier&count=number

The startId identifies the first device to include in the next batch. To continue from the previous batch, specify the endId value from the returned pageInfo. For example, if the returned endId value is 203:10719, specify this as the startID in the URL. The data returned from the second batch then starts with device 10720. For the second batch, the batch number is different because each call's returned data is identified by a unique batch number.

count is the number of devices to query, and it corresponds to the number of records returned in the batch. Number can be from 1 through 10000.

Statistics Bulk API Operations

Statistics bulk API operations collect statistics for all Viptela devices in the overlay network.

To collect statistic information in a bulk API operation, issue a call to the following URL:

https://{vManage-ip-address}/dataservice/data/device/statistics/data-type?endDate=yyyy-MM-ddThh:mm:ss&startDate=yyyy-MM-ddThh:mm:ss&count=number&timeZone=timezone

• data-type is the parameter to collect statistics about. It can be one of the following. The parameters are case-sensitive.

After the?, specify the following parameters:

• endDate=yyyy-MM-ddThh:mm:ss—(Mandatory) End of time range to gather statistics.
• startDate=yyyy-MM-ddThh:mm:ss—(Mandatory) Beginning of time range to gather statistics.
• count—(Optional) Number of devices to query, corresponding to the number of records returned in the batch. Number can be from 1 through 10000. The default is 1000.
• timeZone=timezone—(Optional) Timezone for the start and end dates. Use the three-letter timezone.

A statistics bulk API call returns data and batch (page) information in the following format:

{
  "header": {
    "generatedOn": timestamp,
    "viewKeys": {
      "uniqueKey": [],
      "preferenceKey": "output-format"
    },
    "columns": [

    ],
    "fields": [

    ]  
  },
  "data": [

  ],
  "pageInfo": {
    "startTime": "start-time",
    "endTime": "end-time",
    "scrollId": "scroll-identifier",
    "hasMoreData": (false | true),
    "count": number,
    "totalCount": number
  }
}

The output of a bulk API statistics call contains the following sections:

• header—The system provides a timestamp marking when the data collection occurred, a unique operation identifier, and formatting options for the collected data.
• data—Statistics data itself.
• pageInfo—Information about the batch (page) itself:
• startTime—Time, in UNIX time format, when the system collected the first statistic in the batch.
• endTime—Time, in UNIX time format, when the system collected the last statistic in the batch.
• scrollId—Identifier of the current bulk session. It acts as a pointer to the next data entry. The scroll identifier expires after 10 minutes.
• hasMoreData—Flag that indicates whether more data is available for given state type. If no more data is available, this field returns false. If more data is available, this field returns true.
• count—Number of records returned in the batch.
• totalCount—Total number of data records available.

If hasMoreData is true, retrieve the next batch of data by issuing a call with the following URL:

https://{vManage-ip-address}/dataservice/data/device/statistics/data-type?scrollId=scroll-identifier

Device State

Bulk API calls returning state information for Cisco Catalyst SD-WAN Manager components:

• BFD Sessions
  Display information about BFD sessions.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/BFDSessions?count=count

• BGP Neighbor
  List BGP neighbors.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/BGPNeighbor?count=count

• Control Connection
  Display information about active control plane connections.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/ControlConnection?count=count

• Control Local Property
  Display the basic configuration parameters and local properties that are related to the control plane.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/ControlLocalProperty

• Control WAN Interface
  Display information about the WAN interface control connection.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/ControlWanInterface?count=count

• Hardware Alarms
  Display information about currently active hardware alarms.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/HardwareAlarms?count=count

• Hardware Environment
  Display status information about router components, including component temperature.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/HardwareEnvironment?count=count

• Hardware Inventory
  Display an inventory of router hardware components, including serial numbers.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/HardwareInventory?count=count

• Interface
  Display information about IPv4 interfaces.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/Interface?count=count

• OMP Peers
  Display information about active OMP peering sessions.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/OMPPeer?count=count

• System Status
  Display logging, reboot, and configuration history.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/SystemStatus?count=count

• System
  Display summary of the parameters configured in the system hierarchy.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/System?count=count

Request Parameters

Response Object for Bulk State API Calls

The bulk state API call responses have the following format:

{
   "header": {
      "generatedOn": timestamp,
      "viewKeys": {
         "uniqueKey": [
           "vdevice-dataKey"
         ],
         "preferenceKey": "display-format"
      },
      "columns": []'
      "fields": [],
   },
   "data": [
      returned data
   ],
   "pageInfo": {
      "startId": "batch:start-device-identifier",
      "endID": "batch:end-device-identifier",
      "moreEntries": (true | false),
      "count": number
   }
}

Example:
Request information about the first 100 BFD sessions:

GET https://{vmanage-ip-address}/dataservice/data/device/state/BFDSessions?count=100

The following output shows the data portion for a single BFD session. The attribute recordID identifies each record, and the attribute vdevice-dataKey identifies the unique key for each record in the output. The system returns the output on one line, but line breaks enhance readability.

"data":[
{
    "recordId":"#49:11450",
    "src-ip":"184.111.20.2",
    "dst-ip":"184.118.101.2",
    "vdevice-name":"172.16.251.20",
    "color":"mpls",
    "src-port":12346,
    "createTimeStamp":1466913875186,
    "system-ip":"172.16.248.101",
    "dst-port":12346,
    "site-id":80000001,
    "transitions":0,
    "vdevice-host-name":"vm6020",
    "local-color":"mpls",
    "detect-multiplier":"7",
    "vdevice-dataKey":"172.16.251.20-mpls-172.16.248.101-mpls",
    "proto":"ipsec",
    "lastupdated":1467763286045,
    "state":"up",
    "tx-interval":1000
},

The last section in the returned data, named pageInfo, reports the range of devices for which the system returned information. The moreEntries field indicates whether more entries are present.

"pageInfo":
{
    "startId":"49:32",
    "endId":"49:11133",
    "moreEntries":true,
    "count":100
}

If more entries are present, issue a second call, specifying the startId, to view data for additional devices. For example:

GET https://{vmanage-ip-address}/dataservice/data/device/state/BFDSessions?startId=49:11134&count=100

Continue issuing similar calls until the moreEntries field returns "false".

"pageInfo":
{
    "startId":"49:32",
    "endId":"49:11439",
    "moreEntries":false,
    "count":90
}

Device Statistics

Bulk API calls returning statistics for Cisco Catalyst SD-WAN Manager components:

• Alarms
  Display statistics about device alarms.

  GET https://{vmanage-ip-address}/dataservice/data/device/statistics/alarm?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• Application-aware Routing Status
  Display statistics about data traffic characteristics for all operational router data plane tunnels.

  GET https://{vmanage-ip-address}/dataservice/data/device/statistics/approutestatsstatistics?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• Audit Logs
  Display statistics about device audit logs.

  GET https://{vmanage-ip-address}/dataservice/data/device/statistics/auditlog?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• Bridge Interfaces
  Display statistics about bridge interfaces on a router.

  GET https://{vmanage-ip-address}/dataservice/data/device/statistics/bridgeinterfacestatistics?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• Bridge MAC Addresses
  Display statistics about MAC addresses that are learned by bridging domains.

  GET https://{vmanage-ip-address}/dataservice/data/device/statistics/bridgemacstatistics?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• Cflowd
  Display Router Cflowd Packet Statistics.

  GET https://{vmanage-ip-address}/dataservice/data/device/statistics/cflowdstatistics?startDate=yyyy-mm-dd hh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• CloudExpress Service
  Display statistics about CloudExpress service.

  GET https://{vmanage-ip-address}/dataservice/data/device/statistics/cloudxstatistics?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• Device Configuration
  Display device configuration statistics.

  GET https://{vmanage-ip-address}/dataservice/data/device/statistics/deviceconfiguration?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• Device Events
  Display notifications about events that have occurred on devices.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/deviceevent?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• Device System Status
  Display time and process information for devices, as well as CPU, memory, and disk usage data.

  GET https://{vmanage-ip-address}/dataservice/data/device/state/devicesystemstatusstatistics?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• DPI
  Display summary statistics for router DPI flows.

  GET https://{vmanage-ip-address}/dataservice/data/device/statistics/dpistatistics?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• Flow Logs
  Display statistics about packet flow logs.

  GET https://{vmanage-ip-address}/dataservice/data/device/statistics/flowlogstatistics?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• Interfaces
  Display interface statistics.

  GET https://vmanage-ip-address/dataservice/data/device/statistics/interfacestatistics?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

• WLAN Clients
  Display statistics about clients that are connected to a WLAN.

  GET https://vmanage-ip-address/dataservice/data/device/statistics/wlanclientinforstatistics?startDate=yyyy-mm-ddThh:mm:ss&endDate=yyyy-mm-ddThh:mm:ss

Request Parameters

Response Object for Bulk Statistics API Calls

The bulk statistics API call responses have the following format:

{
   "header": {
      "generatedOn": timestamp,
      "viewKeys": {

      },
      "fields": [
         column headers for output fields
      ],
      "chart": {}
   },
   "data": [
      returned data
   ],
   "pageInfo": {
      "startTime": "", start date of the current document set
      "endTime": "", end date of the current document set
      "scrollId": "" Scroll ID,
      "hasMoreData": [true | false] whether there is more data,
      "count": number of documents returned in the current request
   }
}

Example:

The URL in the following example displays cflowd statistics within a specified time range:

https://{vmanage-ip-address}/dataservice/data/device/statistics/cflowdstatistics?startDate=2016-07-07T00:00:00&endDate=2016-07-07T15:00:00

The following output shows that the data returned. The system returns the output on one line, but line breaks add readability. The scrollId field in the pageInfo section at the end of the output acts as a pointer to the next data entry. The hasMoreData field in the pageInfo section indicates that more cflowd statistics are available.

{
"header":
{"generatedOn":1467932473449,
"viewKeys":{"uniqueKey":[],
"preferenceKey":"grid-raw_cflowdstatistics"},
"columns":[
    {"title":
        "Device IP","property":"vdevice_name","dataType":"ip"},
    {"title":
        "Host name","property":"host_name","dataType":"string"},
    {"title":
        "Device Model","property":"device_model","dataType":"string"},
    {"title":
        "Time","property":"statcycletime",
        "displayFormat":"DD MMM YYYY h:mm:ss A z",
        "inputFormat":"unix-time","dataType":"date"},
    {"title":
        "entry_time","property":"entry_time",
        "displayFormat":"DD MMM YYYY h:mm:ss A z",
        "inputFormat":"unix-time","dataType":"date"},
    {"title":
        "Start Time","property":"start_time",
        "displayFormat":"yyyy-MM-dd'T'HH:mm",
       "inputFormat":"unix-time","dataType":"date"},
    {"title":
        "Total Pkts",
        "property":"total_pkts",
        "dataType":"number"},
    {"title":
        "Total bytes",
        "property":"total_bytes",
        "dataType":"number"}
],
"fields":[
    {"property":"vdevice_name","dataType":"ip"},
    {"property":"host_name","dataType":"string"},
    {"property":"device_model","dataType":"string"},
    {"property":"statcycletime","dataType":"date"},
    {"property":"entry_time","dataType":"date"},
    {"property":"vip_idx","dataType":"number"},
    {"property":"vpn_id","dataType":"number"},
    {"property":"remote_system_ip","dataType":"string"},
    {"property":"local_color","dataType":"string"},
    {"property":"remote_color","dataType":"string"},
    {"property":"proto","dataType":"string"},
    {"property":"src_ip","dataType":"string"},
    {"property":"dest_ip","dataType":"string"},
    {"property":"src_port","dataType":"number"},
    {"property":"dest_port","dataType":"number"},
    {"property":"dscp","dataType":"number"},
    {"property":"ip_proto","dataType":"number"},
    {"property":"ingress_intf","dataType":"string"},
    {"property":"egress_intf","dataType":"string"},
    {"property":"start_time","dataType":"date"},
    {"property":"total_pkts","dataType":"number"},
    {"property":"total_bytes","dataType":"number"},
    {"property":"start_time","dataType":"number"},
    {"property":"total_pkts","dataType":"number"},
    {"property":"start_time","dataType":"number"},
    {"property":"total_pkts","dataType":"number"},
    {"property":"total_bytes","dataType":"number"},
    {"property":"flow_active","dataType":"string"}
],
"chart":{}},
"data":[
    {"vdevice_name":"172.16.251.19",
        "host_name":"vm6019",
        "statcycletime":1467877201570,
        "device_model":"vedge-cloud",
        "entry_time":1467702002189,"vip_idx":1229,"vpn_id":1,
        "remote_system_ip":"NA","local_color":"NA",
        "remote_color":"NA",
        "src_ip":"144.200.168.122",
        "dest_ip":"74.240.139.218",
        "src_port":30999,
        "dest_port":23989,
        "dscp":48,
        "ip_proto":18,
        "ingress_intf":"cpu",
        "egress_intf":"ge0\/0",
        "total_pkts":2734203,
        "total_bytes":99979047,
        "flow_active":"false",
        "tunnel_color":"NA:NA",
        "destination":"74.240.139.218:23989",
        "id":"AVXETKjlfZiW1snFrC7n"
    },
    {"vdevice_name":"172.16.251.19",
        "host_name":"vm6019",
        "statcycletime":1467877201570,
        "device_model":"vedge-cloud",
        "entry_time":1467702002689,
        "vip_idx":351,"vpn_id":1,
        "remote_system_ip":"NA",
        "local_color":"NA",
        "remote_color":"NA",
        "src_ip":"20.99.232.228",
        "dest_ip":"122.100.201.80",
        "src_port":61921,
        "dest_port":11316,
        "dscp":48,
        "ip_proto":18,
        "ingress_intf":"cpu",
        "egress_intf":"ge0\/0",
        "total_pkts":7169028,
        "total_bytes":12981555,
        "flow_active":"false",
        "tunnel_color":"NA:NA",
        "destination":"122.100.201.80:11316",
        "id":"AVXETKjlfZiW1snFrC7o"
    }
],

The last section in the returned data, named pageInfo, reports the range of devices for which the system returned information. The scrollId field identifies the current bulk session and acts as a pointer to the next data entry. Use this value to scroll through data in the bulk session one entry at a time. The scroll identifier expires after 10 minutes. The moreEntries field indicates whether more entries are present.

"pageInfo":{
    "startTime":"1467877201570",
    "endTime":"1467877201570",
    "scrollId":"cXVlcnlUaGVuRmV0Y2g7MjA7OTk4MDU6Z3IxbFltaHRUOXlKZkxBUHFuTDBCUTs5OTgwMTpncjFsWW1odFQ5eUpmTEFQcW5MMEJROzk5ODAyOmdyMWxZbWh0VDl5SmZMQVBxbkwwQlE7OTk4MDM6Z3IxbFltaHRUOXlKZkxBUHFuTDBCUTs5OTgwNjpncjFsWW1odFQ5eUpmTEFQcW5MMEJROzk5ODA3OmdyMWxZbWh0VDl5SmZMQVBxbkwwQlE7OTk4MDQ6Z3IxbFltaHRUOXlKZkxBUHFuTDBCUTs5OTgwODpncjFsWW1odFQ5eUpmTEFQcW5MMEJROzk5ODA5OmdyMWxZbWh0VDl5SmZMQVBxbkwwQlE7OTk4MTA6Z3IxbFltaHRUOXlKZkxBUHFuTDBCUTs5OTgxMTpncjFsWW1odFQ5eUpmTEFQcW5MMEJROzk5ODEyOmdyMWxZbWh0VDl5SmZMQVBxbkwwQlE7OTk4MTM6Z3IxbFltaHRUOXlKZkxBUHFuTDBCUTs5OTgxNDpncjFsWW1odFQ5eUpmTEFQcW5MMEJROzk5ODE1OmdyMWxZbWh0VDl5SmZMQVBxbkwwQlE7OTk4MTY6Z3IxbFltaHRUOXlKZkxBUHFuTDBCUTs5OTgxNzpncjFsWW1odFQ5eUpmTEFQcW5MMEJROzk5ODE4OmdyMWxZbWh0VDl5SmZMQVBxbkwwQlE7OTk4MTk6Z3IxbFltaHRUOXlKZkxBUHFuTDBCUTs5OTgyMDpncjFsWW1odFQ5eUpmTEFQcW5MMEJROzA7_2",
    "hasMoreData":true,
    "count":50,
    "totalCount":9423934
  }
}

If more entries are present, issue a second call, specifying the scrollId as the starting point:

https://{vmanage-ip-address}/dataservice/data/device/statistics/cflowdstatistics?scrollId=cXVlcn...

Repeat the previous command, with the appropriate ScrollID, until the hasMoreData field returns as false.