Cisco pxGrid API

Technical Overview

pxGrid is a protocol framework that defines the control mechanisms to facilitate machine-to-machine communications

Protocol framework

pxGrid is a protocol framework of high level mechanisms that can be use with any type of transport protocol. It does not bind to any transport. The implementation of pxGrid protocol describe in this document is specific to Cisco ISE. In other words, only the mechanism follows pxGrid protocol framework, the data formats and transports are specific to Cisco ISE.

Control mechanisms

Borrowing the terminology "Control" and "Data" from networking:

  • pxGrid defines only "Control" mechanisms:
    • Discovery
    • Authentication
    • Authorization
Control, not Data

It is important to see the separation of "Control" and "Data".

"Data" between consumers and providers is not part of pxGrid. "Data" does not go through pxGrid.

Cisco ISE also provides "Data" such as Session Directory, ANC... etc, However, they are just providers that make use of pxGrid "Control" facilities. Other "Data" providers are free to choose whatever data transport mechanisms they see fit. As a result, pxGrid can adapt to any new or existing "Data" transport.

Machine-to-machine

As pxGrid provides centralized discovery, authentication and authorization, it reduces the complexity of meshed network where every machine needs to communicate with each other. It also eliminates the need for human-generated passwords and ensure machine access configurations are not shared between devices.

STOMP and pxGrid

pxGrid will use port 8910 on ISE for pxGrid-related REST and Websocket communication. The server will play Websocket’s ping-pong game to detect offline, slow, or faulty pxGrid clients and network failures. Messages over Websocket will be sent and received in binary format. These messages should conform to the STOMP messaging protocol. You can read more about STOMP here. The following STOMP commands are supported on pxGrid:

  • CONNECT

  • DISCONNECT

  • SUBSCRIBE

  • UNSUBSCRIBE

  • SEND

  • MESSAGE

  • ERROR

pxGrid consumers will typically implement SUBSCRIBE while providers will implement SEND.

Note: Consumers can and should subscribe to multiple pxGrid services using the same connection.

Typical pxGrid Client Flow

The following provides a very high-level overview of a typical pxGrid client’s logical flow. Please note that the code samples provided on github do not implement this entire flow completely. They are provided to demonstrate only the client-to-server communication portions:

pxGrid Client Flow

  1. Create Certificate SSL Context / Account Create

    All clients must authenticate either via certificate-based SSL authentication or username-password authentication. While clients have the option to generate passwords via the Account Create API, it is highly recommended to use certificates instead. Certificate-based SSL authentication is far more secure and is the recommended way to authenticate clients onto pxGrid. You can quickly generate and reuse certificates for pxGrid clients using the ISE admin UI, if needed.

  2. Account Activate

    All clients must request to activate their accounts on the pxGrid server. The respective REST API handles this process. Typically, clients should poll on this call until they receive the “ENABLED” message from the server. Server-side auto-approval is available from the ISE admin UI just like in previous implementations, however, it is not recommended to use this option in production-level environments.

  3. Register / Unregister Service

    Service providers will use these API’s to provide and update the necessary information (for example, resource URLs) from which their services are accessible for other pxGrid clients.

  4. Service Lookup

    All clients can use this API to dynamically discover all available provider services and their locations. With this information, clients can proceed to the next steps. It is recommended to implement some sort of reconnection-mechanism in pxGrid clients so that in the case a connection drops or resource location updates, the client can retrieve the service again using this API.

  5. Access Secret

    For every service returned that interest a particular client, that client must also query pxGrid for an access secret in order to obtain the information provided by the service. When a service is changed or needs to be reconnected, an access secret must be regenerated via this API.

  6. Service Query / Subscribe

    With the access secret and service location information in hand, clients can then perform REST-based queries or build Websocket connections to receive information.

The following is a visual representation of the above client-to-server interaction flows with the new REST and Websocket architecture:

pxGrid client server interaction

The pxGrid development team has provided samples for all of these actions in a public github project as noted before: https://github.com/cisco-pxgrid/pxgrid-rest-ws/wiki. This project provides snippets of code-examples and detailed documentation for all of the APIs discussed above. It is recommended to explore and experiment with these before attempting to implement your own pxGrid client.

Cisco ISE implementation

Cisco ISE implements the pxGrid Controller that handles the "Control".

pxGrid 2.0

pxGrid 2.0 was introduced in ISE 2.3 and officially supported in ISE 2.4 and above to include REST APIs support. All APIs are described in more details in the github site: https://github.com/cisco-pxgrid/pxgrid-rest-ws/wiki

pxGrid Control REST APIs

The followings are new REST APIs that implements pxGrid Control mechanisms:

  • ServiceLookup (Discovery)
    • This is used by consumers to lookup for services
  • ServiceRegister (Discovery)
    • For providers to register their services
  • ServiceReregister (Discovery)
    • For providers to invoke periodically to signify the service is still alive
  • ServiceUnregister (Discovery)
    • For providers to indicate the services are not active anymore
  • AccessSecret (Authentication)
    • Obtain unique secrets between source and destination nodes. Used by both provider and consumer.
  • Authorization
    • For providers to check if the operation is permitted by consumers
pxGrid account management
  • AccountCreate
    • Used for password-based authentication. This will return a generated password for subsequence REST calls.
  • AccountActivate
    • Account must be approved by ISE administrators before pxGrid can be used. By calling this, the new account will move to a PENDING state, awaiting for approval. When the account is approved, the state will becomes ACTIVE.
Why use REST?

Cisco ISE is migrating pxGrid to REST for the following reasons:

  • Easy adoption as developers can choose any library and language for their REST clients
  • Support any number of pxGrid nodes as current Cisco ISE XMPP implementation restricts to active-standby with only 2 pxGrid nodes
  • Works together with new WebSocket Pubsub service in Cisco ISE which is horizontally scalable
Be ready for pxGrid 2.0

Developers should start implementing pxGrid 2.0 REST APIs because Cisco ISE is migrating from XMPP to REST. XMPP support will remain but will not be updated. As for migration strategy, pxGrid 2.0 should be used when available with pxGrid 1.0 (XMPP) as fallback.

Use this to check the pxGrid version running: https://[hostname]:8910/pxgrid/control/version

ISE 2.3 use pxGrid version 2.0.0.7. ISE 2.3 migrated most XMPP services except ANC and MDM. ISE 2.4 use pxGrid version 2.0.0.13. ISE 2.4 migrated all XMPP services.

Using pxGrid as consumer

A consumer perform the following steps:

  1. AccountActivation: Activate once at consumer startup
  2. ServiceLookup: Get properties like URLs, topic... etc
  3. AccessSecret: Get unique secret between 2 nodes Subsequence actions depend on the services, the consumer can perform a REST call or connect/subscribe to WebSocket Pubsub server. For details, check out the github page: https://github.com/cisco-pxgrid/pxgrid-rest-ws/wiki

Using pxGrid as provider

A provider performs the followings:

  1. AccountActivation: Activate once at provider startup
  2. ServiceRegister/ServiceReregister/ServiceUnregister
  3. AccessSecret: Get unique secret between 2 nodes
  4. Authorization: If required, this is used to check authorization Subsequence actions depend on the services, the provider can listen for REST call or connect/publish to WebSocket Pubsub server. For details, check out the github page: https://github.com/cisco-pxgrid/pxgrid-rest-ws/wiki

Cisco ISE Services support

This table shows versions when these services were being supported for REST/WS and XMPP.

Service Description ISE version for REST/WS ISE version for XMPP
ANC For Quarantine of endpoints 2.4 2.0
MDM Events for MDM 2.4 2.1
Profiler Configuration of profiler 2.3 2.0
Pubsub A scalable pubsub service 2.3 n/a
Radius Info of Radius events 2.3 n/a
Session Directory Info of all endpoint sessions 2.3 1.4
System Info about the system 2.3 n/a
Trustsec Configurations and events of Trustsec 2.3 2.0
Trustsec SXP Events for Trustsec SXP bindings 2.3 2.0

The API references are available in GitHub: https://github.com/cisco-pxgrid/pxgrid-rest-ws/wiki

pxGrid Filtering

From Cisco ISE Release 3.4, pxGrid supports filtering of information based on specific requirements of the clients. Prior to Cisco ISE Release 3.4, pxGrid published all the information it received from the publishers to the clients. The pxGrid filtering feature enables clients to only receive relevant information from the publisher on a per-subscription basis. The filtering of information is achieved using the filtering API on the pxGrid server. pxGrid filtering of relevant information based on the applied filters takes place in the following two cases:

  1. Before a bulk download
  2. Before publishing live data to clients

In Cisco ISE Release 3.4, pxGrid filtering is currently supported in three topics:

  1. TrustSec SXP
  2. Session Directory - Session Topic
  3. Session Directory - Group Topic

For more information and working examples, see the pxGrid GitHub page: https://github.com/cisco-pxgrid/pxgrid-rest-ws/wiki.

Screen capture for using samples

Sample codes are available in github: https://github.com/cisco-pxgrid/pxgrid-rest-ws These are by no means production ready code, please use only for reference.

Before running samples

Enable pxGrid

To start using pxGrid, enable it in deployment page:

Create a client certificate

There are many ways to create a client certificate to use for authentication of pxGrid nodes. If there is no existing infrastructure for issuing certificates, Cisco ISE can act as an internal CA for issuing certificates.

To create client certificate for pxGrid in Cisco ISE:

The zip file contains the client key/cer files with its certificate chain files.

Running the samples

Java samples use .jks files for certificates. Python and Go uses pem/key files.

A example of showing the running SessionSubscribe to show the help page:

Code Snippet
CopyALEI-M-147Y:java alei$ mvn exec:java -Dexec.mainClass="com.cisco.pxgrid.samples.ise.SessionSubscribe" -Dexec.args="-?"
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building pxgrid-rest-ws-samples 2.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ pxgrid-rest-ws-samples ---
usage: SessionSubscribe
 -a,--hostname <arg>             Host name (multiple accepted)
 -d,--description <arg>          Description (optional)
 -k,--keystorefilename <arg>     Keystore .jks filename (not required if
                                 password is specified)
 -p,--keystorepassword <arg>     Keystore password (not required if
                                 password is specified)
 -q,--truststorepassword <arg>   Truststore password
 -t,--truststorefilename <arg>   Truststore .jks filename
 -u,--nodename <arg>             Node name
 -w,--password <arg>             Password (not required if keystore is
                                 specified)
ALEI-M-147Y:java alei$

First time running sample will put node in PENDING state

Code Snippet
CopyALEI-M-147Y:java alei$ mvn exec:java -Dexec.mainClass="com.cisco.pxgrid.samples.ise.SessionSubscribe" -Dexec.args="-a pxgrid-071.cisco.com -u user1 -k /Users/alei/Downloads/my-pxgrid-071/keycert.jks -p Cisco123 -t /Users/alei/Downloads/my-pxgrid-071/trust.jks -q Cisco123"
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building pxgrid-rest-ws-samples 2.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ pxgrid-rest-ws-samples ---
------ config ------
  hostname = pxgrid-071.cisco.com
  nodename = user1
  password = (not specified)
  description = (not specified)
  keystorefilename = /Users/alei/Downloads/my-pxgrid-071/keycert.jks
  keystorepassword = Cisco123
  truststorefilename = /Users/alei/Downloads/my-pxgrid-071/trust.jks
  truststorepassword = Cisco123
--------------------
18:49:25.872 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.PxgridControl - AccountActivate request={}
18:49:26.642 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.PxgridControl - AccountActivate response={"accountState":"PENDING","version":"2.0.0.13"}
18:50:26.637 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.PxgridControl - AccountActivate request={}
18:50:26.747 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.PxgridControl - AccountActivate response={"accountState":"PENDING","version":"2.0.0.13"}

Cisco ISE administrator will require to approve the new node:

After approval, subscription is successful:

Code Snippet
CopyALEI-M-147Y:java alei$ mvn exec:java -Dexec.mainClass="com.cisco.pxgrid.samples.ise.SessionSubscribe" -Dexec.args="-a pxgrid-071.cisco.com -u user1 -k /Users/alei/Downloads/my-pxgrid-071/keycert.jks -p Cisco123 -t /Users/alei/Downloads/my-pxgrid-071/trust.jks -q Cisco123"
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building pxgrid-rest-ws-samples 2.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ pxgrid-rest-ws-samples ---
------ config ------
  hostname = pxgrid-071.cisco.com
  nodename = user1
  password = (not specified)
  description = (not specified)
  keystorefilename = /Users/alei/Downloads/my-pxgrid-071/keycert.jks
  keystorepassword = Cisco123
  truststorefilename = /Users/alei/Downloads/my-pxgrid-071/trust.jks
  truststorepassword = Cisco123
--------------------
18:38:36.086 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.PxgridControl - AccountActivate request={}
18:38:36.805 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.PxgridControl - AccountActivate response={"accountState":"ENABLED","version":"2.0.0.13"}
18:38:36.805 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.SessionSubscribe - pxGrid controller version=2.0.0.13
18:38:36.808 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.PxgridControl - ServiceLookup request={"name":"com.cisco.ise.session"}
18:38:36.920 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.PxgridControl - ServiceLookup response={"services":[{"name":"com.cisco.ise.session","nodeName":"ise-mnt-pxgrid-071","properties":{"sessionTopic":"/topic/com.cisco.ise.session","groupTopic":"/topic/com.cisco.ise.session.group","wsPubsubService":"com.cisco.ise.pubsub","restBaseURL":"https://pxgrid-071.cisco.com:8910/pxgrid/mnt/sd","restBaseUrl":"https://pxgrid-071.cisco.com:8910/pxgrid/mnt/sd"}}]}
18:38:36.920 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.SessionSubscribe - wsPubsubServiceName=com.cisco.ise.pubsub sessionTopic=/topic/com.cisco.ise.session
18:38:36.921 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.PxgridControl - ServiceLookup request={"name":"com.cisco.ise.pubsub"}
18:38:37.022 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.PxgridControl - ServiceLookup response={"services":[{"name":"com.cisco.ise.pubsub","nodeName":"ise-pubsub-pxgrid-071","properties":{"wsUrl":"wss://pxgrid-071.cisco.com:8910/pxgrid/ise/pubsub"}}]}
18:38:37.022 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.SessionSubscribe - wsUrl=wss://pxgrid-071.cisco.com:8910/pxgrid/ise/pubsub
18:38:37.023 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.PxgridControl - AccessSecret request={"peerNodeName":"ise-pubsub-pxgrid-071"}
18:38:37.124 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.PxgridControl - AccessSecret response={"secret":"h91eqfEwPdiHd3HL"}
18:38:38.037 [Grizzly(2)] INFO com.cisco.pxgrid.samples.ise.StompPubsubClientEndpoint - WS onOpen
18:38:38.039 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.StompPubsubClientEndpoint - STOMP CONNECT host=pxgrid-071.cisco.com
18:38:38.043 [com.cisco.pxgrid.samples.ise.SessionSubscribe.main()] INFO com.cisco.pxgrid.samples.ise.StompPubsubClientEndpoint - STOMP SUBSCRIBE topic=/topic/com.cisco.ise.session
18:38:38.073 [Grizzly(1)] INFO com.cisco.pxgrid.samples.ise.StompPubsubClientEndpoint - STOMP CONNECTED version=1.2
press <enter> to disconnect...
Next