Unified CVP REST Request Throttling

CVP REST API is a web service running on Tomcat Server and the API is exposed through REST. As authentication is the only mechanism to control the requests processed by web service, there is a need for throttling the requests to control the request admission rate (requests per second), and the number of concurrent requests being processed by the server.

The throttling is based on the number of concurrent requests being processed, as it does not control the rate of request admission but controls the concurrent requests in process. A simple throttling mechanism used for limiting the request admission rate at the server is described below.

Throttling for WSM Request Processing

The request groups or throttling channels are defined as follows:

• Media request channel for CREATE, UPDATE and DELETE requests.

• VXMLApp request channel for DEPLOY and DELETE requests.

• Media request channel for CREATE, UPDATE and DELETE requests.

For each channel, a threshold is defined which is configurable using a configuration property (throttling.request.media.threshold , throttling.request.vxmlapp.threshold and throttling.request.generic.threshold ). A threshold for the total number of requests in process (throttling.request.total.threshold) is also available in %CVP_HOME%/conf/cvpwsmconfig.properties file.

When a new request is received, total requests in process are compared with the threshold (throttling.request.total.threshold). If it is within the limit, then the request counter for the request channel to which the new request belongs is compared to its threshold. If the request counter is also within the limit, then the request is admitted for processing and the counter for total number requests and the counter for the request channel are incremented.

When the request processing is complete, counter for the request channel and counter for total number of requests are decremented. All counters are cleared when an application restarts. If the request is rejected by the throttler, then an HTTP Status code of 503 (Service Unavailable) is sent to the client with an appropriate error message indicating that the server is busy.

Throttling for ORM Request Processing

When a request is sent from WSM to ORM either for the media operation or VXML application operation, the request admission is controlled by the ORM. This is to ensure that enough resources are available for the call processing services (such as VXML Server, Call Server and Media Server) running on the Unified CVP Server.

The following separate thread pools are maintained by REST service on the Unified CVP Server.

• VXMLAppThread Pool - This thread pool runs with a single worker thread for request processing. Internal queue length is controlled by a configurable parameter (throttling.threadpool.vxmlapp.queuesize) defined ORM.PROPERTIES file. VXML Application DEPLOY and DELETE requests are processed by this thread pool.
• Generic Thread Pool - This thread pool handles all requests other than the ones mentioned in VXMPApp Thread Pool. Number of Worker Threads and the queue size are configured through configuration parameters (throttling.threadpool.generic.queuesize & throttling.threadpool.generic.theads)

The following table lists the parameter default values