Rate Limiting
The API is rate limited to protect against overload situations.
Overview
Rate limiting on the API is designed to protect the platform from excessive requests. When the number of request received within a time window reaches a threshold number, then further requests are rejected until the rate falls back below that threshold. Also there is a maximum size of query response that can be selected using the .maxResults query parameter. Also the maximum number of concurrent requests (per user and across all users) that the server will handle can be configured. Once this limit is reached further requests will be rejected. There are five thresholds that can be set by the system administrator of the platform:
- The maximum number of requests for all users within the window period.
- The maximum number of requests per user within the window period.
- The maximum number of concurrent requests per user.
- The maximum number of concurrent requests for all users.
- The maximum size of paged request.
If any of these thresholds is exceeded, then requests will be rejected with status code of 503 (Service Unavailable). The text of the response will contain information about the cause as appropriate:
- NBI Global Rate limit exceeded (More than xxx in yyy ms)
- NBI Rate limit for user
exceeded (More than xxx in yyy ms) - Per user concurrent query count exceeded Currently xxx : Limit is yyy
In the case of the per user threshold being exceeded, other users will continue to have access to the NBI as normal.
Note that requests rejected by the rate limiter do not count in calculation of the threshold. Requests that fail due to other factors are still counted.
Configuration
Rate and size limiting is configured with properties as defined in the table below. These properties can only be changed by a systems administrator, and require a platform restart to take effect.
The properties must be saved in a new file named nbi.properties and placed in the directory /opt/CSCOlumos/conf/.
| Property | Allowed Values | Default Value | Description |
|---|---|---|---|
| nbi.rateLimit.windowSize | An integer representing a number of milliseconds. | 1000 | This property is the size of the sliding window used to count requests. |
| nbi.rateLimit.windowSegments | A integer value such that windowSize/windowSegements is a whole number. | 10 | This property is used internally to divide the window into segments. The number of segments determine the rate at which the window decrements the request counter. For example with 10 segments in a 1000 ms window, the count will be reduced every hundred ms, by the number of counts held in the oldest segment. (The count is increased immediately after receiving a new request). |
| nbi.rateLimit.totalRequestThreshold | An integer. Use -1 to disable the limiting of total requests. | 20 | This is the number of total request that can be handled in windowsize ms. |
| nbi.rateLimit.perUserThreshold | An integer. Use -1 to disable the limiting of per user requests. | 5 | This is the number of requests for any given user that can be handled in windowsize ms. |
| nbi.rateLimit.maxConcurrentQueries | A non-zero positive integer. | 10 | This is the number of concurrent requests allowed for any given user. |
| nbi.rateLimit.maxAllUserConcurrentQueries | A non-zero positive integer.Use -1 to disable the limiting of all user concurrent requests | 45 | This is the number of concurrent requests allowed for all users. |