Getting Started
Cisco AppDynamics exposes various APIs to programmatically manage the health rules for the entities.
Base URI
Every health rule API request begins with this Base URI:
Copyhttps://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules
Authentication
The Cisco Observability Platform Health Rule API uses the Open Authentication 2.0 (OAuth2) token-based authentication. The API requests adhere to a standard OAuth2 request format. You need a Service Principal and a valid JSON Web Token (JWT) to use this API. Service Principals are identities, represented as code, used by created applications, services, and automation tools to access specific resources. See Cisco Observability Platform Authentication.
HTTP Methods Allowed for Different User Roles
The users can use the following HTTP request methods for the API based on their Cisco Observability Platform Tenant roles:
Role | HTTP Methods Allowed |
---|---|
Configuration Manager | GET, PATCH, POST, PUT, and DELETE |
Troubleshooter | GET, PATCH, POST, PUT, and DELETE |
Observer | GET |
For more information about the Cisco Observability Platform Tenant Roles, see Assign Roles for Cisco Observability Platform Tenants.
Sample Codes
Create a Health Rule
Resource URL:
CopyPOST https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules
Scenario 1
Create a health rule for a pod.
Sample Request:
Copycurl --location --request POST 'https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules' \
--header 'Authorization: Bearer <access-token>' \
--header 'Content-Type: application/json' \--data-raw '
{
"name": "Pod HR",
"description": "Health rule monitoring pod's memory usage",
"enabled": false,
"scheduleName": "Always",
"waitTimeAfterViolation": "5m",
"evaluationObjects":{
"topologyExpression": "entities(k8s:pod)[attributes(k8s.pod.name) in ['sample-pod'] && attributes(k8s.cronjob.name) in ['image-pruner','nost-ston-cj']]"
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Critical",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].max() > 95",
"evaluateToTrueOnNoData": true
}]
},
"warningCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Warning",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].min() > 85",
"evaluateToTrueOnNoData": true
}]
}
}'
Sample Output for 201 (Success) Response:
Copy{
"name": "Pod HR",
"description": "Health rule monitoring pod's memory usage",
"enabled": false,
"scheduleName": "Always",
"waitTimeAfterViolation": "5m",
"evaluationObjects":{
"topologyExpression": "entities(k8s:pod)[attributes(k8s.pod.name) in ['sample-pod'] && attributes(k8s.cronjob.name) in ['image-pruner','nost-ston-cj']]"
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Critical",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].max() > 95",
"evaluateToTrueOnNoData": true
}]
},
"warningCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Warning",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].min() > 85",
"evaluateToTrueOnNoData": true
}]
},
"entityType": "k8s:pod",
"createdAt": "2023-06-29T06:32:40Z",
"updatedAt": "2022-06-29T06:32:40Z",
"id": "6c2661b3-4842-40ed-a97b-882a2aa6a7cc"
}
Scenario 2
You can enter the application_name
in the metadata.applications
parameter to create a health rule for the pod for the application_name
.
Sample Request:
Copycurl --location --request POST 'https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules' \
--header 'Authorization: Bearer <access-token>' \
--header 'Content-Type: application/json' \--data-raw '
{
"name": "Pod HR",
"description": "Health rule monitoring pod's memory usage",
"enabled": false,
"scheduleName": "Always",
"waitTimeAfterViolation": "5m",
"metadata": {
"applications": [
"app-cco"
]
}
"evaluationObjects":{
"topologyExpression": "entities(k8s:pod)[attributes(k8s.pod.name) in ['sample-pod'] && attributes(k8s.cronjob.name) in ['image-pruner','nost-ston-cj']]"
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Critical",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].max() > 95",
"evaluateToTrueOnNoData": true
}]
},
"warningCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Warning",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].min() > 85",
"evaluateToTrueOnNoData": true
}]
}
}'
Sample Output for 201 (Success) Response:
Copy{
"name": "Pod HR",
"description": "Health rule monitoring pod's memory usage",
"enabled": false,
"scheduleName": "Always",
"waitTimeAfterViolation": "5m",
"metadata": {
"applications": [
"app-cco"
]
},
"evaluationObjects":{
"topologyExpression": "entities(k8s:pod)[attributes(k8s.pod.name) in ['sample-pod'] && attributes(k8s.cronjob.name) in ['image-pruner','nost-ston-cj']]"
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Critical",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].max() > 95",
"evaluateToTrueOnNoData": true
}]
},
"warningCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Warning",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].min() > 85",
"evaluateToTrueOnNoData": true
}]
},
"entityType": "k8s:pod",
"createdAt": "2023-06-29T06:32:40Z",
"updatedAt": "2022-06-30T04:32:40Z",
"id": "6c2661b3-4842-40ed-a97b-882a2aa6a7cc"
}
Update a Health Rule
Resource URL:
CopyPUT https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules/{healthRuleId}
Sample Request:
Copycurl --location --request POST 'https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules' \
--header 'Authorization: Bearer <access-token>' \
--header 'Content-Type: application/json' \--data-raw '
{
"id": "{healthRuleId}",
"name": "Pod HR",
"description": "Health rule monitoring pod's memory usage",
"enabled": false,
"scheduleName": "Always",
"waitTimeAfterViolation": "5m",
"evaluationObjects":{
"topologyExpression": "entities(k8s:pod)[attributes(k8s.pod.name) in ['sample-pod'] && attributes(k8s.cronjob.name) in ['image-pruner','nost-ston-cj']]"
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Critical",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].max() > 95",
"evaluateToTrueOnNoData": true
}]
},
"warningCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Warning",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].min() > 85",
"evaluateToTrueOnNoData": true
}]
}
}'
Sample Output for 201 (Success) Response:
Copy{
"name": "Pod HR",
"description": "Health rule monitoring pod's memory usage",
"enabled": false,
"scheduleName": "Always",
"waitTimeAfterViolation": "5m",
"evaluationObjects":{
"topologyExpression": "entities(k8s:pod)[attributes(k8s.pod.name) in ['sample-pod'] && attributes(k8s.cronjob.name) in ['image-pruner','nost-ston-cj']]"
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Critical",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].max() > 95",
"evaluateToTrueOnNoData": true
}]
},
"warningCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Warning",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].min() > 85",
"evaluateToTrueOnNoData": true
}]
},
"entityType": "k8s:pod",
"createdAt": "2023-06-29T06:32:40Z",
"updatedAt": "2022-06-30T04:32:40Z",
"id": "6c2661b3-4842-40ed-a97b-882a2aa6a7cc"
}
Retrieve a Health Rule
Resource URL:
CopyGET https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules/{healthRuleId}
Sample Request:
Copycurl --location --request GET 'https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules/{healthRuleId}' \
--header 'Authorization: Bearer <access-token>'
Sample Output for 200 (Success) Response:
Copy{
"name": "Pod HR",
"description": "Health rule monitoring pod's memory usage",
"enabled": false,
"scheduleName": "Always",
"waitTimeAfterViolation": "5m",
"evaluationObjects":{
"topologyExpression": "entities(k8s:pod)[attributes(k8s.pod.name) in ['sample-pod'] && attributes(k8s.cronjob.name) in ['image-pruner','nost-ston-cj']]"
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Critical",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].max() > 95",
"evaluateToTrueOnNoData": true
}]
},
"warningCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition Memory Usage Warning",
"label": "A",
"conditionExpression": "metrics('k8s:memory.usage','infra-agent',)[timestamp > now - 2m].min() > 85",
"evaluateToTrueOnNoData": true
}]
},
"entityType": "k8s:pod",
"createdAt": "2023-06-29T06:32:40Z",
"updatedAt": "2022-06-30T04:32:40Z",
"id": "6c2661b3-4842-40ed-a97b-882a2aa6a7cc"
}
Retrieve All Health Rules
Resource URL:
CopyGET 'https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules
Scenario 1
Get all health rules for a tenant with a limit of 2.
Sample Request:
Copycurl --location --request GET 'https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules?max=2' \
--header 'Authorization: Bearer <access-token>'
Sample Output for 200 (Success) Response:
Copy{
"items": [{
"name": "health rule 1",
"description": "Health rule one",
"enabled": true,
"scheduleName": "Always",
"waitTimeAfterViolation": "30m",
"evaluationObjects": {
"topologyExpression" : "entities(apm:service)"
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition 1",
"label": "A",
"conditionExpression": "metrics('apm:calls_min','sys:derived')[timestamp > now - 30m].value() > 10",
"evaluateToTrueOnNoData": false
}]
},
"entityType": "apm:service",
"createdAt": "2023-06-22T15:18:10Z",
"updatedAt": "2023-06-29T09:01:54Z",
"id": "6c2661b3-4842-40ed-a97b-882a2aa6a7cc"
}, {
"name": "health rule 2",
"description": "Health rule 2",
"enabled": true,
"scheduleName": "Always",
"waitTimeAfterViolation": "30m",
"evaluationObjects": {
"topologyExpression" : "entities(apm:service)"
},
"warningCriteria": {
"warningExpression": "A",
"conditions": [{
"name": "Condition 1",
"label": "A",
"conditionExpression": "metrics('apm:response_time','sys:derived').value() + metrics('sys:derived','apm:errors_min').value()[timestamp > now - 30m] > 5",
"evaluateToTrueOnNoData": false
}]
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition 1",
"label": "A",
"conditionExpression": "metrics('apm:response_time','sys:derived')+ metrics('sys:derived','apm:errors_min')[timestamp > now - 30m].value() > 10",
"evaluateToTrueOnNoData": false
}]
},
"entityType": "apm:service",
"createdAt": "2022-09-21T13:45:07Z",
"updatedAt": "2022-09-29T09:01:54Z",
"id": "8be1f1e1-7f8e-4132-9125-99dd801971f1"
}]
}
Scenario 2
Get health rules with matching IDs for a tenant.
Sample Request:
Copycurl --location --request GET 'https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules?filter=URL_ENCODING(id eq "hrId1" or id eq "hrId2")' \
--header 'Authorization: Bearer <access-token>'
Note
The filter by IDs are only supported with the Equal (
eq
) operator and OR (or
) as the logical operators. For example id eq "hrId1" or id eq "hrId2".
Sample Output for 200 (Success) Response:
Copy{
"items": [{
"name": "health rule 1",
"description": "Health rule one",
"enabled": true,
"scheduleName": "Always",
"waitTimeAfterViolation": "30m",
"evaluationObjects": {
"topologyExpression" : "entities(apm:service)"
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition 1",
"label": "A",
"conditionExpression": "metrics('apm:calls_min','sys:derived')[timestamp > now - 30m].value() > 10",
"evaluateToTrueOnNoData": false
}]
},
"entityType": "apm:service",
"createdAt": "2023-06-22T15:18:10Z",
"updatedAt": "2023-06-29T09:01:54Z",
"id": "6c2661b3-4842-40ed-a97b-882a2aa6a7cc"
}, {
"name": "health rule 2",
"description": "Health rule 2",
"enabled": true,
"scheduleName": "Always",
"waitTimeAfterViolation": "30m",
"evaluationObjects": {
"topologyExpression" : "entities(apm:service)"
},
"warningCriteria": {
"warningExpression": "A",
"conditions": [{
"name": "Condition 1",
"label": "A",
"conditionExpression": "metrics('apm:response_time','sys:derived').value() + metrics('sys:derived','apm:errors_min').value()[timestamp > now - 30m] > 5",
"evaluateToTrueOnNoData": false
}]
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition 1",
"label": "A",
"conditionExpression": "metrics('apm:response_time','sys:derived')+ metrics('sys:derived','apm:errors_min')[timestamp > now - 30m].value() > 10",
"evaluateToTrueOnNoData": false
}]
},
"entityType": "apm:service",
"createdAt": "2022-09-21T13:45:07Z",
"updatedAt": "2022-09-29T09:01:54Z",
"id": "8be1f1e1-7f8e-4132-9125-99dd801971f1"
}]
}
Scenario 3
You can pass the metadata.applications co "<application_name>"
in the filter parameter to fetch all health rules for the application_name
.
Sample Request:
Copycurl --location --request GET 'https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules?max=2&filter=metadata.applications co "app-cco"' \
--header 'Authorization: Bearer <access-token>
Sample Output for 201 (Success) Response:
Copy{
"items": [{
"name": "health rule 1",
"description": "Health rule one",
"enabled": true,
"scheduleName": "Always",
"waitTimeAfterViolation": "30m",
"metadata": {
"applications": [
"app-cco"
]
},
"evaluationObjects": {
"topologyExpression" : "entities(apm:service)"
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition 1",
"label": "A",
"conditionExpression": "metrics('apm:calls_min','sys:derived')[timestamp > now - 30m].value() > 10",
"evaluateToTrueOnNoData": false
}]
},
"entityType": "apm:service",
"createdAt": "2023-06-22T15:18:10Z",
"updatedAt": "2023-06-29T09:01:54Z",
"id": "6c2661b3-4842-40ed-a97b-882a2aa6a7cc"
}, {
"name": "health rule 2",
"description": "Health rule 2",
"enabled": true,
"scheduleName": "Always",
"waitTimeAfterViolation": "30m",
"metadata": {
"applications": [
"app-cco"
]
},
"evaluationObjects": {
"topologyExpression" : "entities(apm:service)"
},
"warningCriteria": {
"warningExpression": "A",
"conditions": [{
"name": "Condition 1",
"label": "A",
"conditionExpression": "metrics('apm:response_time','sys:derived').value() + metrics('sys:derived','apm:errors_min').value()[timestamp > now - 30m] > 5",
"evaluateToTrueOnNoData": false
}]
},
"criticalCriteria": {
"criteriaExpression": "A",
"conditions": [{
"name": "Condition 1",
"label": "A",
"conditionExpression": "metrics('apm:response_time','sys:derived')+ metrics('sys:derived','apm:errors_min')[timestamp > now - 30m].value() > 10",
"evaluateToTrueOnNoData": false
}]
},
"entityType": "apm:service",
"createdAt": "2022-09-21T13:45:07Z",
"updatedAt": "2022-09-29T09:01:54Z",
"id": "8be1f1e1-7f8e-4132-9125-99dd801971f1"
}]
}
Delete a Health Rule
Resource URL:
CopyDELETE https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules/{healthRuleId}
Sample Request:
Copycurl --location --request DELETE 'https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules/{healthRuleId}' \
--header 'Authorization: Bearer <access-token>'
Enable Health Rules
Resource URL:
CopyPUT https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules/enable
Sample Request:
Copycurl --location --request PUT 'https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules/enable' \
--header 'Authorization: Bearer <access-token>' \
--header 'Content-Type: application/json' \
--data-raw ' {
"healthRuleIds": ["< health rule id 1 >", "< health rule id 2 >"]
}'
Sample Output for 200 (Success) Response:
Copy
{
"updateResults": {
"< health rule id 1 >": {
"status": "SUCCESS"
},
"< health rule id 1 >": {
"status": "SUCCESS"
}
}
}
Disable Health Rules
Resource URL:
CopyPUT https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules/disable
Sample Request:
Copycurl --location --request PUT 'https://<tenant-name>.observe.appdynamics.com/alerting/v1beta2/healthRules/disable' \
--header 'Authorization: Bearer <access-token>' \
--header 'Content-Type: application/json' \
--data-raw ' {
"healthRuleIds": ["< health rule id 1 >", "< health rule id 2 >"]
}'
Sample Output for 200 (Success) Response:
Copy{
"updateResults": {
"< health rule id 1 >": {
"status": "SUCCESS"
},
"< health rule id 1 >": {
"status": "SUCCESS"
}
}
}
Sample Health Rule
The following example is a health rule that shows all the possible scenarios:
Copy{
"name":"sample HR",
"description":"sample HR",
"scheduleName":"Always",
"enabled":true,
"waitTimeAfterViolation":"45m",
"evaluationObjects":{
"topologyExpression":"entities(k8s:cluster).out.to(k8s:namespace)[attributes(k8s.namespace.name) = 'namespace-1'].out.to(k8s:deployment).out.to(k8s:pod)[attributes(k8s.pod.name) = 'test-pod' && attributes(k8s.cronjob.name) IN ['test-cronjob'] && attributes(k8s.pod.has_liveness_probe) != false]",
"evaluationEntityType":"k8s:pod"
},
"rollupHealthConfig":{
"rollupTo":"k8s:deployment",
"criteria":{
"threshold":"30",
"thresholdType":"PERCENTAGE"
},
"rollupHealthConfig":{
"rollupTo":"k8s:namespace",
"criteria":{
"threshold":"2",
"thresholdType":"COUNT"
},
"rollupHealthConfig":{
"rollupTo":"k8s:cluster",
"criteria":{
"threshold":"1",
"thresholdType":"COUNT"
}
}
}
},
"criticalCriteria":{
"criteriaExpression":"(A OR B) AND (C OR D)",
"conditions":[
{
"name":"Condition 1",
"label":"A",
"evaluateToTrueOnNoData":true,
"conditionExpression":"metrics('k8s:cpu.limits', 'infra-agent')[timestamp > (now - 30m)].value() > 30"
},
{
"name":"Condition 2",
"label":"B",
"evaluateToTrueOnNoData":false,
"conditionExpression":"metrics('k8s:cpu.usage', 'sys:derived')[timestamp > (now - 30m) && max > 3].count() > 15"
},
{
"name":"Condition 3",
"label":"C",
"evaluateToTrueOnNoData":false,
"conditionExpression":"metrics('k8s:cpu.requests', 'infra-agent')[timestamp > (now - 30m)].min() between metrics('k8s:cpu.requests', 'infra-agent').baseline('Daily Trend - Last 30 days').stdDevRange(1)"
},
{
"name":"Condition 4",
"label":"D",
"evaluateToTrueOnNoData":false,
"conditionExpression":"metrics('k8s:cpu.requests', 'infra-agent')[timestamp > (now - 30m) && value > metrics('k8s:cpu.requests', 'infra-agent').baseline('All data - Last 15 days').percentageRange(50)].count() > 20"
}
]
},
"warningCriteria":{
"criteriaExpression":"A OR B AND C",
"conditions":[
{
"name":"Condition 1",
"label":"A",
"evaluateToTrueOnNoData":true,
"conditionExpression":"(metrics('k8s:memory.usage', 'sys:derived').value()/100 + metrics('dynamicmetrics:k8s.pod.restarts', 'infra-agent').sum())[timestamp > (now - 30m)] > 0"
},
{
"name":"Condition 2",
"label":"B",
"evaluateToTrueOnNoData":false,
"conditionExpression":"(metrics('k8s:cpu.usage', 'infra-agent').max()/metrics('k8s:cpu.limits', 'infra-agent').min())[timestamp > (now - 30m) && value < 100].count() > 10"
},
{
"name":"Condition 3",
"label":"C",
"evaluateToTrueOnNoData":false,
"conditionExpression":"(metrics('k8s:memory.usage', 'sys:derived')/100 + metrics('dynamicmetrics:k8s.pod.restarts', 'infra-agent'))[timestamp > (now - 30m)].value() > 0"
},
]
}
}