Aggregation Queries

Aggregation queries allow you compute values for a group of entities. They provide a way to determine the:

  • number of nodes in a cluster
  • number of pods that are healthy or unhealthy per namespace
  • number of workloads of different types
  • average response time across all the services
  • events that were produced by all pods in a namespace

A query becomes an aggregation query when an aggregation function is used within the FETCH block.

Supported aggregation functions:

  • count - count the number of entities in a group.
  • Events - fetch events aggregated for entities in the group.
  • metrics - aggregate metrics in group of entities.
  • sources - aggregate metric sources in group of entities.
  • traces - aggregate traces for entities in a group.
  • spans - aggregate spans for entities in a group.

Note Currently, only topology aggregations are supported. This means that you can aggregate entities but cannot aggregate MELT data.

Aggregate Everything

The simplest aggregation query is to count everything.

In the FETCH block, if you ask for a count, the query returns a single number that represents the results that match your FROM clause:

Simplest count aggregation 

FETCH
   count
FROM
   entities(k8s:pod)

Aggregaton of logs from all pods 

FETCH
   Events(common:logs)
FROM
   entities(k8s:pod)

Aggregate and Group

Aggregations can be further divided using topology observations in the FETCH block along with aggregation functions. Such observations are called grouping keys in an aggregation query.

Example of grouping on entity attributes

FETCH
  count,
  attributes(region)
FROM
  entities(k8s:cluster)

Grouping keys split the set of all results into separate groups where entities in the same group have the same values for all grouping keys. Aggregation functions are applied on each group individually. Grouping is done on the grouping key values and will result in as many results as there are distinct grouping key values (which may even be 0). This is the same behavior as in SQL.

The previous example query splits all clusters into as many groups as there are distinct regions and then returns a tuple of each region and the number of clusters with the region value.

Add more grouping keys to obtain distinct division of the results.

Example of grouping with multiple keys

FETCH
  count,
  attributes("k8s.cluster.name"),
  attributes("k8s.namespace.name")
FROM
  entities(k8s:pod)

This query returns the number of pods per cluster and namespace. Each resulting tuple contains a cluster name, a namespace name and the count of pods with these cluster and namespace names.

Supported grouping keys:

  • attributes(<attribute-name>) - group by entity attribute
  • tags(<tag-name>) - group by entity tag
  • type - group by entity type
  • id - "groups" contains a single entity. Useful to return metrics, sources, and events per individual entities.

Note

  • Grouping keys are automatically returned as part of the results and cannot be omitted.
  • If you want events, metrics or sources for individual entities you have to separate each entity into its own group. Fetch the entity id to do this.
  • ofType is not supported as a grouping key.
---
caption: Example of spliting on individual entities
emphasize-lines: 3, 4, 5
---
FETCH
  id,
  events(logs:generic_record)
FROM
  entities(k8s:workload)

Note

  • You cannot compare complex types such as all attributes or all tags; this results in each entity being in its own group the same way as if you specified the entity id.
  • In an aggregation query all items in the FETCH block must either be aggregation functions or supported grouping keys.

Result

The result of aggregation queries are tuples containing values for grouping keys and aggregation functions.

Example

The following UQL query:

FETCH
  c: count,
  t: type,
  h: attributes("health.status"),
  r: tags("cloud.region")
FROM
  entities(k8s:workload)

returns:

  • c - a non-negative integer
  • t - namespace:typeName format from AppD schema
  • h - a long (the attribute's type)
  • r - string (tag values are string)
When grouping on attributes or tags not all entities may have a value for a grouping key.
Such entities get counted in tuples that have null values.