This documentation and the Cisco Observability Platform functionalities it describes are subject to change. Data saved on the platform may disappear and APIs may change without notice.
Spans and Traces
A trace represents a request or a transaction made by a user or an application. It is a collection of operations that represents a unique transaction handled by an application and its constituent services. In a multi-service architecture, as the request traverses through multiple services to serve a user need, a trace records all the paths taken by the request. It provides a holistic view of a transaction or a request.
A trace consists of multiple spans, which can run in parallel or in a linear sequence. A span represents a single operation within a trace. It indicates the operation performed by an individual service or a component in the process of fulfilling the user request. In other words, a span represents each unit of work within a trace. A trace represents the complete process of a request, from the beginning to the end of a request lifecycle.
Span is of two types- root span and child span. A trace contains a single root span and multiple child spans. The root span measures the end-to-end latency of an entire request. For example, the root span calculates the total time required to complete a request, from its initiation to completion. Child spans represent function calls, calls to other microservices, a call to an authentication API, and so on. It provides granular visibility into each part of the request. Spans can delegate processes to other spans. In case a span fails to execute the assigned process, it may lead to the failure of the trace itself. A tracing system tracks all the traces and its spans. It assigns a unique ID to a trace and segregates the traces into spans. The traces and spans together provide the status of the processes involved in an operation. Analyzing all the spans in a trace, you get insights into how the request is performed across its lifecycle.
A span provides the following information:
| Key | Description |
|---|---|
entityId |
The unique identifier of the entity that produced this span. |
traceId |
The unique identifier of the associated trace. It is used to group all spans of a specific trace. |
spanId |
The unique identifier of the span. |
parentId (optional) |
The unique identifier of the parent span. |
name |
The work represented by the span. For example, an RPC method name, a function name, or the name of a subtask or stage within a larger computation. |
spanKind |
The type of a span. It can be one of the following types: |
Internal: Indicates that the span represents an internal operation within an application, as opposed to an operation happening at the boundaries. |
|
Server: Indicates that the span covers server-side handling of remote procedure call or other remote network request. |
|
Producer: Indicates that the span describes a producer sending a message to a broker. Unlike client and server, there is often no direct critical path latency relationship between producer and consumer spans. A producer span ends when the message is accepted by the broker while the logical processing of the message might take a much longer time. |
|
Consumer: Indicates that the span describes a consumer receiving a message from a broker. Like a Producer kind, there is often no direct critical path latency relationship between producer and consumer spans. |
|
derivedEntityIds (optional) |
A set of unique identifiers of the entities that are derived from the span. |
startedAt |
The start time stamp of the span. |
endedAt |
The end time stamp of the span. |
attributes |
A list of one or more key-value pairs. For example Client IP, HTTP Method, and Status Code. |
events |
A set of one or more events. Each event consists of timestamp, names , and attributes. |
links |
Links to one or more related spans. |
statusCode |
One of the following: OK or Error |
errorMessage |
The message related to an error encountered. |
tags (optional) |
A list of tags. |
source |
The source of the span. |
Example of a trace and its spans (including root span and child spans):
[
{
"traceId": "e04c1be394978541f3140f96e0bab8da",
"parentId": "f8c16aec7bfa529f",
"id": "d9aea47590756ece",
"kind": "CLIENT",
"name": "http get",
"timestamp": 1598909716012348,
"duration": 14784,
"localEndpoint": {
"serviceName": "service2",
"ipv4": "192.xx.xx.xx"
},
"tags": {
"http.method": "GET",
"http.status_code": "200",
"http.url": "<your local host url>",
"net.peer.name": "localhost",
"net.peer.port": "xxxx"
}
},
{
"traceId": "e04c1be394978541f3140f96e0bab8da",
"parentId": "b6c7da87a2731274",
"id": "f8c16aec7bfa529f",
"name": "transactions.demo",
"timestamp": 1598909715997576,
"duration": 30365,
"localEndpoint": {
"serviceName": "service2",
"ipv4": "192.xx.xx.xx"
}
},
{
"traceId": "e04c1be394978541f3140f96e0bab8da",
"parentId": "2bcc1e23bd7b56d4",
"id": "b6c7da87a2731274",
"kind": "SERVER",
"name": "get /txn/demo/{count}",
"timestamp": 1598909715995002,
"duration": 53508,
"localEndpoint": {
"serviceName": "service2",
"ipv4": "192.xx.xx.xx"
},
"annotations": [
{
"timestamp": 1598909716047596,
"value": "exception"
}
],
"tags": {
"http.client_ip": "127.xx.xx.xx",
"http.flavor": "HTTP/1.1",
"http.method": "GET",
"http.response_content_length": "17",
"http.status_code": "200",
"http.url": "<your local host url",
"http.user_agent": "Apache-HttpClient/4.3.4 (java 1.5)",
"net.peer.ip": "127.xx.xx.xx",
"net.peer.port": "56325",
"servlet.path": ""
}
},
{
"traceId": "e04c1be394978541f3140f96e0bab8da",
"parentId": "f082d70f03f091fd",
"id": "b8e3559c6b1282bc",
"kind": "CLIENT",
"name": "http get",
"timestamp": 1598909715926428,
"duration": 124102,
"localEndpoint": {
"serviceName": "service2",
"ipv4": "192.xxx.x.xx"
},
"tags": {
"http.method": "GET",
"http.status_code": "200",
"http.url": "<your local host url",
"net.peer.name": "localhost",
"net.peer.port": "xxxx"
}
},
{
"traceId": "e04c1be394978541f3140f96e0bab8da",
"parentId": "8d8f25b0ff8c3c5f",
"id": "f082d70f03f091fd",
"name": "transactions.demo",
"timestamp": 1598909715892270,
"duration": 159186,
"localEndpoint": {
"serviceName": "service2",
"ipv4": "192.xxx.x.xx"
}
},
{
"traceId": "e04c1be394978541f3140f96e0bab8da",
"parentId": "059586ca6e169128",
"id": "05d31ee89db9e2a8",
"name": "transactions.demo",
"timestamp": 1598909716018497,
"duration": 84,
"localEndpoint": {
"serviceName": "service1",
"ipv4": "192.xxx.x.xx"
}
},
{
"traceId": "e04c1be394978541f3140f96e0bab8da",
"parentId": "119a8f24f8f3cd08",
"id": "1667f6cf4bb9807f",
"name": "transactions.demo",
"timestamp": 1598909715969072,
"duration": 78839,
"localEndpoint": {
"serviceName": "service1",
"ipv4": "192.xxx.x.xx"
}
},
{
"traceId": "e04c1be394978541f3140f96e0bab8da",
"parentId": "b8e3559c6b1282bc",
"id": "119a8f24f8f3cd08",
"kind": "SERVER",
"name": "get /txn/demo/{count}",
"timestamp": 1598909715951019,
"duration": 99577,
"localEndpoint": {
"serviceName": "service1",
"ipv4": "192.xxx.x.xx"
},
"annotations": [
{
"timestamp": 1598909716050322,
"value": "exception"
}
],
"tags": {
"http.client_ip": "127.x.x.x",
"http.flavor": "HTTP/1.1",
"http.method": "GET",
"http.response_content_length": "17",
"http.status_code": "200",
"http.url": "<your local host url>",
"http.user_agent": "Apache-HttpClient/4.3.4 (java 1.5)",
"net.peer.ip": "127.x.x.x",
"net.peer.port": "xxxx",
"servlet.path": ""
}
}
]