Documentation for Metrics collection
In EI agent release 1.12.0, a metrics collection feature is provided.
Background:
- The metrics collection process will have all the resource utilization information for the application, i.e., the application's CPU, memory, network utilization, and much more.
- A CSV file with the name "metrics_collect.csv" will be used to store this data in a comma-separated format.
- Every day after 11:59 p.m., the current file will be compressed and the above-mentioned CSV file will be created.
- The application stores all the files for seven days. The oldest compressed file will be removed after seven days and replaced with a new file.
- The metrics collect link updates the csv file every five minutes with a new metric row entry that includes all of the column values. This can also be configure to a different value; it is the default time period.
Fields in metrics_collect.csv file:
The fields listed below are the file's headers:
| Headers | Description |
|---|---|
| ts(epoch) | This field provides the time stamp in seconds elapsed from January 1, 1970, i.e., in epoch time. |
| overall_cpu_usage(%) | This section displays the percentage of the load that each processor core can handle when running different applications by completing IOX. |
| cpu_load_avg_1_min | This field provides the CPU's average load over a 1-minute period for the whole IOX. |
| cpu_load_avg_5_min | This field provides the CPU's average load over a 5-minute period for the whole IOX. |
| cpu_load_avg_10_min | This field provides the CPU's average load over a 10-minute period for the whole IOX. |
| azure_fd_cnt | This field is a non-negative integer, displays how many files the azure process is currently utilising. The developers will use this field to understand system performance. For example, fd_cnt would be 10 if Azure process opens 10 different files. |
| broker_fd_cnt | This field is a non-negative integer, displays how many files the broker process is currently utilising. The developers will use this field to understand system performance. For example, fd_cnt would be 10 if broker process opens 10 different files. |
| normalization_fd_cnt | This field is a non-negative integer, displays how many files the normalization process is currently utilising. The developers will use this field to understand system performance. For example, fd_cnt would be 10 if normalization process opens 10 different files. |
| mqtt_fd_cnt | This field is a non-negative integer, displays how many files the MQTT process is currently utilising. The developers will use this field to understand system performance. For example, fd_cnt would be 10 if MQTT process opens 10 different files. |
| opcua_fd_cnt | This field is a non-negative integer, displays how many files the OPCUA process is currently utilising. The developers will use this field to understand system performance. For example, fd_cnt would be 10 if OPCUA process opens 10 different files. |
| modbus_fd_cnt | This field is a non-negative integer, displays how many files the modbus process is currently utilising. The developers will use this field to understand system performance. For example, fd_cnt would be 10 if modbus process opens 10 different files. |
| serial_fd_cnt | This field is a non-negative integer, displays how many files the serial process is currently utilising. The developers will use this field to understand system performance. For example, fd_cnt would be 10 if serial process opens 10 different files. |
| rsu_fd_cnt | This field is a non-negative integer, displays how many files the RSU process is currently utilising. The developers will use this field to understand system performance. For example, fd_cnt would be 10 if RSU process opens 10 different files. |
| eip_fd_cnt | This field is a non-negative integer, displays how many files the EIP process is currently utilising. The developers will use this field to understand system performance. For example, fd_cnt would be 10 if EIP process opens 10 different files. |
| ntcip_fd_cnt | This field is a non-negative integer, displays how many files the NTCIP process is currently utilising. The developers will use this field to understand system performance. For example, fd_cnt would be 10 if NTCIP process opens 10 different files. |
| metrics_fd_cnt | This field is a non-negative integer, displays how many files the metrics process is currently utilising. The developers will use this field to understand system performance. For example, fd_cnt would be 10 if metrics process opens 10 different files. |
| azure_pid | PID is a process ID in general. This field indicates an Azure process and is an unique, non-negative integer. The developers will use this field to understand operational aspects of processes related to Azure. |
| broker_pid | PID is a process ID in general. This field indicates an broker process and is an unique, non-negative integer. The developers will use this field to understand operational aspects of processes related to broker. |
| normalization_pid | PID is a process ID in general. This field indicates an normalization process and is an unique, non-negative integer. The developers will use this field to understand operational aspects of processes related to normalization. |
| mqtt_pid | PID is a process ID in general. This field indicates an MQTT process and is an unique, non-negative integer. The developers will use this field to understand operational aspects of processes related to MQTT. |
| opcua_pid | PID is a process ID in general. This field indicates an OPCUA process and is an unique, non-negative integer. The developers will use this field to understand operational aspects of processes related to OPCUA. |
| modbus_pid | PID is a process ID in general. This field indicates an modbus process and is an unique, non-negative integer. The developers will use this field to understand operational aspects of processes related to modbus. |
| serial_pid | PID is a process ID in general. This field indicates an serial process and is an unique, non-negative integer. The developers will use this field to understand operational aspects of processes related to serial. |
| rsu_pid | PID is a process ID in general. This field indicates an RSU process and is an unique, non-negative integer. The developers will use this field to understand operational aspects of processes related to RSU. |
| eip_pid | PID is a process ID in general. This field indicates an EIP process and is an unique, non-negative integer. The developers will use this field to understand operational aspects of processes related to EIP. |
| ntcip_pid | PID is a process ID in general. This field indicates an NTCIP process and is an unique, non-negative integer. The developers will use this field to understand operational aspects of processes related to NTCIP. |
| metrics_pid | PID is a process ID in general. This field indicates an metrics process and is an unique, non-negative integer. The developers will use this field to understand operational aspects of processes related to metrics. |
| azure_res(kB) | This field provides the actual RAM size that the Azure process allotted in kilobytes out of the total IOX RAM. |
| broker_res(kB) | This field provides the actual RAM size that the broker process allotted in kilobytes out of the total IOX RAM. |
| normalization_res(kB) | This field provides the actual RAM size that the normalization process allotted in kilobytes out of the total IOX RAM. |
| mqtt_res(kB) | This field provides the actual RAM size that the MQTT process allotted in kilobytes out of the total IOX RAM. |
| opcua_res(kB) | This field provides the actual RAM size that the OPCUA process allotted in kilobytes out of the total IOX RAM. |
| modbus_res(kB) | This field provides the actual RAM size that the modbus process allotted in kilobytes out of the total IOX RAM. |
| serial_res(kB) | This field provides the actual RAM size that the serial process allotted in kilobytes out of the total IOX RAM. |
| rsu_res(kB) | This field provides the actual RAM size that the RSU process allotted in kilobytes out of the total IOX RAM. |
| eip_res(kB) | This field provides the actual RAM size that the EIP process allotted in kilobytes out of the total IOX RAM. |
| nctip_res(kB) | This field provides the actual RAM size that the NTCIP process allotted in kilobytes out of the total IOX RAM. |
| metrics_res(kB) | This field provides the actual RAM size that the metrics process allotted in kilobytes out of the total IOX RAM. |
| azure_vss(kB) | The virtual memory size allotted to the Azure process is specified in this field in kilobytes. It represents the total amount of memory that a process could possibly access. The developers will use this field to understand system performance. |
| broker_vss(kB) | The virtual memory size allotted to the broker process is specified in this field in kilobytes. It represents the total amount of memory that a process could possibly access. The developers will use this field to understand system performance. |
| normalization_vss(kB) | The virtual memory size allotted to the normalization process is specified in this field in kilobytes. It represents the total amount of memory that a process could possibly access. The developers will use this field to understand system performance. |
| mqtt_vss(kB) | The virtual memory size allotted to the MQTT process is specified in this field in kilobytes. It represents the total amount of memory that a process could possibly access. The developers will use this field to understand system performance. |
| opcua_vss(kB) | The virtual memory size allotted to the OPCUA process is specified in this field in kilobytes. It represents the total amount of memory that a process could possibly access. The developers will use this field to understand system performance. |
| modbus_vss(kB) | The virtual memory size allotted to the modbus process is specified in this field in kilobytes. It represents the total amount of memory that a process could possibly access. The developers will use this field to understand system performance. |
| serial_vss(kB) | The virtual memory size allotted to the serial process is specified in this field in kilobytes. It represents the total amount of memory that a process could possibly access. The developers will use this field to understand system performance. |
| rsu_vss(kB) | The virtual memory size allotted to the RSU process is specified in this field in kilobytes. It represents the total amount of memory that a process could possibly access. The developers will use this field to understand system performance. |
| eip_vss(kB) | The virtual memory size allotted to the EIP process is specified in this field in kilobytes. It represents the total amount of memory that a process could possibly access. The developers will use this field to understand system performance. |
| nctip_vss(kB) | The virtual memory size allotted to the NTCIP process is specified in this field in kilobytes. It represents the total amount of memory that a process could possibly access. The developers will use this field to understand system performance. |
| metrics_vss(kB) | The virtual memory size allotted to the metrics process is specified in this field in kilobytes. It represents the total amount of memory that a process could possibly access. The developers will use this field to understand system performance. |
| azure_cpu(%) | This field provides the percentage of the IOX's CPU that Azure processes are utilising. |
| broker_cpu(%) | This field provides the percentage of the IOX's CPU that broker processes are utilising. |
| normalization_cpu(%) | This field provides the percentage of the IOX's CPU that normalization processes are utilising. |
| mqtt_cpu(%) | This field provides the percentage of the IOX's CPU that MQTT processes are utilising. |
| opcua_cpu(%) | This field provides the percentage of the IOX's CPU that OPCUA processes are utilising. |
| modbus_cpu(%) | This field provides the percentage of the IOX's CPU that modbus processes are utilising. |
| serial_cpu(%) | This field provides the percentage of the IOX's CPU that seial processes are utilising. |
| rsu_cpu(%) | This field provides the percentage of the IOX's CPU that RSU processes are utilising. |
| eip_cpu(%) | This field provides the percentage of the IOX's CPU that EIP processes are utilising. |
| ntcip_cpu(%) | This field provides the percentage of the IOX's CPU that NTCIP processes are utilising. |
| metrics_cpu(%) | This field provides the percentage of the IOX's CPU that metrics processes are utilising. |
| rx(bytes/s) | This field shows the total number of bytes the EI agent application received each second via the network. |
| tx(bytes/s) | This field shows the total number of bytes the EI agent application transmitted each second via the network. |
| iowait(%) | This field provides the total wait time of CPU in percentage for whole IOX. This field will be used by the developers to understand system performance. |
| read_per_second(Blk_read/s) | This field provides the total blocks read per second for the EI application data-mounted drive. |
| write_per_second(Blk_wrtn/s) | This field provides the total blocks read per second for EI application data-mounted drive. |
| available_memory(kB) | This field provides the total RAM memory available in kilobytes out of IOX's entire RAM. |
| available_diskspace(kB) | This field provides the total disc space available for the root ("/") directory in kilobytes for IOX. |
| cpu_cores | This field will provide the total CPU cores present in the IOX. |
| cpu_usage_per_core0..n | This field will provide the CPU utilization per core present in IOX. There will be different entries for each CPU core utilization. |
Important Points:
- The metrics collection process likewise starts and ends simultaneously with the EI agent application.
- According to the above table, the first row of the CSV files consists of headers; the following rows will have data that corresponds to the header.
- The values are added to the CSV file on a periodic basis. The configuration of the dslink.json file for the metrics_collect process defines the period.
- The range of supported configuration values is from 1 minute to 1440 minutes, or 24 hours.
- The process allows for configuration in decimal values as well; for example, 1.5 minute stands for 1 minute and 30 seconds.
- If the metric collection process is configured with an "a" value in minutes and if it gets updated by "b", then a new configuration value will be in effect after the time "a" has elapsed. For example, if a configuration for 10 minutes is set up initially, then modified with a configuration for 5 minutes, the revised 5 minute configuration will take effect after 10 minutes have passed since the initial setup.
How to set the configured interval for metric collection:
- Login to the IOX local manager.
- Click on EI application manage and navigate to the App-DataDir tab.
- Download the dslink/metrics/dslink.json file.
- Add the necessary minutes value to the metrics-frequency-minute field in the downloaded json file.
- Delete the existing dslink/metrics/dslink.json file.
- Upload the modified dslink.json file to the path dslink/metrics.
- Restart the EI application.
How to download the metric_collection files:
We can download the metrics collection files from two locations.
Using Edge intelligence cloud UI:
- We can only download metric_collection from the EI cloud if the EDM onboards the EI agent.
- Log in to the cloud where the Edge intelligence agent is registered.
- Click on the agent's name from the list of EI Agents.
- Go to the Troubleshooting tab and click on Download EI Agent Log.
- Create a techsupport file, then download it.
- We get Platform_ts.tar.gz after extracting the techsupport file.
- We will get a directory with the same name as a techsupport file when we extract platform_ts.tar.gz.
- Traverse through the directory "tech_support"/logs/applications/"ei_app_name"/metrics_collect.csv.
- In the same directory, beginning with metrics_collect, all compressed files will be present.
Using local IOX manager:
There are two ways to download metrics_collection file from local IOX manager.- Login to the local IOX manager and click on Manage EI application.
- First way: Go to logs tab, download metrics_collect.csv and also all metrics_collect compressed files.
- Second way: Go to System Troubleshoot tab, click on Generate snapshot file.
- Download the newly generated techsupport file.
- We get Platform_ts.tar.gz after extracting the techsupport file.
- We will get a directory with the same name as a techsupport file when we extract platform_ts.tar.gz.
- Traverse through the directory "tech_support"/logs/applications/"ei_app_name"/metrics_collect.csv.
- In the same directory, beginning with metrics_collect, all compressed files will be present.