Automated Telegraf, InfluxDB and Grafana (TIG) setup via Docker, Python and Jinja templates to collect Model Driven Telemetry (MDT):
This project will help you to deploy a TIG stack in order to leverage quickly the MDT.
git clone https://github.com/xaviervalette/cisco-devnet-mdt-tig
└── cisco-devnet-mdt-tig/
+ ├── config.yml
├── src/
│ └── ...
└── template/
└── ...
config.yml
file, add the following variables:#config.yml --- host: <your_host> username: <your-username> password: <your-password> influxdb_org: <your-influxdb-org> influxdb_bucket: <your-influxdb-bucket> influxdb_token: <your-token> ...
#config.yml --- host: 10.142.78.4 username: admin password: mysuperstrongpassword influxdb_org: valettefamily.com influxdb_bucket: devnet influxdb_token: test-token ...
⚠️ After creating the config.yml
file, you will need to generate the docker-compose.yml
file and the required configuration files. To make it smooth, I've created template with Jinja2
, so you will just have to run the following command:
python3 src/generate-conf.py
The following files should be created:
└── cisco-devnet-mdt-tig/ ├── config.yml ├── src/ │ └── ... └── template/ │ └── ... + ├── docker-compose.yml + ├── telegraf/ + │ └── ... + └── grafana/ + └── ...
cisco-devnet-mdt-tig
, and start the TIG stack:docker-compose up
xvalette@raspberrypi4:~$ cd cisco-devnet-mdt-tig/ xvalette@raspberrypi4:~/cisco-devnet-mdt-tig$ docker-compose up Starting influxdb ... done Starting telegraf ... done Starting grafana ... done Attaching to influxdb, telegraf, grafana ...
Service | InfluxDB GUI | Grafana GUI |
---|---|---|
URL | http://<your-device-IP>:8086 |
http://<your-device-IP>:3000 |
Example | http://10.142.78.4:8086 |
http://10.142.78.4:3000 |
Output |
You can log in using the username
and password
that you define in the config.yml
file (admin:admin
in the example)
You can start creating your own dashboards.
To help you at this stage, I've created ready to use dashboards:
import requests import json from datetime import datetime, timedelta import yaml import time def get_previous_hour_timestamp(): # get the current time now = datetime.now() # subtract an hour from the current time previous_hour = now - timedelta(hours=1) previous_hour = previous_hour.replace(minute=0,second=0, microsecond=0) return(int(previous_hour.timestamp())) def write_data_to_influxdb(host, org, bucket, precision, auth_token, payload): # Create the URL for writing data to the InfluxDB database url = f"http://{host}:8086/api/v2/write?org={org}&bucket={bucket}&precision={precision}" # Set the HTTP headers for the request headers = { "Authorization": f"Token {auth_token}", "Content-Type": "text/plain; charset=utf-8", "Accept": "application/json" } # Make the API request to write the data to the InfluxDB database response = requests.post(url, headers=headers, data=payload) # Return the status code of the API response return response.status_code def get_poe_consumption(): # Open the config.yml file and load its contents into the 'config' variable with open('config.yml', 'r') as file: config = yaml.safe_load(file) # Loop through each network defined in the config file for network in config["meraki"]["networks"]: # Loop through each network defined in the config file for switch in network["devices"]["switches"]: # Get the current timestamp #current_timestamp = int(time.time()) timestamp = get_previous_hour_timestamp() # Create the URL for retrieving all VLANs in the network url = f"https://api.meraki.com/api/v1/devices/{switch}/switch/ports/statuses?timespan=3600" # Set the HTTP headers for the request headers = { "Content-Type": "application/json", "Accept": "application/json", "X-Cisco-Meraki-API-Key": config["meraki"]["api_key"] } # Empty payload payload = {} # Make the API request using the requests library response = requests.get(url, headers=headers, data=json.dumps(payload)) # Print the status code of the response print("\nRequest status code : " + str(response.status_code) + "\n") # Parse the response as JSON responseJson = response.json() total_switch_poe = 0 # Iterate through each port in the response for port in responseJson: # Skip over ports 9 and 10 if port["portId"] not in ["9", "10"]: # Calculate the POE usage for the current port poe_usage = port["powerUsageInWh"] total_switch_poe = total_switch_poe + port["powerUsageInWh"] payload = f'meraki,device={switch} poeUsage={total_switch_poe} {timestamp}' # Print the payload string print(payload) # Write the payload data to the InfluxDB database status_code = write_data_to_influxdb( host=config["influxdb"]["host"], org=config["influxdb"]["org"], bucket=config["influxdb"]["bucket"], precision="s", auth_token=config["influxdb"]["api_key"], payload=payload ) # Print the status code of the write_data() API response print(status_code) def get_clients_usage(): # Open the config.yml file and load its contents into the 'config' variable with open('config.yml', 'r') as file: config = yaml.safe_load(file) # Loop through each network defined in the config file for network in config["meraki"]["networks"]: # Get the current timestamp current_timestamp = int(time.time()) # Create the URL for retrieving all VLANs in the network url = f"https://api.meraki.com/api/v1/networks/{network['network_id']}/clients?timespan=600" print(url) # Set the HTTP headers for the request headers = { "Content-Type": "application/json", "Accept": "application/json", "X-Cisco-Meraki-API-Key": config["meraki"]["api_key"] } # Empty payload payload = {} # Make the API request using the requests library response = requests.get(url, headers=headers, data=json.dumps(payload)) # Print the status code of the response print("\nRequest status code : " + str(response.status_code) + "\n") # Parse the response as JSON responseJson = response.json() print(responseJson) for client in responseJson: # Iterate through each port in the response # Format the payload string with the POE usage data payload = f'meraki,client={client["mac"]} downloadKbytes={client["usage"]["recv"]} {current_timestamp}' # Write the payload data to the InfluxDB database status_code = write_data_to_influxdb( host=config["influxdb"]["host"], org=config["influxdb"]["org"], bucket=config["influxdb"]["bucket"], precision="s", auth_token=config["influxdb"]["api_key"], payload=payload ) payload = f'meraki,client={client["mac"]} uploadKbytes={client["usage"]["sent"]} {current_timestamp}' # Write the payload data to the InfluxDB database status_code = write_data_to_influxdb( host=config["influxdb"]["host"], org=config["influxdb"]["org"], bucket=config["influxdb"]["bucket"], precision="s", auth_token=config["influxdb"]["api_key"], payload=payload ) # Print the status code of the write_data() API response print(status_code)
Example of configuration required on the C9800 to send the expected telemetry:
!
! TRAFFIC STATS
!
telemetry ietf subscription 101
encoding encode-kvgpb
filter xpath /client-oper-data/traffic-stats/bytes-tx
source-address 192.168.1.98
stream yang-push
update-policy periodic 60000
receiver ip address 10.142.78.4 57000 protocol grpc-tcp
!
telemetry ietf subscription 102
encoding encode-kvgpb
filter xpath /client-oper-data/traffic-stats/bytes-rx
source-address 192.168.1.98
stream yang-push
update-policy periodic 60000
receiver ip address 10.142.78.4 57000 protocol grpc-tcp
!
! CLIENTS STATS
!
telemetry ietf subscription 110
encoding encode-kvgpb
filter xpath /wireless-mobility-oper:mobility-oper-data/wlan-client-limit
source-address 192.168.1.98
stream yang-push
update-policy on-change
receiver ip address 10.142.78.4 57000 protocol grpc-tcp
If you want to securely access to your application (InfluxDB, Grafana) from outside your network, you can deploy a Duo Network Gateway (reverse proxy + SAML IDP) → Check my repository Cisco Duo Network Gateway Raspberry PI for more details.
Owner
Contributors
Categories
Programming Languages
PythonLicense
Code Exchange Community
Get help, share code, and collaborate with other developers in the Code Exchange community.View Community