af-metrics

Python package for monitoring the Coffea-Casa Analysis Facility using Prometheus Monitoring tool and generating dashboards from the collected metrics using Grafana. The package deploys Prometheus locally on localhost:8003 and establishes a connection between Prometheus and the JSON Data collected from the specific Jupyter Notebook to be monitored in the Coffea-Casa Analysis Facility using the Prometheus Client Package. This JSON data serves as the custom metrics for the Prometheus server. These Prometheus metrics serve as the data for generating convenient dashboards using the Grafana tool.

The current infrastructure can be represented as:

In the existing infrastructure of Coffea Casa Analysis Facility, the individual users run their own Jupyterhub and Dask Servers at the corresponding endpoints. In our project, we try to scrape these individual endpoints and also that of the Coffea Casa Analysis Facility as a whole using useful metrics and thus monitor the facility using the Prometheus tool. Based on the scraped metrics, we can visualize the results conveniently by generating dashboards using the Grafana tool. So Prometheus connects to the Jupyterhub and Dask Servers of the existing users and the Facility to monitor the specific metrics and Grafana, in turn, connects to Prometheus and generates useful dashboards based on the results of the monitoring process. The process of dashboard generation can also be automated using grafanalib library which automatically generates a suitable json file specifying the dashboard data source as well as the appearance of the dashboard.

Install the package with the command:

  pip install af-metrics

This converts the JSON Data into Prometheus metrics and it can now be easily accessed by the user once he starts the server.

You can now test the prometheus metrics by running a similar code snippet:

from af_metrics import CustomCollector
import time
from prometheus_client.core import REGISTRY
from prometheus_client import start_http_server

if __name__ == '__main__':
    start_http_server(8000)
    REGISTRY.register(CustomCollector())
    while True:
        time.sleep(1)

This will start the server at localhost:8000.

Open the browser and start localhost:8000 to test the JSON Data converted to Prometheus Metrics.