Monitoring Metrics of .Net Core Web API project using Prometheus and Grafana

Monitoring resources is one of the most useful and major features of an application, especially when its in need of scalability, pick load and suffer from time to time performance issues or just your customer enjoy watching some fancy analysis dashboard that makes him feel happy about pay you money 😄

Recently I am requested to develop a pretty dashboard within .Net Core Web Application(3.1) that customers can monitor resource usage as well as system performance in detail.
As an average middle-aged developer eager to learn I had started to investigate every topic, forums, articles and every possible solution includes GitHub RTR projects. Many of them relating .net core 2.x and various, not stable packages, none of them works this is why I have decided to write this article(Don’t judge me if you already found a working solution, maybe I missed it. Btw why are you here if you have found it? let us discuss it another topic 😄).

There are many 3rd party tools out there, What I am about to do here is; building a .Net Core Web Application from scratch using 2 of these most widely used open-source software, Prometheus and Grafana.

What is Prometheus?

Prometheus is an open-source system monitoring and alerting toolkit originally built at SoundCloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, and the project has a very active developer and user community. It is now a standalone open source project and maintained independently of any company. To emphasize this, and to clarify the project’s governance structure, Prometheus joined the Cloud Native Computing Foundation in 2016 as the second hosted project, after Kubernetes. Prometheus is a big topic by itself, to find out more information in detail click here

What is Grafana?

Grafana is an open source visualization and analytics software. It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored. In plain English, it provides you with tools to turn your time-series database (TSDB) data into beautiful graphs and visualizations. If you need more information, you can take a look in official web site

Both of these software has much more to talk about and contains huge documentaris in its official web sites, but I want to cut off talking short and dive into coding!

Lets create new .Net Core Web API project;

you can choose whatever you wish, I have selected “API” as my initial project template to speed up goal, just mind .netcore version of your project. By the time I write this article, the latest version of .Net Core Framework is 3.1, I don’t think there will be a problem for new incoming versions. All you need to do update your pages regarding Prometheus and AppMetric. (But game rules might change again on 4.x 😊, especially for middleware which I’m about to share).

Now install Prometheus packages via Package Manager Console running commands below;

Install-Package prometheus-net
Install-Package prometheus-net.AspNetCore

After install packages add the line below into Startup.cs to to start exposing a default set of metrics. After editing it should look like below;

Cautions!: In the Configure method you must add these lines above of app.UseEndPoints(…); or it won't work

it means; you will not get any metric data, I have experienced this searching many hours in pain😭. At least I will save yours…

Mission Complete!, now let's test it; Run your application and send a get request to https://localhost:44335/metrics the endpoint, If you see a page like this;

It means well done!, these are default metrics data which has been set as default configuration by Prometheus.

Customize your metrics data (Optional)

This stuff makes Prometheus running, but in real life scenario, you probably want to gather some extra or filter metrics specifically so take a step forward to customize our metrics;

There are some different types of metric data supported by Prometheus and available to report according to their official documentation.

Some of them;
Counter; A counter is a cumulative metric that represents a single monotonically increasing counter whose value can only increase or be reset to zero on restart, so its better usage place when you want to track the total number of requests that your services handle during Req/Resp middleware lifecycle

Histogram; A histogram samples observations (usual things like request durations or response sizes) and counts them in configurable buckets. It also provides a sum of all observed values. We can use histograms metrics to gather durations of requests handled by services

We need to implement our own custom middleware to gather these metrics;

Create a “Middleware” folder and add a new class file; “MetricMiddleware.cs

Copy and paste content below;

Now we will create our own metric collector class; create a new class with name of “MetricReporter.cs” and copy-paste the codes below;

Notice Counter and Histogram instances here, are included from the Prometheus library which we installed before. So add “using Prometheus;” and “using Microsoft.Extensions.Logging;” to log these data

Now open your Startup.cs file and add your custom middleware; your file should look like below after edited;

Finally open Program.cs and copy-paste below;

This is configuration settings of how our application metrics data formatted, we have set output formatter with MetricsPrometheusTextOutputFormatter to make Prometheus server able to get data peacefully

Our business with the application is done! Now open your browser go and download the latest version of the Prometheus server that matches your operating system.

Unzip and open “prometheus.yml” file;
I will not explain all properties in detail since it already contains a brief description as comments if you want further information you can access it here

Quick Tips; If you didn’t select SSL configuration while creating your.Net Core Web application remove “scheme: https” since its set to https as default behind the scene;
Add your applications running port number as targets as well as metrics_path of job which indicates Prometheus service communicate with our app to gather metrics,

Note; Now I think you see why we have used ‘/metrics-text’ endpoint in our custom metric-middleware 😊

Now close and save prometheus.yml file and double click “prometheus.exe”. You should see something like the pic on the left. Beware your web application is running and configuration settings are correct in your web application and Prometheus server otherwise it will be stuck at “Server is ready to receive web requests” message.

It means Prometheus server up and running in foreground now and last setup we need to make is; yes Grafana!

Now open go and download latest version of Grafana Server that matches your operating system.

And now...
The most epic challenge of your development career has come!

Unzip the file and double click “grafana-server.exe”, that's all. 🤣

To access Grafana open browser and make a request to http://localhost:3000, then you will see the login page of Grafana UI

Username: admin
Password: admin

These are initial credentials of Grafana after you log in the system it will ask you to change, feel free to change as you wish

After log in to grafana, click ‘Add your first data source’

As you have seen here Grafana has ability to communicate with plenty of data source technologies, we will choose Prometheus as our data source.

Incoming form, type : “http://localhost:9090” which is endpoint of running prometheus server in our machine. Leave default rest and scroll down click “Save & Test” then you should see some notification toaster message that says process completed successfully

Now we need to create a dashboard template to demonstrate our metrics data with delightful analytical UI elements.
There is more than one way to do this in Grafana, you can create your own custom dashboard template or you can even buy ready to use a template which composed for very specific purposes.

Again to speed up the process and make it easy we gonna use a ready-to-use free dashboard template provided by the Grafana team to help us young tech kiddies like me.
All we need the unique ID of the template; I already know its “2204”

Now follow these steps to create your first Grafana template;

▹ Now go to “Create -> Import” via navbar to create your first dashboard.

▹Type 2204 and click “Load

▹ Then to the following settings type your name of the dashboard

▹ Select “General” as Folder and select your Prometheus data source which
you created before then click “Import

And here is the awesome magic of Grafana;

Its real time analysis dashboard, want to see some action?
Go to your running .net application and make some requests to provide more metrics data to Grafana

I am using Postman to make request to web application, it doesn’t matter you can do this in browser as well.

After making some dummy requests you will see Grafana reacts it in a few seconds

As you see above, rpm(“request per minute”) increased respectively.

Grafana also shows the environment info about our application; Environment, application, data source.
Summarize; this value indicates which period your dashboard updates

That’s all I am willing to share about integrating .net core web application with Prometheus and Grafana. Actually as you see we only integrate our app with Prometheus and Grafana totally independent from this communication it only gets data from Prometheus to draw dashboard.

I hope it helps and goes to aid of people who seek furter assistance.

I share the source code of this demo application in my Github repo.

Happy coding!

I’m an IT guy who loves discover and learn new things, nowadays playing around of React & .Net Core