.NET Core Web API Metrics with Prometheus and Grafana
You can use a few lines of code and a NuGet package with Prometheus and Grafana to quickly show metrics of your .NET Core 2.2/3 Web APIs. This is how I did it.
Introduction
Let’s say you have a set of Web APIs for your application. And now you want to see the number of requests, 404’s, method calls, and overall usage over time periods for your application. That is fair. And you want to show it as simple graphs or visuals since “a picture is worth 1,000 words” at a quick glance. That makes sense as well.
I recently came across the same thing with my OSS project. I thought “I hear a lot about Prometheus for gathering metrics and Grafana for showing those metrics”. So I set out to find out HOW to do that with my .NET Core 2.2 projects. What I found out is that it is fairly easy to do! That was very cool to see. And the steps below are what I did to get started on that journey.
My goal is that once you read through this and understand it, and you view the example GH repo I link below, you can add metrics to your web APIs in 10 minutes. And show them in another 5. Time well spent!
Note: I am using the ASP.NET middleware part of the Prometheus component for .NET Core 2.2. You can use that same thing, or you can use the push gateway type of setup. See the links at the bottom for finding the GH prometheus-net project to read more. And find what works best for you and your needs.
Prerequisites for this Project
I am assuming you have .NET 2.2 or higher installed and that you have Docker and Docker Compose installed as well. If not follow the links, set them up, and come back. And you need some kind of editor. I use VS Code personally, but even a text editor, nano or vi will work. And a terminal or console window to run commands.
See the GH repo at https://github.com/Cingulara/dotnet-core-prometheus-grafana for more information. I won’t bore you with every single detail on how to do this since I can supply you with real code to test out and play with. Just clone it and follow along below.
Setting Up Your Project
First, setup a directory (or folder) to put all your stuff into. For this example project we are running through, I have a few .NET Core Web APIs to use so I made a new directory and inside that ran a few of the commands below in a terminal (or console) window. I gave the “-n” different names to make the API folders. You can see the final version of all of this in the GH repo I link at the bottom.
dotnet new webapi -n peopleapi
dotnet new webapi -n locationsapiI am going to setup just 2 simple Web APIs to show how this works for metrics using Prometheus and Grafana. You can add more or fewer APIs and add more endpoints than I do to them. The concepts we go over here and the process to show the metrics stays the same. In our ending directory we will have the code for the APIs, the Prometheus setup file, and the docker-compose.yml to run it all together.
Now in each one of the API folders run the below command in that same terminal window to add the prometheus-net package into them. Remember we are using the ASP.NET middleware for an included /metrics API endpoint for this. There are 2 prometheus-net packages you can use. See the GH repo for prometheus-net to see which package you should use.
dotnet add package prometheus-net.AspNetCore
dotnet restoreNow that each one of the APIs has the right component, you only need to modify the Startup.cs file to make the magic happen. That is pretty sweet! I did not believe that at first, but it is true for what I needed. At the top of the Startup.cs file in the API folder, add your “using Prometheus;” statement.
Then add the below code inside the Configure( … ) method at the bottom of your Startup.cs file. This one is for the People API project in the GH repo linked in this article. Repeat the same thing for the Location API project, or in your project if you are using this as a guide for your own APIs.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
// Custom Metrics to count requests for each endpoint and the method
var counter = Metrics.CreateCounter("peopleapi_path_counter", "Counts requests to the People API endpoints", new CounterConfiguration
{
LabelNames = new[] { "method", "endpoint" }
});app.Use((context, next) =>
{
counter.WithLabels(context.Request.Method, context.Request.Path).Inc();return next();
});// Use the Prometheus middleware
app.UseMetricServer();
app.UseHttpMetrics();...
}
The “peopleapi_path_counter” in the code above is what this custom metric we added be named in Prometheus so name it accordingly if you have multiple APIs for your application. That way you know what metrics is for what API. The rest of them fall under the general metrics that Prometheus pulls. We will see when we look at the Prometheus section below.
Once all that is done, modify your API Controller(s) in your API for whatever calls you wish to have and then make sure it compiles. Realize the ASP.NET middleware we are using calls a /metrics API endpoint. So DO NOT USE THAT!
In my case for this example, I also put the API code into a Dockerfile so I could make it a container and run all this within one docker-compose.yml file. See the GH repo at https://github.com/Cingulara/dotnet-core-prometheus-grafana for more information on that. I won’t bore you with all the details since I can supply you with real code to test out and play with.
When you are done with your API Controllers and you have your Dockerfile setup to make a container for each of your APIs, you are ready to put all this into action!
Containerize It All
(hint: see the https://github.com/Cingulara/dotnet-core-prometheus-grafana below for how I did all this)
Once you have your projects ready go to you can use a similar Dockerfile to what I have in my project and run the commands below to build your containers. Do that for each of the APIs you are going to use, in this specific case we have 2. And I always list images after to make sure what I saw in the output logs actually put images into my local Docker registry. See my commands I used below.
cd peopleapi
docker build -f Dockerfile -t peopleapi .
cd ../locationsapi
docker build -f Dockerfile -t locationsapi .
cd ..
docker images | moreThe docker-compose.yml file I am using starts the 2 APIs, Prometheus, and Grafana all within a single Docker network so they can talk to each other. Prometheus is exposed on port 9090 so we can see it. And Grafana is exposed on port 3000 for the same reason. If you do not want to see Prometheus directly you do not have to expose its port. Just specify “9090” (versus “9090:9090") under the “ports:” area in the docker-compose file.
Run the below command inside your root directory for all of this to come up and watch all the logs come up in your console or terminal window. You can now access your APIs, your Prometheus server, and your Grafana server! Congratulations! If this does not work, make sure you have created a container image for each API and you have access to the Internet to pull the Grafana, Prometheus as well as the .NET 2.2 SDK to make the end product API containers.
docker-compose upNow let’s call some APIs to get data going in there for the metrics to show up! For this included GH repo example, make calls to http://localhost:8082, http://localhost:8082/423423432432423424, http://localhost:8084, and http://localhost:8084/o23uo23uo324u to resemble calling the People API and Locations API. This is all canned data so it is not important what the data returned looks like. Just that it actually returns. The actual calls are important to produce the metrics we desire.
We also should call some bad URLs to get some 404 type data in there. So make some calls to http://localhost:8082/thisisbad/23o8249237423 and http://localhost:8084/thiswontwork/123242?helpme=true to see how improper URLs also make metrics data.
Using Straight Prometheus
If all is up and running, you should be able to go to http://localhost:9090/ and see the main Prometheus interface. To see what we did, choose the locationsapi_path_counter from the dropdown and click Execute (see image below). You should see some metrics there showing on the screen. Choose the Graph tab and press the “-” key until you get to a lower timeframe. Then click Execute again to see the update.
You can remove that information from the large textbox at the top of Prometheus and then choose the people custom metric as well to see the same information for that API. To learn more about Prometheus searching you can read over the PromQL documentation.
You can see from the prometheus.yml file linked here from our GH repo that Prometheus queries the 2 APIs via their internal docker-compose name and port (8080) every 15 seconds. We can use the docker internal names as this is all running in one docker-compose file in the same docker network.
This is all good, you can now see metrics about your APIs. And you can repeat for your own APIs as well now. But how do you get the easy dashboard with all the graphs in one spot? That is where Grafana comes in!
Setting Up Grafana Dashboards
Now go to the http://localhost:3000/ URL and you get the login prompt for Grafana. Use the ‘admin’ login and then the password that is in the docker-compose file in my repo and log in. Once you login you get a page ready to setup your first dashboard.
Click the Add data source highlighted icon and we will point to our Prometheus setup. On the Add data source page, choose Prometheus (usually the top listing) and fill in the Name field with whatever you want that makes sense. The URL needs to be http://prometheus:9090/ and leave the rest alone. Click Save & Test at the bottom to make sure it all works. If it does, you see a Green bar along the bottom telling you so. Then select the Back button.
Now click the + icon in the far left top area of the Grafana menu and choose Import on the popup menu. This will allow you to import a pre-made dashboard for you to show .NET Core metrics. On the next screen enter 10427 to use a dashboard from Ahmad Chehre in the available Grafana Dashboards online.
Click Load and then enter a dashboard name you want in the Name field. Click in the Prometheus field and choose the Prometheus data source you just made. Click the Import button and you should get a screen similar to below. Remember we called our APIs earlier to feed your system metric data. There is your data!
You can remove the top right graph if you wish (I do). And then in the top right of the overall Grafana page change the timeframe to view (15 minutes, 30 minutes, etc. ) as well as a refresh rate.
Repeat the steps we just went through on importing a second dashboard but use 10915 as the ID from this dashboard created by sandersaares. Choose the Prometheus data source again and specify a good name. Save that dashboard and let it load.
This shows requests received by API controllers as well as errors and request duration. As with any of the Grafana dashboards you can always edit, modify or add sections to data points. But these two will get you started. Get used to Prometheus Query Language and have at it!
Conclusion
Congratulations! You just setup 2 web APIs and metrics to track their usage using great OSS tools in Prometheus and Grafana. Again, check the links below for great resources and a link to my GH repo that has all this code ready for you to learn, play, and get better at tracking metrics.
To stop all the containers you can run the below command from your terminal window. That should shut down all containers, remove the network, and clean up any used ports we were exposing.
docker-compose downIf you want something more advanced, you can click on the arrows next to the individual graph titles in Grafana and click the Edit menu. See the PromQL queries they are calling and play around in Prometheus directly to see what data is returned. Add or edit information you want to show in Grafana using your new skills. I am in the middle of doing just that.
Useful Links
Here are some useful links on my copy of all this code in GH for you to use, the packages used for the metrics server, and 2 Grafana dashboards to show metrics.
I found the great example of adding metrics on the method per API endpoint reading over https://www.olivercoding.com/2018-07-22-prometheus-dotnetcore/ as well. I have yet to add this to a dashboard, but am in the middle of learning how to do that.
Happy coding!
