Monitoring and logging .NET apps with OpenTelemetry

Efficient logging mechanisms play a crucial role in maintaining and optimizing .NET applications. Logging is critical for diagnosing issues, monitoring performance, and gaining insights into your application’s behavior.

OpenTelemetry is an open-source tool for monitoring and logging in modern software applications. It provides a set of APIs, SDKs, and tools that promote observability, helping you analyze application performance and behavior.

This article demonstrates how to seamlessly integrate OpenTelemetry for logging and monitoring within a .NET application.

Using OpenTelemetry for monitoring and logging in .NET applications

Rather than providing a dedicated logging system, OpenTelemetry integrates with existing logging frameworks, enhancing log correlation with traces and spans. Its real-time logging and event monitoring offer a holistic view of your application’s behavior, enabling you to troubleshoot issues, optimize performance, better understand how your software operates, and maintain the reliability and performance of your .NET applications.

Prerequisites

To follow along with this tutorial, ensure you have the following software installed:

• Visual Studio Code
• .NET7 SDK
• Postman

Setting up a simple .NET application

Start by creating a simple .NET Minimal API application.

Open a terminal or command prompt, move to the target folder for your application, and run the following command:

dotnet new web --name MonitoringAndLogging

Integrating OpenTelemetry into your .NET application

To integrate OpenTelemetry into your .NET application and add all the relevant NuGet packages to the project, execute the following commands:

dotnet add package OpenTelemetry.Exporter.Console --prerelease 
dotnet add package OpenTelemetry.Exporter.Jaeger --prerelease 
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol --prerelease 
dotnet add package OpenTelemetry.Extensions.Hosting --prerelease  
dotnet add package OpenTelemetry.Instrumentation.AspNetCore --prerelease  
dotnet add package OpenTelemetry.Instrumentation.Http --prerelease 

Next, open the Program.cs file and replace its contents with the following code:

using System.Diagnostics; 
using Microsoft.AspNetCore.Mvc;  
using OpenTelemetry.Logs; 
using OpenTelemetry.Resources;  
using OpenTelemetry.Trace; 

var builder = WebApplication.CreateBuilder(args);  

All code changes will be made in Program.cs, and all commands will be executed in the terminal.

Now, you need to add an activity source for the application. Use the following code:

var uploadActivitySource = new ActivitySource(name: "UploadActivitySource");   

Next, configure the OpenTelemetry resources:

var openTelemetry = builder.Services.AddOpenTelemetry(); 

openTelemetry.ConfigureResource(resource => resource  
    .AddService(serviceName: builder.Environment.ApplicationName));   

Now, you need to configure tracing. Add the following code, which adds tracing for ASP.NET Core and the custom activity source, then configures the tracing to output to the console:

openTelemetry.WithTracing(tracing => 
{ 
    tracing.AddAspNetCoreInstrumentation(); 
    tracing.AddHttpClientInstrumentation();  
    tracing.AddSource(uploadActivitySource.Name);  
    tracing.AddConsoleExporter(); 
});    

At this point, you need to integrate an endpoint to take advantage of the instrumentation features:

var app = builder.Build();  

app.MapPost("/upload", (ILogger logger, [FromBody] FileDetails data) =>  
{  
    using var activity = uploadActivitySource.StartActivity("UploadActivity");  

    logger.LogInformation($"Uploading file {data.Filename} ({data.Size}MB)");  
    activity?.SetTag("upload", data.Filename);  
    logger.LogInformation($"File {data.Filename} uploaded."); 

    return Results.Ok("Upload successful!");  
}); 

app.Run(); 

internal record FileDetails(string Filename, int Size);    

When using the .NET logger to log messages to the console, OpenTelemetry collects the messages and exports them to the console exporter. You can view this in the POST endpoint, which logs file upload activity.

Switch to the project folder and run the application with the following commands:

cd MonitoringAndLogging 
dotnet run    

Open Postman and send a POST request to the /upload endpoint using the URL http://localhost:5236/upload and the following JSON body:

{  
    "Filename": "presentation.ppt",  
    "Size": 100  
}     

The console will show a trace output message. The top part shows activity from the custom UploadActivitySource, while the bottom shows activity from the ASP.NET Core activity source:

Activity.TraceId:            9768f744af16e8f8770cb7e98eb1f34b  
Activity.SpanId:             976a12b26f257bbc  
Activity.TraceFlags:         Recorded  
Activity.ParentSpanId:       19cb3a36bfc3a593  
Activity.ActivitySourceName: UploadActivitySource 
Activity.DisplayName:        UploadActivity 
Activity.Kind:               Internal 
Activity.StartTime:          2023-10-08T19:20:43.3979137Z 
Activity.Duration:           00:00:00.0009908  
Activity.Tags:  
    upload: presentation.ppt  
Resource associated with Activity:  
    service.name: MonitoringAndLogging  
    service.instance.id: d8510577-f34a-4d12-96da-d279f9665ebf 
    telemetry.sdk.name: opentelemetry 
    telemetry.sdk.language: dotnet 
    telemetry.sdk.version: 1.6.0  

Activity.TraceId:            9768f744af16e8f8770cb7e98eb1f34b  
Activity.SpanId:             19cb3a36bfc3a593  
Activity.TraceFlags:         Recorded 
Activity.ActivitySourceName: Microsoft.AspNetCore 
Activity.DisplayName:        /upload  
Activity.Kind:               Server  
Activity.StartTime:          2023-10-08T19:20:43.2855453Z  
Activity.Duration:           00:00:00.1358450  
Activity.Tags:  
  net.host.name: localhost  
   net.host.port: 5236 
    http.method: POST  
    http.scheme: http  
    http.target: /upload 
    http.url: http://localhost:5236/upload 
    http.flavor: 1.1  
    http.user_agent: PostmanRuntime/7.33.0  
    http.status_code: 200  
Resource associated with Activity: 
    service.name: MonitoringAndLogging  
    service.instance.id: d8510577-f34a-4d12-96da-d279f9665ebf  
    telemetry.sdk.name: opentelemetry 
    telemetry.sdk.language: dotnet  
    telemetry.sdk.version: 1.6.0    

Next, you need to configure OpenTelemetry to export traces from the collector to a tracing back end. This example uses Jaeger, a popular open-source distributed tracing platform.

Jaeger collects the activities and visualizes them for analysis and troubleshooting. To get started, download the Jaeger binaries, extract the contents, and open a new terminal inside the folder where the jaeger-all-in-one.exe file resides.

Next, run the following command:

./jaeger-all-in-one --collector.otlp.enabled     

In the console, find a line that looks like this:

{"level":"info","ts":1696781657.0638707,"caller":"otlpreceiver@v0.86.0/otlp.go:83","msg":"Starting GRPC 
server","endpoint":"0.0.0.0:4317"}      

"0.0.0.0:4317" is the port where Jaeger is listening for OpenTelemetry Protocol (OTLP) traffic via gRPC. After tracing.AddConsoleExporter();, add the following tracing configuration:

tracing.AddOtlpExporter(otlpOptions => 
     {  
         otlpOptions.Endpoint = new Uri("http://localhost:4317/");  
     });       

Restart the application and navigate to http://localhost:16686/ in your browser to view the Jaeger UI. Be sure to select the MonitoringAndLogging service in the Service input field from the left sidebar. Make a call to the API endpoint to see a trace on this interface:

Clicking on the trace gives you a more detailed view of the activity displayed as a gantt chart:

In the code above, you can add tags as key-value data pairs to an activity using the SetTag method. You can also add events to the activity, which are time-stamped messages containing additional diagnostic data. Add an event using the following command:

activity?.AddEvent(new ActivityEvent($"File {data.Filename} uploaded."));        

Here’s the result in the console:

Activity.Tags:  
    upload: presentation.ppt  
Activity.Events:  
    File presentation.ppt uploaded. [2023/10/09 05:10:53 +00:00] 
              

Each activity can submit a status to OpenTelemetry, indicating the level of success of the work. You use the tag names otel.status_code and otel.status_description to store the StatusCode and Description, respectively. The StatusCode must be one of the string values "UNSET," "OK," or "ERROR." The Description is a custom string value that provides more information about the error. Here’s an example:

activity?.SetTag("otel.status_code", "ERROR"); 
activity?.SetTag("otel.status_description", $"File {data.Filename} not found!"); 
              

Now, you need to add the following activity tags to the logs:

Activity.Tags:  
    upload: presentation.ppt  
    StatusCode: ERROR  
    Activity.StatusDescription: File presentation.ppt not found! 
              

Sometimes, it helps to filter out certain traces so you only log pertinent data. You can achieve this by filtering out specific requests. To do so, replace tracing.AddAspNetCoreInstrumentation(); in the tracing configuration with the following:

tracing.AddAspNetCoreInstrumentation(options => { options.Filter = (httpContent) => httpContent.Request.Path !=  "/upload"; });  
                

This code filters out requests to the /upload endpoint. If you restart the application and send a request to the /upload endpoint, nothing will be logged.

Testing your setup: Generating and viewing logs

Now, update the code to simulate a variety of logs. Replace the /upload endpoint with the following code:

app.MapPost("/upload", (ILogger<Program> logger, [FromBody] FileDetails data) => 
{  
    using var activity = uploadActivitySource.StartActivity("UploadActivity"); 

    activity?.SetTag("upload", data.Filename);  
    activity?.AddEvent(new ActivityEvent($"File upload ({data.Filename})."));  

    if (Path.GetExtension(data.Filename) != ".ppt") 

    { 
        activity?.SetTag("otel.status_code", "ERROR"); 
        activity?.SetTag("otel.status_description", $"File format {Path.GetExtension(data.Filename)} not supported!");  

        return Results.Problem($"File format {Path.GetExtension(data.Filename)} not supported!"); 
    }  

    var rand = new Random().Next(0, 3);  
    if (rand == 2)  
    { 
        try  
        {  
            throw new Exception("Simulated exception."); 
        }  
        catch (Exception ex)  
        { 
            activity?.SetTag("otel.status_code", "ERROR");  
            activity?.SetTag("otel.status_description", ex.Message);  

            throw;  
       }  
    } 

    activity?.SetTag("otel.status_code", "OK");  
    activity?.SetTag("otel.status_description", $"File {data.Filename} uploaded successfully!"); 

    return Results.Ok("Upload successful!");  
});   
              

You just added two if statements to the original code. The first if statement logs an error if the uploaded file is in an unsupported format. The second if statement simulates an exception and logs an error inside the catch block.

To simulate a successful upload and exception, set the Filename in the request body to "presentation.ppt". Set the Filename to "presentation.pdf" to simulate the “file format not supported” error. Here’s the result in Jaeger:

Click one of the traces to see the trace view, which shows the timeline of spans that make up the trace. You can search for traces by various attributes, such as service name, operation name, or time range.

Click on individual spans to view their details. Here, you’ll find information about logs and attributes associated with that span:

Conclusion

Logging and monitoring are crucial to maintaining and improving your .NET applications. OpenTelemetry enhances observability with tracing capabilities while also allowing you to integrate with your preferred logging framework. By combining logging and tracing under one umbrella, OpenTelemetry offers more profound insights into application behavior. This depth of information means you can make more well-informed decisions, ensuring that your .NET applications are stable, secure, and performant.

Ready to enhance your application’s observability and performance? Dive deeper into monitoring your .NET applications with Site24x7’s Microsoft .NET Application Performance Monitoring tool. Pinpoint slow code, streamline database calls, and gain comprehensive insights into your application’s inner workings—all with the convenience and precision that Site24x7 brings to the table. Don’t miss out on elevating your monitoring experience—start a free 30-day trial today.