.NET Core Log Correlation - Arbitrary Log Properties

Pass a header from the frontend site, and have it automatically appear in all log entries on the backend service using a custom property name so I can easily search the logs.

I want to pass a header from the frontend site, and have it automatically appear in all log entries on the backend service using a custom property name so I can easily search the logs.

This is part 2 of my blog post series on log correlation between a frontend website and a backend service.

  1. Send an ID between API calls and have it included in the log files.
  2. The ID should be in the same property in all logs.
  3. Easily access the correlation ID.
  4. When possible, it should automatically be added to the correct HttpClient requests.

Unfortunately, the extension areas in the .NET core pipeline does not allow customizing the top-level log scope when the connection is first established which is where the CorrelationId property is added (among a few others). We can add additional properties through middleware, which is the next best option. However, there are still some log entries before and after the middleware is executed. So, my second goal of using the same property ID in all log entries currently isn't possible. Which also means that I can't have additional properties that I want in all of the messages. The reason I can't do this is because I can't change the properties in the frontend site once they've been added to the scope, the frontend correlation id is null. The necessary entry points for doing this are slated for Core 3.0 and when that is released, there will be a post on how to do it.

It is technically possible if we built our own context factory and whatnot, but, that just isn't worth it to get a few entries that we can look up by other means if we really need to.  With this in mind, we'll set up the custom logging in middleware and get it in as many log entries as we can.

Here's a middleware class that will add arbitrary, in this case, X-SessionId headers to the properties of the log messages, you would use this in your back-end service:

using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;

namespace Vecc.LogCorrelation.Example.Target.Services.Middleware
{
    public class LogHeaderMiddleware
    {
        private readonly RequestDelegate _next;

        public LogHeaderMiddleware(RequestDelegate next)
        {
            this._next = next;
        }

        public async Task InvokeAsync(HttpContext context)
        {
            var header = context.Request.Headers["X-SessionId"];
            if (header.Count > 0)
            {
                var logger = context.RequestServices.GetRequiredService<ILogger<LogHeaderMiddleware>>();
                using (logger.BeginScope("{@SessionId}", header[0]))
                {
                    await this._next(context);
                }
            }
            else
            {
                await this._next(context);
            }
        }
    }
}

To use it, in your Startup class, in the Configure method at the beginning of it, put in an app.UseMiddleware<LogHeaderMiddleware>(); Putting this at the top of the Configure method will make it so that this middleware will encompass as much of the request pipeline and log entries as it can.

An example of a basic MVC application's Configure method:

        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            app.UseMiddleware<LogHeaderMiddleware>();

            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
            else
            {
                app.UseHsts();
            }

            app.UseHttpsRedirection();
            app.UseMvc();
        }

And we now have solved my second requirement, adding a log property, based on a header, to as many log messages as possible.

You can use this middleware to add as many properties with any name to your log entries with any data that is available at the time the middleware is executed. Be careful though, the more properties you add, the larger your log entries will be.

On to the 3rd goal, making a header easily accessible so we can easily use it, namely in making additional requests.