Introductions are tough, I’m glad that’s out of the way. This post is going to dig straight into events and levels. If concepts like events, timestamps and logging levels are old news for you, this might be one post in this series to skip. Saying a few words about these things up front will make it easier to dig into the nitty-gritty later though, with everyone on the same page.

We instrument a complex applications with logging so that when things go wrong (or when we need to verify that things are going right!) there’s a trail of events that have taken place in the system, for us to inspect.

Thinking about system events as, well, events is fairly natural: something happened in the system at a particular time and needs to be recorded. But, if you’re coming from a purely text-based logging mechanism (perhaps just writing logs to System.Console then it’s a good time now to pause and consider that events are truly a first-class concept in structured logging.

At a minimum, an event is a timestamp and some data, usually comprising a list of named properties.

EventsTimeline

The timestamp is recorded automatically when a log event is captured, so despite being one of the most important elements of an event, it doesn’t show up much in logging code.

To make real-world logging systems convenient to work with there are other special properties, common to all events, that are given additional meaning by the system.

SpecialProperties

Among these special properties you’ll usually encounter a message of some kind, but how exactly that works in Serilog and similar libraries is the topic of the next post, so we’ll park that here and just consider messages to be simple strings.

Beside the timestamp and message, the level (or ‘severity’) of an event is the next thing you’ll find. This is so familiar and ubiquitous that it’s worth discussing what levels are, and are not, in structured logging systems.

Application logging is a balancing act, constrained by the performance cost of producing events, the infrastructure cost of transmitting and storing them, and the attention cost of someone reading or being interrupted by them.

In traditional logging, where the level is one of the few first-class concepts apart from the message, it’s the basis for controlling most of these cost trade-offs. When events are recorded, they’re given a level like debug, warning, or error.

Log.Debug("Checking for new mail");
Log.Information("Setting new pricing");
Log.Warning("Incoming projectile detected");
Log.Error("Could not connect to configuration server");

If you want to follow along and experiment in Visual Studio, now’s a great time to follow the Getting Started instructions on the Serilog wiki to create a basic console application with Serilog configured.

Levels are arranged in a hierarchy from least to most important, and the application is configured to generate events at a certain level or higher. Logging statements below the minimum level are no-ops.

This role - maintaining application performance by reducing log volume under normal operation - is an important function of the level even in apps that use structured logging.

In Serilog, the default level is Information, so the first event above won’t be emitted. The MinimumLevel option can be used to override this:

Log.Logger = new LoggerConfiguration()
    .MinimumLevel.Debug()
    .WriteTo.LiterateConsole()
    .CreateLogger();

Besides using levels like debug to control verbosity, the error level may flag something that should be fixed, and the warning level flag something to investigate, but operationally, levels take a back seat once structured data comes into the picture. Levels might be considered for log retention, and also when generating alerts and notifications, but other application-specific properties often play as much or more of a role in these scenarios.

My rule of thumb, when in doubt, is that if it’s worth recording in production an event is Information. The Serilog wiki has a little more information on the levels provided there.

Custom levels, a common feature of older libraries like the log4* family, exist because without rich structured data to examine, the level field has to take on a lot of responsibility. Instead of having a less-important-error level for exceptions in a non-critical subsystem, and a more-important-error level for another code path, properties of a structured log event describe the context of an error and can be used to make decisions about what to do with it.

With the basic terminology covered, I’m looking forward to discussing message templates and properties in the next post, where modern structured logging starts to diverge from what most of us are familiar with.