The Five Levels of Logging

I increasingly feel that there should be two separate logs, one for the user with the Error&#x2F;Warning&#x2F;Info information (what the user wanted to be done and how well it went) and the other for the developer with the Debug&#x2F;Trace information (capture key information about what the program was doing, especially on the interface - what files it was looking for, what database calls it did etc.).<p>I always get complaints from users that there is too much useless stuff in the logs through which they need to go. I also think that the developer log should be structured differently, possibly to be easily read by some automation.

Info: What is the program up to. Occasional messages that indicate it is alive and doing what is expected.<p>Warn: Problems that you know about and wish would not happen, but do anyways. A service call has failed, unexpected input that violates the contract of the method, etc. You may be handling these conditions gracefully and&#x2F;or returning an error to the caller, but still want to know when they happen. Usually you should have a ticket open in your bug tracker for these.<p>Error: Problems that you do not know about. When an error is logged, a developer should look at it. Some kind of action should be taken immediately. Put the fire out if it is an emergency. If not an emergency, create a ticket in your bug tracker, then change the level to warn.<p>I keep warn and info separate so that I can easily grep&#x2F;filter on one. I often want to just see info messages to know what the status of things are.

Talking about logging got me a rejection at Google. The very kind 22 year old interviewing me asked what my thoughts on commenting code was and I said most every time it&#x27;s used as an excuse to show off a piece of overly clever code, so normally I discourage it in favor of logging, as they are code comments you can use in production. I realize now that was my first graybeard moment.

I support these 5 levels of logging, but given the discussion below I also would like to highlight a lesson from an SRE talk I once attended[citation needed] that really there are only three levels of alert:<p>1. DEBUG - Look at this when debugging, otherwise no one will ever see it. 2. TICKET - Open a ticket and have someone on your team look at this when it gets to the top of your queue. 3. PAGER - Drop what you&#x27;re doing and handle this, now.<p>Of course, you want to track INFO and ERROR type messages because a sufficient number of them might cause someone to raise a ticket... but at scale, you probably should&#x27;ve built that monitoring already, and just drop INFO&#x2F;ERROR down to DEBUG.

Debug: If you can read this, you are standing too close to the program.

The article is a bit confusing, because the title says 5, the article gives 6 (Error, Warning, Info, Debug, Trace, and Fatal), but the article casts doubt on whether either of those is right when it says this:<p>&gt; I would like to talk about the five (or six, or four) that I find are the most crucial, the fundamental set to base your own levels (and your thinking) on.<p>Which are the 2 from the 6 given whose status as levels of logging is in doubt?

This article reminded me of a writing that an ex-colleague of mine did at a previous company: <a href="https:&#x2F;&#x2F;nerds.kueski.com&#x2F;better-logging&#x2F;" rel="nofollow">https:&#x2F;&#x2F;nerds.kueski.com&#x2F;better-logging&#x2F;</a><p>Everybody developer knows that they should include logging in their systems, however <i>how</i> and <i>what</i> to log is not something so well documented.<p>When you start working in a system (i.e. at a startup) you put the odd log here and there, without too much attention. However, as your system(s) starts growing, you have to start paying attention at your logs&#x27; structure and semantics, so that they are more useful in ELK, Sumologic and similar log searching engines.

I really believe only two levels are necessary most of the time: error, and info. Error is for anything that a dev should be notified about (AKA if you send an error log message, assume someone will be notified about it immediately), and info is for any other useful debugging information, like &quot;we got a request for &#x2F;foo&#x2F;bar at 10 pm&quot;. I also think structured logging is absolutely essential. Log levels are really just a way to get some of the benefits of structured logging.

I was <i>really</i> hoping this might continue with &#x27;six, if you work with IoT devices&#x27; or something like that, as in the Larry Miller[0] routine.[1] Never mind, I&#x27;ll leave the links here anyway...<p>0. <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Larry_Miller_(comedian)" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Larry_Miller_(comedian)</a><p>1. <a href="https:&#x2F;&#x2F;youtu.be&#x2F;eceSHKqwUPY?t=18" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;eceSHKqwUPY?t=18</a>

&gt; Connecting to server:4242 as &quot;user&quot;:&quot;hunter2&quot;<p>as an example of DEBUG logging is very wrong. Passwords should not end up in your logs, ever.

I have always used them as follows. Deciding between DEBUG and INFO is the most difficult. INFO, WARN, and ERROR are simple decisions.<p>DEBUG: Information which is relevant for tracing activity and data through the program.<p>INFO: Important workflow decisions made within the normal operation of the program. Allows you to tell what path a particular operation flowed down, but not with tracing every activity.<p>WARN: Non-fatal errors which do not prevent the operation from completing. Example: Failing to close a file after having read from it.<p>ERROR: Fatal errors which prevent the operation from completing. Example: Failing to open the file or reading from it.

trace: used for visualizing flow of program, extremely noisy and never on prod<p>debug: low level flow of program, inputs&#x2F;outputs<p>info: what I want to see at runtime in prod, high level flow of program, details about operation so I know what&#x27;s going on if I want to look<p>warn: things that may indicate errors or other unexpected situations to be aware of that may be relevant to problems elsewhere<p>error: something serious or unexpected that probably indicates a defect.. issue that needs to be looked into... unhandled&#x2F;unexpected state

Counterpoint: <a href="https:&#x2F;&#x2F;dave.cheney.net&#x2F;2015&#x2F;11&#x2F;05&#x2F;lets-talk-about-logging" rel="nofollow">https:&#x2F;&#x2F;dave.cheney.net&#x2F;2015&#x2F;11&#x2F;05&#x2F;lets-talk-about-logging</a><p>tl;dr: Only Info and Debug levels carry their weight.

I&#x27;ve always preferred levels of logging rather than these arbitrary names (what&#x27;s the difference between trace and debug, and which one is lower level?).<p>Give me three levels of logging, 1, 2, and 3, and then warning, and error. Maybe allow to mix and match them, if this makes sense (the typical nomenclature makes them all mutually exclusive, which is dumb).<p>There are your five levels.

Thanks for this! Was just about to start implementing a logging subsystem myself so the timing is perfect :)

I&#x27;m going to weigh in on my opinionated library for logging: [woveon-logger](<a href="https:&#x2F;&#x2F;www.npmjs.com&#x2F;package&#x2F;woveon-logger" rel="nofollow">https:&#x2F;&#x2F;www.npmjs.com&#x2F;package&#x2F;woveon-logger</a>) (js). Sorry if that&#x27;s a no-no, but nobody seems to use my much loved library anyway.<p>The multiple levels are nice, especially for logging errors in production and tracking general information of your software. But in development, logging is often about tracking a particular aspect in your code. You need something that logs to a particular <i>aspect</i> that you are working on, and then can turn off that <i>aspect</i> once you move on. In the future, you often find yourself back to debug and need to flip that <i>aspect</i> back on again.<p>Summary: Log levels of info, warn and error all have a place. Logging to aspects is valuable during development, easy to turn off for production and useful when returning to debug.

Netscaler logging deserves its own level of hell. You&#x27;re given 15 undocumented logging levels, with the suggestion not to leave it running longer than necessary, and that level 11 should be enough for anybody.

The issue with logging levels is they only convey basic information through the levels and the rest through the actual message which results in the mess that is log processing.

Logging is very important because it also allows you to debug&#x2F;resolve issues easily on production systems. But caring about the appropriate log levels and the ability to configure them at runtime is also super important. It&#x27;s just insane and ridiculous to log everything on a trace&#x2F;debug level and stream it to the Expensive Innovative Big Data Machine Learning Smart Log Analytics Product. Yes, I&#x27;m looking at you Application Insights.