This post is mostly a self-reminder to think more on a few logging ideas I encountered. I figured others might benefit from them too.

Canonical Log Lines

The first is canonical log lines. Stipe had the idea to output a summative log line per unit of work. This allows quick adhoc analysis of system behaviors without corrolating multiple log lines. It also enables significant analysis with fairly simple text search tooling.

I haven’t really thought about write-time transformation of logs before, but it makes sense. It’s often hard to get the full picture of events in a single request out of the noise of many concurrent logs. Log analytic tools can help you, but it adds complexity quickly.

It has me wondering what other kinds of transforms would be useful. I should also investigate how different structures or conventions could empower logging.

He specifically suggest use of counters over rates. This enables incremental computation, where new statistics are always old aggregate + new record. Each metric update is fixed-time, no matter how many records we accumulate. Any re-calculation is also perfectly distributable.