Show Navigation

Grails Blog

Configuring Rolling Logging with Logback

By Zachary Klein

November 1, 2016

Tags: #logging #logback #config

A common requirement in web applications is to support a rolling log system, so that log files are rolled over on a schedule and archived after a certain point. Grails® 3 uses Logback (considered the successor to log4j) as its logging library, and it's quite simple to configure a rolling appender using Logback's Groovy config format.

The Grails framework includes a default Logback configuration at grails-app/conf/logback.groovy. By default (as of Grails 3.2.1), this file (which follows the standard Logback groovy config format) configures a single appender, an instance of ConsoleAppender called STDOUT, and conditionally (when in development mode) an instance of FileAppender called FULL_STACKTRACE. These may then be used by logger instances, which can target specific package names and log levels, and write to one or more appenders. You have access to the Grails Environment, so you can configure different combinations of appenders for development and production.

The default logback.groovy file in Grails 3.2.1 is shown below.

//grails-app/confg/logback.groovy
import grails.util.BuildSettings
import grails.util.Environment

// See https://logback.qos.ch/manual/groovy.html for details on configuration
appender('STDOUT', ConsoleAppender) {
    encoder(PatternLayoutEncoder) {
        pattern = "%level %logger - %msg%n"
    }
}

def targetDir = BuildSettings.TARGET_DIR
if (Environment.isDevelopmentMode() && targetDir != null) {
    appender("FULL_STACKTRACE", FileAppender) {
        file = "${targetDir}/stacktrace.log"
        append = true
        encoder(PatternLayoutEncoder) {
            pattern = "%level %logger - %msg%n"
        }
    }
    logger("StackTrace", ERROR, ['FULL_STACKTRACE'], false)
    root(ERROR, ['STDOUT', 'FULL_STACKTRACE'])
}
else {
    root(ERROR, ['STDOUT'])
}

Let's add an instance of RollingFileAppender for our production environment. Let's say we want to split out our log files by day, and keep 30 days worth of log files around (deleting any older ones). In addition, we don't have unlimited hard drive space, so we'll also set a file size cap so that the total disk space used by our logs never exceeds 2GB.

Here's our new appender:

//grails-app/confg/logback.groovy
import ch.qos.logback.core.rolling.RollingFileAppender
import ch.qos.logback.core.rolling.TimeBasedRollingPolicy
import ch.qos.logback.core.util.FileSize


def HOME_DIR = "."

appender("ROLLING", RollingFileAppender) {
  encoder(PatternLayoutEncoder) {
      pattern = "%level %logger - %msg%n"
  }
  rollingPolicy(TimeBasedRollingPolicy) {
      fileNamePattern = "${HOME_DIR}/logs/myApp-%d{yyyy-MM-dd_HH-mm}.log"
      maxHistory = 30
      totalSizeCap = FileSize.valueOf("2GB")
  }
}

An instance of RollingFileAppender needs two "policies", a rollingPolicy (to define how to perform the rollover), and a triggerPolicy (which specifies when the rollover should occur). In this case, our rollingPolicy is TimeBasedRollingPolicy, which happens to implement the TriggeringPolicy interface and therefore satisfies both policy requirements. TimeBasedRollingPolicy is one of the most common rolling policies, and it will meet the majority of rolling log requirements.

TimeBasedRollingPolicy gets both it's rolling behavior (creating a new log file with the current date/time in the file name) and it's triggering behavior (rollover will occur based on the specified timestamp pattern) from the fileNamePattern property.

What's In a Name?

Maybe you'd rather not have the filenames of your log files contain the trigger interval. No worries, RollingFileAppender also supports a file property which can override this behavior, so your log files can be rolled over monthly (for example) without actually containing the date string in their filenames. See the documentation for RollingFileAppender for more details.

The trigger policy is the most interesting part here - it takes an approach that bases the rollover occurrence on how specific you define the timestamp in the fileNamePattern. So if you specify down to the month, rollover will occur each month. Specify a pattern down to the day, and it will occur daily). It's easier to understand when you see it in action, so here's some example patterns taken from Logback's documentation:

  fileNamePattern = "/myApp-log.%d{yyyy-MM}.log"	      //Rollover at the beginning of each month, file format: myApp-log.2016-11.log
  fileNamePattern = "/myApp-log.%d{yyyy-ww}.log"	      //Rollover at the first day of each week. Note that the first day of the week depends on the locale.
  fileNamePattern = "/myApp-log.%d{yyyy-MM-dd_HH}.log"	//Rollover at the top of each hour.

Note that in the above examples we are configuring the timestamp in the filename of the log files. We can also use the timestamp to create a file directory structure, like this example:

fileNamePattern = "/logs/%d{yyyy/MM}/myApp.log"	//Rollover at the beginning of each month.
//Each log file will be stored in a year/month directory, e.g: /logs/2016/11/myApp.log, /logs/2016/12/myApp.log, /logs/2017/01/myApp.log

Finally, adding a zip or gz file extension to our fileNamePattern will apply the selected compression to the rolled-over log files:

fileNamePattern = "/myApp-log.%d{yyyy/MM}.gz"	      //Rollover at the beginning of each month, compress the rolled-over file with GZIP

Going back to our previous example, we're setting a couple more properties on our TimeBasedRollingPolicy - maxHistory and totalSizeCap. These are pretty simple to understand; maxHistory sets the upper limit on how many log files to preserve (when the max is reached the oldest file is deleted), and totalSizeCap sets a cap on how much disk space our log files are allowed to use (again, when the cap is reached the oldest files are deleted). There are other useful options you can set here, see the TimeBasedRollingPolicy documentation for a full list and explanation.

Warning!

While the Logback docs suggest that totalSizeCap can be specified as a plain String (i.e, "2GB"), I've found that it needs to be specified as a FileSize to avoid casting exceptions - so make sure to use FileSize.valueOf("2GB") to evaluate your total size. This also applies to the maxFileSize property used in the SizeBasedRollingPolicy and TimeAndSizeBasedRollingPolicy.

There are several more rolling & triggering policies that are available, including SizeBasedTriggerPolicy and SizeAndTimeBasedTriggeringPolicy, and FixedWindowRollingPolicy

Finally, let's specify that we want our new RollingFileAppender to be used in production mode only, while keeping the default ConsoleAppender for development mode.

//grails-app/confg/logback.groovy
import grails.util.BuildSettings
import grails.util.Environment
import ch.qos.logback.core.rolling.RollingFileAppender
import ch.qos.logback.core.rolling.TimeBasedRollingPolicy
import ch.qos.logback.core.util.FileSize

def HOME_DIR = "."

// See https://logback.qos.ch/manual/groovy.html for details on configuration
appender('STDOUT', ConsoleAppender) {
    encoder(PatternLayoutEncoder) {
        pattern = "%level %logger - %msg%n"
    }
}

appender("ROLLING", RollingFileAppender) {
  encoder(PatternLayoutEncoder) {
      pattern = "%level %logger - %msg%n"
  }
  rollingPolicy(TimeBasedRollingPolicy) {
      fileNamePattern = "${HOME_DIR}/logs/myApp-%d{yyyy-MM-dd_HH-mm}.log"
      maxHistory = 30
      totalSizeCap = FileSize.valueOf("2GB")
  }
}

def targetDir = BuildSettings.TARGET_DIR
if (Environment.isDevelopmentMode() && targetDir != null) {
    appender("FULL_STACKTRACE", FileAppender) {
        file = "${targetDir}/stacktrace.log"
        append = true
        encoder(PatternLayoutEncoder) {
            pattern = "%level %logger - %msg%n"
        }
    }

    logger("StackTrace", ERROR, ['FULL_STACKTRACE'], false)
    root(ERROR, ['STDOUT', 'FULL_STACKTRACE'])
}
else {
    root(ERROR, ['ROLLING'])
}

And with that, we have our rolling log system. Enjoy!

Resources

You might also like ...