Ultimate log4j Tutorial for Java Logging – Best Practices, Resources and Tips

Matt Watson Live Queue

Logging is a critical feature of any application. In this tutorial we will cover some log4j best practices that can help you get started and improve how you do logging with log4j.

What is log4j and why should you use it, or any Java logging framework?

A logging framework is important for any Java/J2EE based application. By changing the configuration, you can easily write your logs to different places. You can write your Java application logs to a database, files on the disk, a log management system, a console, Syslog or possibly other places in the network without changing your application code.

Java Logging & log4j Best Practices

1. Use static modifier for LogManager Object

When you declare any variable in your code, it comes with overhead. You can overcome this overhead by declaring the static Logger reference as shown below. If you invoke constructors on the LogManager object then it will consume a lot of CPU and if you declare it as static then you will not have to hard code the class type which will save CPU cycles.

/* Get the logger for the actual class name to be printed on */
	private static final Logger log = Logger.getLogger(App.class);

2. How to enable log4J’s Internal Debug Logging

There could be scenarios when you can face problems with a specific appender, or get into issues while working with it. You can resolve these issues after enabling the internal debugging by setting the log4j.debug system property in the configuration file. You can also add -Dlog4j.debug to the JVM system variables to enable log4j’s own internal debug logging as shown below.

java -Dlog4j.debug -cp ... some.class.name

3. Do Not Send Your Logs to a Database Table with the JDBCAppender

You can directly persist logs details into your database by simply creating a table (shown below) in a database (here oracle) and setting the JDBCAppender configuration into log4j.properties file (shown below). When you persist logs into database and try to search those logs through queries in SQL then it becomes very difficult to get the expected search result when you log any real volume. It is advisable to send the logs to a log management service or write them into a file which can provide full-text indexing and more functionality with logs.

CREATE TABLE LOGS_REP
(USER_ID VARCHAR(20)        NOT NULL,
LOG_DATE    DATE           NOT NULL,
LOGGER      VARCHAR(50)    NOT NULL,
LOG_LEVEL   VARCHAR(10)    NOT NULL,
LOG_MESSAGE VARCHAR(1000)  NOT NULL

Configuration in log4j.properties file for JDBCAppender

# Define the root logger with appender file
log4j.rootLogger = DEBUG, DATABASE
# Define the DATABASE appender
log4j.appender.DATABASE=org.apache.log4j.jdbc.JDBCAppender
# Set JDBC URL
log4j.appender.DATABASE.URL= jdbc:oracle:thin:@::
# Set Database Driver
log4j.appender.DATABASE.driver=com.oracle.jdbc.Driver
# Set database user name and password
log4j.appender.DATABASE.user=db_user_name
log4j.appender.DATABASE.password=db_password
# Set the SQL statement to be executed.
log4j.appender.DATABASE.sql=INSERT INTO LOGS_REP VALUES('%x','%d','%C','%p','%m')
# Define the layout for file appender
log4j.appender.DATABASE.layout=org.apache.log4j.PatternLayout

4. Do Not Send Emails on Every Exception

You can use SMTPAppender to notify the exceptions in the logs to the required stakeholders through emails. But you do not need to send every exception thrown by the application through emails as it may stall the smooth operation of your application as the system may get busy in sending emails. There could be scenarios when you will see emails bursting for lots of recurring exceptions which may result into more errors at the application level which may impact its performance very badly. Therefore, select the critical exceptions only by changing the log level to FATAL, or ERROR, etc. in order to send only the required logging information through emails as shown below. This will not only provide the meaningful information but also there will be no impact on our application performance.

log4j.rootLogger=ERROR, mail
log4j.appender.mail=org.apache.log4j.net.SMTPAppender
[email protected]
[email protected]
log4j.appender.mail.SMTPHost=mail.yourdomain.com
log4j.appender.mail.Threshold=ERROR
log4j.appender.mail.BufferSize=1
log4j.appender.mail.Subject=Application Error
log4j.appender.Mail.layout=org.apache.log4j.PatternLayout 
log4j.appender.Mail.layout.ConversionPattern=%d %-5p %c %x - %m%n

5. How to Send Alerts for Exceptions

You can use the error tracking product in order to send alerts about exceptions thrown by your application by using NTEventLogAppender. Such a product can also dedupe the errors so you can figure out when an error is truly new, track its history, and track error rates. The configuration file of log4j’s NTEventLogAppender for ActiveMQ events will look as shown below.

log4j.rootLogger=ERROR, NTEVENT
log4j.appender.NTEVENT=org.apache.log4j.nt.NTEventLogAppender 
log4j.appender.NTEVENT.source=ActiveMQ 
log4j.appender.NTEVENT.layout=org.apache.log4j.PatternLayout 
log4j.appender.NTEVENT.layout.ConversionPattern=%d | %-5p | %m | %c | %t%n 
log4j.appender.NTEVENT.threshold=ERROR

6. Send Your Logs to a Log Management System to View Them Across Servers

You can capture application logs and write them to a file on a disk which could be compressed and archived periodically. But when you want to search through the logs across multiple servers and applications then you need to send all of our logs to a central repository. There are a lot of log management solutions that can help you with this, or you can even setup your own elastic search cluster for it. My suggestion is to use Splunk which provides fast, Google search engine like searching functionalities across bulky terabytes of logs. You can easily filter the logs by log level or date which makes it easy to correlate into transactions of multiple related log events, etc.

7. Use Filters to Suppress Certain Logging Statements

You can use filters which can be configured to suppress specific log messages. The following is the configuration details for log4j.properties to set up filters in order to suppress certain logging statements.

log4j.rootLogger=info, R, ERROR

#### only INFO
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=FacturaElectronica.log
log4j.appender.R.MaxFileSize=500KB
log4j.appender.R.MaxBackupIndex=1
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
log4j.appender.R.filter.a=org.apache.log4j.varia.LevelRangeFilter
log4j.appender.R.filter.a.LevelMin=INFO
log4j.appender.R.filter.a.LevelMax=INFO

#### only ERROR
log4j.appender.ERROR=org.apache.log4j.RollingFileAppender
log4j.appender.ERROR.File=FacturaElectronicaError.txt
log4j.appender.ERROR.MaxFileSize=500KB
log4j.appender.ERROR.MaxBackupIndex=1
log4j.appender.ERROR.layout=org.apache.log4j.PatternLayout
log4j.appender.ERROR.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
log4j.appender.ERROR.filter.b=org.apache.log4j.varia.LevelMatchFilter
log4j.appender.ERROR.filter.b.LevelToMatch=ERROR
log4j.appender.ERROR.filter.b.AcceptOnMatch=true
log4j.appender.ERROR.Threshold=ERROR

8. Make our Own Custom log4j Appenders

If you want to do something that the standard Appenders do not support, you can either search online or write your own customized Appender. For example, you can make your own custom log4j appender by extending the AppenderSkeleton class. It provides the code for common functionality, such as support for threshold filtering and support for general filters and you can add your functionality on top of it by method overriding as shown below.

package com.stackify.log4j_demo;

import java.util.ArrayList;
import java.util.List;

import org.apache.log4j.AppenderSkeleton;
import org.apache.log4j.spi.LoggingEvent;
/**
 * 
 * @author Aparajita
 *
 */
public class CustomAppender extends AppenderSkeleton{

    List eventsList = new ArrayList();

    @Override
    protected void append(LoggingEvent event) {
        eventsList.add(event);
    }

    public void close() {
    	
    }

    public boolean requiresLayout() {
        return false;
    }
}

9. Customize Your Layout in the Logs with log4j Pattern Layouts

You can modify your configuration file in order to change the pattern layouts format for the fields which you are throwing as output.

# Define the root logger with appender APP
log4j.rootLogger=DEBUG, stdout, APP

# add a ConsoleAppender to the logger stdout to write to the console
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout

# Pattern to output the caller's file name and line number.
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] - %d{yyyy-MM-dd HH:mm:ss.SSS}; - (%F:%L) - %m%n

# Define the file for APP appender
log4j.appender.APP=org.apache.log4j.RollingFileAppender
log4j.appender.APP.File=example.log

#Define Max File Size for APP Appender
log4j.appender.APP.MaxFileSize=100KB

# Keep one backup file for APP Appender
log4j.appender.APP.MaxBackupIndex=1

# Define the layout for APP appender
log4j.appender.APP.layout=org.apache.log4j.PatternLayout
log4j.appender.APP.layout.ConversionPattern=%5p %t - %d{yyyy-MM-dd HH:mm:ss.SSS}; - %c [%thread] - %m%n

The following are the description of the pattern appearing the log4j.properties file.

  • %5p – It writes the level in the log. The “5” in the “%5p” is to set the width of the field to 5 characters.
  • %d{yyyy-MM-dd HH:mm:ss.SSS}; – It writes the date in the given date-time format.
  • %t – It writes the method name in the log.
  • %c – It writes the absolute class name (e.g.com.stackify.log4j_demo.App) in the log.
  • %m%n – It writes the message in the log.
  • %L – Itwrites the line number in the log.
  • %F – It writes the class name in the log.

10. Use the Diagnostic Contexts to Log Additional Fields

You can log custom fields like username, etc. which can help you to provide additional context information about the user, customer or transaction related to the log statements. In log4j, you can use NDC class which is known as per-thread stack of context labels. The labels which are pushed on this stack can be displayed in a PatternLayout by specifying the %x or the %ndc format parameter as shown below.

Configuration of log4j.properties file as follows.

# Define the root logger with appender APP
log4j.rootLogger=DEBUG, stdout, APP

# add a ConsoleAppender to the logger stdout to write to the console
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout

# Pattern to output the caller's file name and line number.
log4j.appender.stdout.layout.ConversionPattern=%x %5p [%t] - %d{yyyy-MM-dd HH:mm:ss.SSS}; - (%F:%L) - %m%n

# Define the file for APP appender
log4j.appender.APP=org.apache.log4j.RollingFileAppender
log4j.appender.APP.File=example.log

#Define Max File Size for APP Appender
log4j.appender.APP.MaxFileSize=100KB

# Keep one backup file for APP Appender
log4j.appender.APP.MaxBackupIndex=1

# Define the layout for APP appender
log4j.appender.APP.layout=org.apache.log4j.PatternLayout
log4j.appender.APP.layout.ConversionPattern=%x %p %t - %d{yyyy-MM-dd HH:mm:ss.SSS}; - %c - %m%n

Let’s log user name into the logs using NDC class.

package com.stackify.log4j_demo;

import java.io.IOException;
import java.sql.SQLException;

import org.apache.log4j.Logger;
import org.apache.log4j.NDC;

/**
 * 
 * @author Aparajita
 *
 */
public class App {
	
	/* Get the logger for the actual class name to be printed on */
	static Logger log = Logger.getLogger(App.class.getName());
	
	public static void main(String[] args) throws IOException, SQLException {                      

		NDC.push("Aparajita ");
		log.fatal("This is a fatal message for log4j");
		log.error("This is an error message for log4j");
		log.debug("This is an debug message for log4j");
		log.warn("This is a warning message for log4j");
		log.info("This is an info message for log4j");
		
	}
}

Output is shown below.

Aparajita  FATAL [main] - 2017-04-09 12:16:36.600; - (App.java:22) - This is a fatal message for log4j
Aparajita  ERROR [main] - 2017-04-09 12:16:36.628; - (App.java:23) - This is an error message for log4j
Aparajita  DEBUG [main] - 2017-04-09 12:16:36.629; - (App.java:24) - This is an debug message for log4j
Aparajita   WARN [main] - 2017-04-09 12:16:36.630; - (App.java:25) - This is a warning message for log4j
Aparajita   INFO [main] - 2017-04-09 12:16:36.630; - (App.java:26) - This is an info message for log4j

11. How to Correlate Log Messages by Web Request Transaction

Additionally, you can assign objects in contexts to use what it calls “active property values.” When the log message is written to a file or console, the ‘toString ()’ method will be called which can dynamically do something.

12.How to Do Structured Logging, or Log an Object or Properties with a Message

By default, you can log an object to it and it will serialize it with its default renderers. If you want to really get the value of the structured logging, you will want to send your logs to a log management tool that can index all the fields and enable powerful searching and analytics capabilities. You can also use MulticolorLayout class along with log4j’s ConsoleAppender to get multiple colors in logs (i.e. to view logs into distinct colors) for which you need to append the following maven repository and dependency.

		
			jcabi.jcabi-dynamo
			https://mvnrepository.com/artifact/com.jcabi/jcabi-dynamo
		
              …
		
			com.jcabi
			jcabi-dynamo
			0.17.1
		 

Next, you have to set the related configuration details in log4j.properties files as shown below.

# Define the root logger with appender APP
log4j.rootLogger=DEBUG, stdout, APP

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=com.jcabi.log.MulticolorLayout
log4j.appender.stdout.layout.ConversionPattern=[%color{%-5p}] %c: %m%n

# Define the file for APP appender
log4j.appender.APP=org.apache.log4j.RollingFileAppender
log4j.appender.APP.File=example.log

13. Make Good Use of Multiple log4j Log Levels and Filter by Them

Be sure to use proper logging levels within your code. One of the big advantages of using a logging framework is being able to turn up or down the verbosity of your logging at any time.
Don’t log everything as Debug. Be sure to think about what information will be helpful later when you are troubleshooting application problems. You have to balance how much logging is noise versus surfacing critical problems.

You can specify in your log4j properties which log4j logging levels you want to log. You can use this to send all logs to a file on disk, but perhaps only fatal problems to a database or other appender.

log4j levels:

  • All – Log everything
  • Debug
  • Info
  • Warn
  • Error
  • Fatal
  • Off – Don’t log anything

About Matt Watson

Matt is the Founder & CEO of Stackify. He has been a developer/hacker for over 15 years and loves solving hard problems with code. While working in IT management he realized how much of his time was wasted trying to put out production fires without the right tools. He founded Stackify in 2012 to create an easy to use set of tools for developers.