Ultimate log4j Tutorial for Java Logging – Best Practices, Resources and Tips

By: mwatson
  |  February 28, 2024
Ultimate log4j Tutorial for Java Logging – Best Practices, Resources and Tips

Logging plays a critical role in every application. In this tutorial, we’ll explore how to use Apache Log4j 2.x and highlight best practices that can help you in getting started and improve logging capabilities. Additionally, we’ll discuss various key features and improvements of Log4j 2.x over its predecessor, Log4j 1.x.

Introduction

First things first, let’s get a basic understanding of Log4j 2.x.

What is Log4j 2.x?

Apache Log4j 2.x is a very popular logging framework for Java/Jakarta EE (formerly Java EE) based applications, known for its simplicity and efficiency.

Why should you use a logging framework?

A logging framework is very critical for any application as it provides vital insights for troubleshooting, debugging, monitoring, performance analysis, usage analytics, compliance, security, and audit purposes.

It is very challenging to detect, diagnose, and fix issues in your application without proper logging. Logging empowers developers, administrators, and stakeholders within an organization to gain visibility into the application’s behavior, make informed decisions, and ensure the application’s reliability and security.

What’s new in Log4j 2.x?

Log4j 2.x significantly improves performance and latency through its next-generation asynchronous logging and low garbage generation during steady-state logging in multi-threaded scenarios. In addition to these improvements, Log4j 2.x offers a range of key features:

  1. Flexible configuration system based on plugins
  2. Customizable logging levels
  3. Enhanced loggers, filters, layouts, and appenders
  4. Support for lambda expressions in Java 8 and newer versions

These features collectively make Log4j 2.x a powerful and versatile logging framework.

A log configuration file is a text file typically consisting of key-value pairs defining different configuration properties and their corresponding values

What is a log configuration file?

A log configuration file is a text file typically consisting of key-value pairs defining different configuration properties and their corresponding values. These properties determine the destination of the logging messages, what format to be utilized, and the desired logging level.

Modifying the configuration properties allows you to easily redirect logs to various locations including databases, files, log management systems, console, Syslog, or other network destinations, all without requiring changes to the application code.

Getting Started with Log4j 2.x

What are the prerequisites for using Log4j 2.x?

Log4j 2.x requires at least Java, but it is recommended to use Java 8 or above for optimal performance and compatibility with some of the optional dependencies for additional features. Java 7 is no longer supported by the Log4j team.

You may also need to install a build management tool like Apache Maven or Gradle to simplify the dependency management process.

How to get Log4j 2.x installed for your project?

To begin using Apache Log4j 2.x in your Java or Jakarta EE application, you need to follow the instructions as provided below:

Step 1: Dependency Management

In your project’s build file (pom.xml for Maven) and add the appropriate version of Log4j 2.x dependency.

<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>2.x.x</version>
</dependency>

By adding this configuration to your pom.xml file, Maven will automatically download and include the Log4j Core library as a dependency for your project during the build process. To leverage Log4j’s logging capabilities within your Java application, you need to import the LogManager class of the  org.apache.logging.log4j package.

Best Practices

1. Use static modifier for LogManager Object

When you declare any variable in your code, it comes with overhead. However, you can overcome this overhead by declaring the static modifier for the Logger reference as below. If you invoke constructors on the LogManager object then it will consume a lot of CPU.  Using static modifier in the declaration does not require you to hardcode the class type saving CPU cycles.

/* Get the logger for the actual class name to be printed on */
private static final Logger logger = Logger.getLogger(App.class);

In the above code snippet, we declare a Logger object using the getLogger() method of the LogManager class and passing the name of the logger (e.g. App.class ) as an argument. Now you can use the logger object to log messages at different levels in your application using methods like info(), debug(), error(), etc.

2. How to enable log4J’s Internal Debug Logging

In case you encounter issues with a specific appender, or face difficulties while working with it. Enabling the internal debugging can help you troubleshoot and resolve these problems by providing additional logging providing insights into its internal operations. To enable internal debugging, set the log4j2.debug system property in the configuration file or add -Dlog4j2.debug to the JVM system variables as below:

java -Dlog4j2.debug -cp ... some.class.name

Note: After you have resolved the issues or completed the debugging process, it is recommended to disable internal debug logging, as it may impact performance and increase log file sizes.

3. Do Not Send Your Logs to a Database Table with the JDBCAppender

You can directly persist logs details into your database by simply creating a table, such as the one shown below, in an Oracle database. You can then configure the JDBCAppender in the log4j.properties file to store logs.

However, when dealing with a significant volume of logs, searching for specific results using SQL queries can become challenging. It is advisable to consider alternative approaches, such as writing them to a file or sending logs to a log management service with full-text indexing and more comprehensive log management capabilities.

CREATE TABLE LOGS_REP
(USER_ID VARCHAR(20)        NOT NULL,
LOG_DATE    DATE           NOT NULL,
LOGGER      VARCHAR(50)    NOT NULL,
LOG_LEVEL   VARCHAR(10)    NOT NULL,
LOG_MESSAGE VARCHAR(1000)  NOT NULL

The above SQL statement creates a table named “LOGS_REP” in a database to store details of log messages such as user ID, log date, logger name, log level, and log messages.

Configuration in log4j.properties file for JDBCAppender

# Define the root logger with appender file
log4j.rootLogger = DEBUG, DATABASE
# Define the DATABASE appender
log4j.appender.DATABASE=org.apache.log4j.jdbc.JDBCAppender
# Set JDBC URL
log4j.appender.DATABASE.URL= jdbc:oracle:thin:@::
# Set Database Driver
log4j.appender.DATABASE.driver=com.oracle.jdbc.Driver
# Set database user name and password
log4j.appender.DATABASE.user=db_user_name
log4j.appender.DATABASE.password=db_password
# Set the SQL statement to be executed.
log4j.appender.DATABASE.sql=INSERT INTO LOGS_REP VALUES('%x','%d','%C','%p','%m')
# Define the layout for file appender
log4j.appender.DATABASE.layout=org.apache.log4j.PatternLayout

The above configuration defines a root logger with a database appender, specifying the JDBC URL, database driver, credentials, SQL statement for insertion, and a pattern layout for logging data into an Oracle database

4. Do Not Send Emails on Every Exception

You can use SMTPAppender to notify the exceptions in the logs to the required stakeholders through emails. But you do not need to send every exception thrown by the application through emails. Sending excessive emails can potentially disrupt the smooth operation of your application, leading to performance issues. Instead, select critical exceptions by adjusting the log level to FATAL, ERROR, or other appropriate levels. By doing so, you can ensure only necessary information is sent via email, providing meaningful insights without compromising application performance.

log4j.rootLogger=ERROR, mail
log4j.appender.mail=org.apache.log4j.net.SMTPAppender
[email protected]
[email protected]
log4j.appender.mail.SMTPHost=mail.yourdomain.com
log4j.appender.mail.Threshold=ERROR
log4j.appender.mail.BufferSize=1
log4j.appender.mail.Subject=Application Error
log4j.appender.Mail.layout=org.apache.log4j.PatternLayout 
log4j.appender.Mail.layout.ConversionPattern=%d %-5p %c %x - %m%n

The above configuration configures Log4j to send email notifications for application errors, specifying the recipient, sender, SMTP host, logging threshold, email subject, and layout format.

5. How to Send Alerts for Exceptions

You can use the error tracking product in order to send alerts about exceptions thrown by your application by using NTEventLogAppender. Such a product can also deduce the errors so you can figure out when an error is truly new, track its history, and track error rates.

The configuration file of log4j’s NTEventLogAppender for ActiveMQ events

log4j.rootLogger=ERROR, NTEVENT
log4j.appender.NTEVENT=org.apache.log4j.nt.NTEventLogAppender 
log4j.appender.NTEVENT.source=ActiveMQ 
log4j.appender.NTEVENT.layout=org.apache.log4j.PatternLayout 
log4j.appender.NTEVENT.layout.ConversionPattern=%d | %-5p | %m | %c | %t%n 
log4j.appender.NTEVENT.threshold=ERROR

The provided configuration sets up log4j’s NTEventLogAppender to log ActiveMQ events to the Windows Event Log, specifying the source name, layout class, and layout pattern for the log message format.

6. Send Your Logs to a Log Management System to View Them Across Servers

When capturing application logs, writing them to a file on disk with compression and periodic archiving is beneficial, but for effective searching through logs across multiple servers and applications, it is recommended to send the logs to a central repository.

I suggest utilizing log management solutions like Splunk or setting up your own Elasticsearch cluster for fast searching capabilities, similar to a Google search engine, even when dealing with large volumes of log data. These solutions provide filtering options by log level or date, facilitating the correlation of related log events into transactions and enabling efficient log analysis.

7. Use Filters to Suppress Certain Logging Statements

You can use filters that can be configured to suppress specific log messages. The following are the configuration details for log4j.properties to set up filters in order to suppress certain logging statements.

log4j.rootLogger=info, R, ERROR
#### only INFO
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=FacturaElectronica.log
log4j.appender.R.MaxFileSize=500KB
log4j.appender.R.MaxBackupIndex=1
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
log4j.appender.R.filter.a=org.apache.log4j.varia.LevelRangeFilter
log4j.appender.R.filter.a.LevelMin=INFO
log4j.appender.R.filter.a.LevelMax=INFO
#### only ERROR
log4j.appender.ERROR=org.apache.log4j.RollingFileAppender
log4j.appender.ERROR.File=FacturaElectronicaError.txt
log4j.appender.ERROR.MaxFileSize=500KB
log4j.appender.ERROR.MaxBackupIndex=1
log4j.appender.ERROR.layout=org.apache.log4j.PatternLayout
log4j.appender.ERROR.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
log4j.appender.ERROR.filter.b=org.apache.log4j.varia.LevelMatchFilter
log4j.appender.ERROR.filter.b.LevelToMatch=ERROR
log4j.appender.ERROR.filter.b.AcceptOnMatch=true
log4j.appender.ERROR.Threshold=ERROR

The above configuration defines two appenders, R and ERROR, with specific settings for logging INFO level messages to a rolling file using a pattern layout and filters, and logging ERROR level messages to another rolling file with matching filters and layouts.

8. Make our Own Custom log4j Appenders

If you want to do something that the standard Appenders do not support, you have two options: search online for existing custom Appenders or create your own by extending the AppenderSkeleton class in log4j. For example, you can make your own custom log4j appender by extending the AppenderSkeleton class. It provides the code for common functionality, such as support for threshold filtering and support for general filters, you override methods to add your custom functionality on top of it as shown below.

package com.stackify.log4j_demo;
import java.util.ArrayList;
import java.util.List;
import org.apache.log4j.AppenderSkeleton;
import org.apache.log4j.spi.LoggingEvent;
public class CustomAppender extends AppenderSkeleton{
   List eventsList = new ArrayList();
   @Override
   protected void append(LoggingEvent event) {
       eventsList.add(event);
   }
   public void close() {}
   public boolean requiresLayout() {
       return false;
   }
}

The above code defines a CustomAppender class that extends the AppenderSkeleton class from log4j, overriding the append() method to add log events to an eventsList and implement the close() and requiresLayout() methods.

9. Customize Your Layout in the Logs with log4j Pattern Layouts

You can modify your configuration file in order to change the pattern layout format for the fields which you are throwing as output.

# Define the root logger with appender APP
log4j.rootLogger=DEBUG, stdout, APP
# add a ConsoleAppender to the logger stdout to write to the console
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
# Pattern to output the caller's file name and line number.
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] - %d{yyyy-MM-dd HH:mm:ss.SSS}; - (%F:%L) - %m%n
# Define the file for APP appender
log4j.appender.APP=org.apache.log4j.RollingFileAppender
log4j.appender.APP.File=example.log
#Define Max File Size for APP Appender
log4j.appender.APP.MaxFileSize=100KB
# Keep one backup file for APP Appender
log4j.appender.APP.MaxBackupIndex=1
# Define the layout for APP appender
log4j.appender.APP.layout=org.apache.log4j.PatternLayout
log4j.appender.APP.layout.ConversionPattern=%5p %t - %d{yyyy-MM-dd HH:mm:ss.SSS}; - %c [%thread] - %m%n

The following are the description of the pattern appearing in the log4j.properties file.

  • %5p – It writes the level in the log. The “5” in the “%5p” is to set the width of the field to 5 characters.
  • %d{yyyy-MM-dd HH:mm:ss.SSS}; – It writes the date in the given date-time format.
  • %t – It writes the method name in the log.
  • %c – It writes the absolute class name (e.g.com.stackify.log4j_demo.App) in the log.
  • %m%n – It writes the message in the log.
  • %L – It writes the line number in the log.
  • %F – It writes the class name in the log.

10. Use the Diagnostic Contexts to Log Additional Fields

You can log custom fields like username, etc. to provide additional context information about the user, customer, or transaction related to the log statements. In log4j, you can use NDC class which is known as the per-thread stack of context labels. The labels pushed on this stack can be displayed  PatternLayout by specifying the appropriate format parameters as shown below.

Configuration of log4j.properties file as follows.

# Define the root logger with appender APP
log4j.rootLogger=DEBUG, stdout, APP
# add a ConsoleAppender to the logger stdout to write to the console
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
# Pattern to output the caller's file name and line number.
log4j.appender.stdout.layout.ConversionPattern=%x %5p [%t] - %d{yyyy-MM-dd HH:mm:ss.SSS}; - (%F:%L) - %m%n
# Define the file for APP appender
log4j.appender.APP=org.apache.log4j.RollingFileAppender
log4j.appender.APP.File=example.log
#Define Max File Size for APP Appender
log4j.appender.APP.MaxFileSize=100KB
# Keep one backup file for APP Appender
log4j.appender.APP.MaxBackupIndex=1
# Define the layout for APP appender
log4j.appender.APP.layout=org.apache.log4j.PatternLayout
log4j.appender.APP.layout.ConversionPattern=%x %p %t - %d{yyyy-MM-dd HH:mm:ss.SSS}; - %c - %m%n

Let’s log user name into the logs using NDC class.

package com.stackify.log4j_demo;
import java.io.IOException;
import java.sql.SQLException;
import org.apache.log4j.Logger;
import org.apache.log4j.NDC;
public class App {
	
	/* Get the logger for the actual class name to be printed on */
	static Logger log = Logger.getLogger(App.class.getName());
	
	public static void main(String[] args) throws IOException, SQLException {                      
		NDC.push("Aparajita ");
		log.fatal("This is a fatal message for log4j");
		log.error("This is an error message for log4j");
		log.debug("This is an debug message for log4j");
		log.warn("This is a warning message for log4j");
		log.info("This is an info message for log4j");
	}
}

Output is shown below.

Aparajita  FATAL [main] - 2017-04-09 12:16:36.600; - (App.java:22) - This is a fatal message for log4j
Aparajita  ERROR [main] - 2017-04-09 12:16:36.628; - (App.java:23) - This is an error message for log4j
Aparajita  DEBUG [main] - 2017-04-09 12:16:36.629; - (App.java:24) - This is an debug message for log4j
Aparajita   WARN [main] - 2017-04-09 12:16:36.630; - (App.java:25) - This is a warning message for log4j
Aparajita   INFO [main] - 2017-04-09 12:16:36.630; - (App.java:26) - This is an info message for log4j

11. How to Correlate Log Messages by Web Request Transaction

Additionally, you can assign objects in contexts to use what it calls “active property values.” When the log message is written to a file or console, the toString() method will be called which can dynamically do something.

12. How to Do Structured Logging, or Log an Object or Properties with a Message

By default, you can log an object to it and it will serialize it with its default renderers. If you want to really get the value of structured logging, you will want to send your logs to a log management tool that can index all the fields and enable powerful searching and analytics capabilities. You can also use MulticolorLayout class along with log4j’s ConsoleAppender to get multiple colors in logs (i.e. to view logs into distinct colors) for which you need to append the following maven repository and dependency.

		
			jcabi.jcabi-dynamo
			https://mvnrepository.com/artifact/com.jcabi/jcabi-dynamo
		
              …
		
			com.jcabi
			jcabi-dynamo
			0.17.1
		 

Next, you have to set the related configuration details in log4j.properties files as shown below.

# Define the root logger with appender APP
log4j.rootLogger=DEBUG, stdout, APP
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=com.jcabi.log.MulticolorLayout
log4j.appender.stdout.layout.ConversionPattern=[%color{%-5p}] %c: %m%n
# Define the file for APP appender
log4j.appender.APP=org.apache.log4j.RollingFileAppender
log4j.appender.APP.File=example.log
Utilize appropriate logging levels in your code to turn up or down the verbosity of your logging at any time

13. Make Good Use of Multiple log4j Log Levels and Filter by Them

Utilize appropriate logging levels in your code to turn up or down the verbosity of your logging at any time.

Avoid logging everything at the Debug level. Be sure to think about what information will be helpful later when you are troubleshooting application problems. Maintain a balance between how much logging is noise versus surfacing critical problems.

You can specify in your log4j properties which log4j logging levels you want to log. You can use this to send all logs to a file on disk, but perhaps only fatal problems to a database or other appender.

Log4j levels along with their explanations

  • All – Log everything
  • Debug – Log detail information useful for debugging and tracing code execution
  • Info – Log informational messages that highlight the progress or events in th application.
  • Warn – Log warning messages that indicate potential issues or non-fatal errors.
  • Error – Log serious error messages like serious issues or failures affecting the application’s functionality.
  • Fatal – Log critical messages like severe errors or failures that may cause the application to terminate or crash.
  • Off – Don’t log anything

Conclusion

In conclusion, logging is a crucial part of any application development process, and Log4j does an excellent job of making logging easier and more efficient. We’ve covered some of the basic concepts of Log4j like logging levels, configuration files, and appenders, as well as discussed Apache Log4j 2, the latest version of Log4j, and why it’s important. We’ve also talked about some resources and best practices that can help you implement Log4j effectively. Logging can be an incredibly powerful tool when used appropriately, and Log4j is one of the best logging frameworks available in Java. So, what are you waiting for, start implementing Log4j today!

Improve Your Code with Retrace APM

Stackify's APM tools are used by thousands of .NET, Java, PHP, Node.js, Python, & Ruby developers all over the world.
Explore Retrace's product features to learn more.

Learn More

Want to contribute to the Stackify blog?

If you would like to be a guest contributor to the Stackify blog please reach out to [email protected]