What Is Python Logging? A Complete Introduction

Logging is a crucial practice that helps developers monitor, debug, and maintain their applications. Python, a versatile and widely used programming language, provides a comprehensive logging module that makes it easy to integrate logging into your applications. This post introduces Python logging, covering the basics, various logging levels, practical implementations, and more. By the end of this post, you should have a concrete understanding of Python logging and how to use it effectively in your projects.

What Is Python Logging?

Logging in Python refers to the process of recording messages generated by a program’s execution. These messages can provide insights into the program’s flow, errors, performance, and other crucial metrics. The Python logging module provides a versatile structure for generating log messages within Python programs, enabling developers to log messages at different severity levels, direct log output to various destinations, and customize log formats. Monitor everything about your Python applications with one tool here.

Python Logging Module

The logging module in Python provides various functions and classes to configure logging effectively, including:

• Loggers: Responsible for creating log entries.
• Handlers: Send log records (created by loggers) to the appropriate destination.
• Formatters: Specify the format of log records.
• Filters: Add filtering functionality to handlers and loggers.

How Python Logging Works

Python logging works by creating loggers that generate log messages, handlers that send these messages to designated destinations, formatters that define the structure of the log messages, and filters that control which log records are output. When a logger records a message, it passes through the filters and is sent to the handlers, where it is formatted and finally output to the configured destination such as a file, console, or external logging service. This modular architecture allows flexible and powerful logging configurations to suit various needs.

Python Logging Levels

Python logging has five standard levels of severity to categorize log messages:

• DEBUG: Provides detailed information, primarily useful during troubleshooting.
• INFO: Confirms that everything is functioning as expected.
• WARNING: Signals something unexpected or a potential problem in the near future (e.g., “low disk space”).
• ERROR: Indicates a serious issue that has prevented the software from performing a function.
• CRITICAL: Represents a very severe error, suggesting the program may not be able to continue running.

Here’s an example demonstrating different logging levels:

import logging

# Configure the logging

# Log messages with different severity levels
logging.debug("This is a debug message.")
logging.info("This is an info message.")
logging.warning("This is a warning message.")
logging.error("This is an error message.")
logging.critical("This is a critical message.")

Code Explanation

• We import the logging module, providing access to its functions and classes for logging.
• We configure the logging module by calling the logging.basicConfig() to set up the basic configuration for logging, specifying the minimum logging level to DEBUG and enforcing it with the force=True parameter.
• We then log messages at various severity levels using functions from the logging module.

Code Output

This code configures logging to display all messages from the DEBUG level and higher.

When you run this code, you will see messages for all severity levels as shown in the figure above since the logging level is set to DEBUG.

How to Perform Python Logging

1. Python Logging to File

To log messages to a file instead of the console, you can configure the logging module with a filename parameter. This is shown below:

import logging

# Configure logging to a file
logging.basicConfig(filename='app.log', level=logging.INFO)

# Log a message
logging.info("This message will be logged to a file.")

This code configures logging to write messages to a file named app.log.

2. Python Logging to stdout

If you want to log messages to stdout, the standard output stream, which is typically the console or terminal where your Python program is running, you can use the stream handler. The code example below shows this:

import logging
import sys

# Configure logging to stdout
logging.basicConfig(stream=sys.stdout, level=logging.INFO)

# Log a message
logging.info("This message will be logged to stdout.")

This code configures logging to write messages to stdout.

3. Python Logging to Console

Logging to the console can be achieved by using the StreamHandler. The code example below shows this:

import logging

# Create a custom logger
logger = logging.getLogger(__name__)

# Create handlers
console_handler = logging.StreamHandler()

# Set level for handlers
console_handler.setLevel(logging.INFO)

# Create formatters and add them to handlers
console_formatter = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
console_handler.setFormatter(console_formatter)

# Add handlers to the logger
logger.addHandler(console_handler)

# Log a message
logger.info("This message will be logged to the console.")

This code creates a custom logger, adds a handler for logging to the console, and logs an informational message.

Python Logging Set Level

Setting the logging level allows you to control the severity of messages that your application logs. Here are examples on how to set different log levels.

Example 1: Set Level to DEBUG

This code sets the log level to DEBUG, so all messages will be logged.

import logging

# Create a logger
logger = logging.getLogger(__name__)

# Set the logging level to DEBUG
logger.setLevel(logging.DEBUG)

# Log messages with different severity levels
logger.debug("This is a debug message.")
logger.info("This is an info message.")
logger.warning("This is a warning message.")
logger.error("This is an error message.")
logger.critical("This is a critical message.")

Code Output

Example 2: Set Level to WARNING

This code sets the log level to WARNING, so only WARNING, ERROR, and CRITICAL messages will be logged.

import logging

# Create a logger
logger = logging.getLogger(__name__)

# Set the logging level to WARNING
logger.setLevel(logging.WARNING)

# Log messages with different severity levels
logger.debug("This is a debug message.")
logger.info("This is an info message.")
logger.warning("This is a warning message.")
logger.error("This is an error message.")
logger.critical("This is a critical message.")

Code Output

Example 3: Set Level to ERROR

This code sets the log level to ERROR, so only ERROR and CRITICAL messages will be logged.

import logging
# Create a logger
logger = logging.getLogger(__name__)

# Set the logging level to ERROR
logger.setLevel(logging.ERROR)

# Log messages with different severity levels
logger.debug("This is a debug message.")
logger.info("This is an info message.")
logger.warning("This is a warning message.")
logger.error("This is an error message.")
logger.critical("This is a critical message.")

Code Output

When and Where to Use Each Log Level

Understanding when and where to use each logging level is crucial for effective logging. Here’s a detailed guide on the practical usage of each logging level in different environments.

DEBUG

• Purpose: Provides detailed information; useful during development and debugging.
• Usage:
  • DEBUG logs trace code execution, inspect variables, and understand the flow of your program.
  • Ideal for local development and testing environments.
• Avoid: Avoid using DEBUG logs in production environments as they can generate a large volume of log messages and potentially expose sensitive information.

INFO

• Purpose: Confirms that things are working as expected; general operational information.
• Usage:
  • INFO logs record high-level events in your application, such as startup and shutdown events, successful transactions, or user activities.
  • Suitable for both development and production environments.
• Avoid: Avoid overusing INFO logs to prevent log files from becoming too verbose.

WARNING

• Purpose: Indicates potential problems or important situations that are not errors but may require attention.
• Usage:
  • WARNING logs highlight unexpected events, deprecated functions, or situations that may cause future issues.
  • Appropriate for both development and production environments.
• Avoid: Do not use WARNING logs for routine information or error messages.

ERROR

• Purpose: Records error events that prevent parts of the program from functioning correctly.
• Usage:
  • ERROR logs capture exceptions, failed operations, or significant problems that need immediate attention.
  • Important for both development and production environments.
• Avoid: Avoid using ERROR logs for critical system failures; use CRITICAL instead.

CRITICAL

• Purpose: Indicates severe error events that might cause the program to terminate or require immediate action.
• Usage:
  • CRITICAL logs record serious issues like system outages, data corruption, or security breaches.
  • Crucial for both development and production environments, especially for alerting and monitoring.
• Avoid: Avoid using CRITICAL logs for less severe issues that do not require immediate attention

We can further summarize each of these log levels into this practical usage:

• Development Environment: Use DEBUG, INFO, WARNING, ERROR, and CRITICAL logs to gain comprehensive insights into your application. DEBUG logs are particularly useful here.
• Testing Environment: Focus on INFO, WARNING, ERROR, and CRITICAL logs to verify the correctness and stability of the application. DEBUG logs can be used selectively for specific test scenarios.
• Production Environment: Primarily use INFO, WARNING, ERROR, and CRITICAL logs. DEBUG logs should generally be avoided due to performance and security concerns.

Why Is Python Logging Important?

Python logging is important for several reasons:

• Debugging: Logs provide insights into the flow of the program and help in diagnosing issues.
• Monitoring: Logs can be monitored to ensure that the application is running smoothly and to detect anomalies.
• Auditing: Logs can be used to keep track of user activities and application events, which is crucial for security and compliance.
• Performance Tuning: By analyzing logs, developers can identify performance bottlenecks and optimize their applications.

Python Logging Best Practices

To make the most out of Python logging, follow these best practices:

1. Use Log Levels Appropriately: Choose the appropriate log level for each message.
2. Configure Logging in a Centralized Manner: Configure logging in one place, such as in a configuration file or a central logging setup function. Here is a guide on how to configure your Python logging.
3. Avoid Logging Sensitive Information: Exercise caution to ensure that sensitive information, like passwords or personal data, is not logged.
4. Use Rotating File Handlers: To prevent log files from growing indefinitely, use rotating file handlers that create new log files after a certain size or time period.
5. Implement Structured Logging: Use structured logging to create logs that are easier to parse and analyze by log management tools.
6. Leverage External Logging Services: For large-scale applications, consider using external logging services like Stackify Retrace for better log management and analysis. Stackify Retrace log management centralizes app and server error logs, offering drill-down and search capabilities. These provide the structure, access, and context needed for faster troubleshooting and performance optimization.

By understanding and implementing best practices, you will enhance the reliability, security, and maintainability of your applications.

Conclusion

Python logging is a powerful tool that aids in developing reliable and maintainable software. By using the features of the logging module, you can create informative log messages that help monitor and debug your applications effectively. Remember to configure logging appropriately, use different logging levels wisely, and follow best practices to ensure your logs serve their intended purpose efficiently. Using products like Stackify Retrace enhances the usefulness of your logs by providing context to performance issues and advance features for fast remediation. Start your free Stackify Retrace trial today.

Theophilus Onyejiaku has over five years of experience as data scientist and a machine learning engineer. His expertise includes data science, machine learning, computer vision, deep learning, object detection, model application development and deployment. He has written over 650 articles in python programming, data analytics, the aforementioned fields, and so much more.

Improve Your Code with Retrace APM

Stackify's APM tools are used by thousands of .NET, Java, PHP, Node.js, Python, & Ruby developers all over the world.
Explore Retrace's product features to learn more.

• App Performance Management
• Code Profiling
• Error Tracking
• Centralized Logging
• App & Server Metrics

Learn More

Author

Stackify

More articles by Stackify