Python Simple Logging

There are lots of options for logging in python. In this post I wanted to write down one way to do python simple logging. This method should be fairly flexible across different

Some General Principles

These aren’t always going to be right, but you’ll see these are the general principles I have used.

Use basicConfig
Only call basicConfig once
Use a separate logger for each module
Use the same logging configuration across each logger

Python Simple Logging Example

Main script

The following it the code for the ‘main’ script in your application.

basicConfig is called once to configure logging including:

Specify the logging level
Set a reasonable format
Output to a file and command line using handlers

This main script calls functions in modules which will be responsible for their own logging. We do not need to separately pass in a logger object or do additional configuration per module.

The logging module handles the behind the scenes details and each module, including the ‘main’ one can do a logging.getLogger() call.

Passing in the name variable means, along with the %(name)s format option, that the logging lines will show the name of the file that generated the logs. This can make interpreting log outputs much easier.

import logging
import my_module
import my_other_module
"""Suggested minimal approach to logging in python
"""

logging.basicConfig(level=logging.DEBUG,
                    format="%(asctime)s %(name)s %(levelname)s:%(message)s",
                    handlers=[
                              logging.StreamHandler(),
                              logging.FileHandler('log.log')
                             ]
                   )

logger = logging.getLogger(__name__)

logger.debug("Debug message")
logger.info("Info message")
logger.warning("Warning message")
logger.error("Error message")

my_module.function_in_module()
my_other_module.function_in_module()

One Separate Module

We can now use this configured-once-only logger in a separate module.

In one module file:

import logging

logger = logging.getLogger(__name__)

def function_in_module():
    logger.error("Error message in my_module")

Another Separate Module

The same approach applies in the other module file:

import logging

logger = logging.getLogger(__name__)

def function_in_module():
    logger.info("Info log from my_other_module")

The resulting logs

The following shows how the log messages appear in the command line (and also in the ‘log.log’ file specified in the basicConfig).

The formatting is the same across each logging line, and the filename of each module is listed.

2021-10-20 19:30:10,659 __main__ DEBUG:Debug message
2021-10-20 19:30:10,669 __main__ INFO:Info message
2021-10-20 19:30:10,669 __main__ WARNING:Warning message
2021-10-20 19:30:10,669 __main__ ERROR:Error message
2021-10-20 19:30:10,669 my_module ERROR:Error message in my_module
2021-10-20 19:30:10,677 my_other_module INFO:Info log from my_other_module