Duplicate Log Output When Using Python Logging Module

Duplicate log output when using Python logging module

The logging.getLogger() is returns the same instance for a given name. (Documentation)

The problem is that every time you call myLogger(), it's adding another handler to the instance, which causes the duplicate logs.

Perhaps something like this?

import os
import time
import datetime
import logging

loggers = {}

def myLogger(name):
    global loggers
    
    if loggers.get(name):
        return loggers.get(name)
    else:
        logger = logging.getLogger(name)
        logger.setLevel(logging.DEBUG)
        now = datetime.datetime.now()
        handler = logging.FileHandler(
            '/root/credentials/Logs/ProvisioningPython' 
            + now.strftime("%Y-%m-%d") 
            + '.log')
        formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
        handler.setFormatter(formatter)
        logger.addHandler(handler)
        loggers[name] = logger
                       
        return logger

Python Logging Duplicate Output

As far as I remember the setting propagates to higher level (ancestor) loggers. You can read about it here.

I would suggest adding logger.propagate = False after initializing logger a:

logger = logging.getLogger('a')
logger.propagate = False
logger.setLevel(logging.DEBUG)

duplicate output in simple python logging configuration

I figured this out, thanks Paco for pointing me in the right direction

it turns out that when getLogger is called, handlers are added to it:

>>> print(effectivehandlers(logger))
[<logging.StreamHandler object at 0x305ad90>, <logging.FileHandler object at 0x305ae10>]
>>> logging.getLogger(name) #the same logger
<logging.Logger object at 0x7fa08fb9b2d0>
>>> print(effectivehandlers(logger))
[<logging.StreamHandler object at 0x305ad90>, <logging.FileHandler object at 0x305ae10>, <logging.StreamHandler object at 0x305ad90>, <logging.FileHandler object at 0x305ae10>]

Now, both the child and the parent have the same handlers. Hence duplicate outputs.

Python logging duplicated

Every time fetch_logger is called it adds a new FileHandler to the logger. Each FileHandler writes to the log file, resulting in the repeating output in the file.

One solution is to call the logger's hasHandlers method. This will return True if any handlers have been configured on the logger, and then you can delete them.

def fetchLogger(name="None") :
    logger = logging.getLogger(__name__)
    if logger.hasHandlers():
        # Logger is already configured, remove all handlers
        logger.handlers = []
    # Configure the logger as before.
    ...

Duplicate log output, but no duplicate handlers

Loggers propagate log events to their ancestors' handlers by default. a_logger may only have one handler, but its parent, the root logger, also has a handler (actually the same handler). a_logger.info('hi') is handled by both handlers (actually the same handler twice).

You don't need to attach the same handler to every logger. Attaching it to the root logger is enough.

log messages appearing twice with Python Logging

You are calling configure_logging twice (maybe in the __init__ method of Boy) : getLogger will return the same object, but addHandler does not check if a similar handler has already been added to the logger.

Try tracing calls to that method and eliminating one of these. Or set up a flag logging_initialized initialized to False in the __init__ method of Boy and change configure_logging to do nothing if logging_initialized is True, and to set it to True after you've initialized the logger.

If your program creates several Boy instances, you'll have to change the way you do things with a global configure_logging function adding the handlers, and the Boy.configure_logging method only initializing the self.logger attribute.

Another way of solving this is by checking the handlers attribute of your logger:

logger = logging.getLogger('my_logger')
if not logger.handlers:
    # create the handlers and call logger.addHandler(logging_handler)

Logging module for print statements: Duplicate log entries

Here's at least a partial answer to your first question. You're getting those blank lines on eachprintbecauseprintstatements in Python 2.x potentially callstdout.write()twice, once with the data from evaluating the expression in the statement, and then again if the optional newline hasn't been suppressed by a trailing comma.

Also, if anINFO level message is sent to a logger set at alevelof'NOTSET', the message will also be echoed on sys.stderrby default according to the logger documentation -- which is why you see the console output even when the Tee is in effect. There are no duplicate log entries that I can see.

To prevent the blanks lines, try using thisTeeclass definition. Note thewrite()method modifications (updated to be a singleton to match your "EDIT SECTION (3)"):

class Tee(object):
    _instance = None
    def __init__(self, logger):
        self.stdout = sys.stdout
        self.logger = logger
        sys.stdout = self
    def __new__(cls, *args, **kwargs):
        if not cls._instance:
            cls._instance = super(Tee, cls).__new__(cls, *args, **kwargs)
        return cls._instance
    def __del__(self):
        sys.stdout = self.stdout
    def write(self, data):
        data = data.rstrip('\r\n')
        if data: # anything left?
           self.logger.info(data)

With this and all your other updates in place it seems to work for me (including with the uncommented aux2 stuff. Are there some remaining problems from your point of view? If so, your initially long question has gotten even more so and should be completely cleaned-up leaving just the most recent code and focus on any remaining issue(s).

Reset python Logger to prevent duplicate logging records

Solved it:

I created a new method destroy_logger which is triggered at the end of the entire process. It closes and deletes all handlers. This is what's inside of the method:

def destroy_logger():
    
    global logger
    
    while logger.hasHandlers():
        logger.handlers[0].close
        logger.removeHandler(logger.handlers[0])

This question here helped me solving it. It also mentions a problem I noticed as well: The log file cannot be deleted as long as the IDE is opened. This problem is also solved by my method above.

---

Related Topics