How to Write to a File, Using the Logging Python Module

How to write to a file, using the logging Python module?

An example of using logging.basicConfig rather than logging.fileHandler()

logging.basicConfig(filename=logname,
                    filemode='a',
                    format='%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s',
                    datefmt='%H:%M:%S',
                    level=logging.DEBUG)

logging.info("Running Urban Planning")

logger = logging.getLogger('urbanGUI')

In order, the five parts do the following:

1. set the output file (filename=logname)
2. set it to append rather than overwrite (filemode='a')
3. determine the format of the output message (format=...)
4. determine the format of the output time (datefmt='%H:%M:%S')
5. and determine the minimum message level it will accept (level=logging.DEBUG).

logger configuration to log to file and print to stdout

Just get a handle to the root logger and add the StreamHandler. The StreamHandler writes to stderr. Not sure if you really need stdout over stderr, but this is what I use when I setup the Python logger and I also add the FileHandler as well. Then all my logs go to both places (which is what it sounds like you want).

import logging
logging.getLogger().addHandler(logging.StreamHandler())

If you want to output to stdout instead of stderr, you just need to specify it to the StreamHandler constructor.

import sys
# ...
logging.getLogger().addHandler(logging.StreamHandler(sys.stdout))

You could also add a Formatter to it so all your log lines have a common header.

ie:

import logging
logFormatter = logging.Formatter("%(asctime)s [%(threadName)-12.12s] [%(levelname)-5.5s]  %(message)s")
rootLogger = logging.getLogger()

fileHandler = logging.FileHandler("{0}/{1}.log".format(logPath, fileName))
fileHandler.setFormatter(logFormatter)
rootLogger.addHandler(fileHandler)

consoleHandler = logging.StreamHandler()
consoleHandler.setFormatter(logFormatter)
rootLogger.addHandler(consoleHandler)

Prints to the format of:

2012-12-05 16:58:26,618 [MainThread  ] [INFO ]  my message

How to save log file using Python?

You can write/save the log file using the below command.

logging.basicConfig(filename='path to the log file', level=...)

There is a filemode option to overwrite the file.
To overwrite you can specifie filemode:w

logging.basicConfig(filename='logs.log',
            filemode='w',
            level=logging.INFO)

filemode: Specifies the mode to open the file, if filename is specified (if filemode is unspecified, it defaults to ‘a’).

How to write logging messages to a file

Here's some boilerplate code I always use for Python logging. This is for Windows. If you're using UNIX you'll need to change the forward slashes to backslashes

import os
import logging
import datetime as dt
import time

LOG_FILE = os.getcwd() + "/logs"
if not os.path.exists(LOG_FILE):
    os.makedirs(LOG_FILE)
LOG_FILE = LOG_FILE + "/" + dt.datetime.fromtimestamp(time.time()).strftime('%Y_%m_%d %H_%M_%S') + ".log"
logFormatter = logging.Formatter("%(levelname)s %(asctime)s %(processName)s %(message)s")
fileHandler = logging.FileHandler("{0}".format(LOG_FILE))
fileHandler.setFormatter(logFormatter)
rootLogger = logging.getLogger()
rootLogger.addHandler(fileHandler)
rootLogger.setLevel(logging.INFO)

logging.info("testing out the logging functionality")

You should end up with a log file that has a date timestamp

Create a log file

You can use the logging module to accomplish this.

At the very easiest level, it will be set up like this:

logging.basicConfig(filename="logfilename.log", level=logging.INFO)

There are a number of different levels that you can use to write to the file, such as:

logging.info('your text goes here')
logging.error('your text goes here')
logging.debug('your text goes here')

You can use these lines anywhere that you want to log to the file. If you want to replace the console printing with logging all together, just replace the print lines with logging.info(.......)

For more info on the topic, such as more configurable options (such as timestamps), check the docs (python 3): https://docs.python.org/3/library/logging.html

How to use python logging to log to files?

UPDATE

Looks like it's not logging because the logging level of the logger is set to logging.WARN. You must explicitly set the level on the logger to logging.DEBUG as well. I think the log/factory.log file is not being created because the log messages are not making it to that point yet. See http://dbgr.cc/v for a live demo of the code below:

import logging
import os

os.makedirs("log")

logger = logging.getLogger('factory')
fh = logging.FileHandler('log/factory.log')
fh.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s %(levelname)s: %(funcName)s:%(lineno)d %(message)s')
fh.setFormatter(formatter)
logger.addHandler(fh)

# notice that only the warning message is written
logger.debug("DEBUG MESSAGE")
logger.warn("WARNING MESSAGE")
logger.info("INFO MESSAGE")

with open("log/factory.log", "r") as f:
    print f.read()

# now update the level on the Logger to logging.DEBUG:
logger.setLevel(logging.DEBUG)

logger.debug("DEBUG MESSAGE")
logger.warn("WARNING MESSAGE")
logger.info("INFO MESSAGE")

with open("log/factory.log", "r") as f:
    print f.read()

Demo the code below at http://dbgr.cc/7:

import logging
import os

os.makedirs("log")

formatter = logging.Formatter('%(asctime)s %(levelname)s: %(funcName)s:%(lineno)d %(message)s')

# create logger with 'spam_application'
logger = logging.getLogger('factory')
logger.setLevel(logging.DEBUG)
fh = logging.FileHandler('log/factory.log')
fh.setLevel(logging.DEBUG)
fh.setFormatter(formatter)

# just so we can see things as they happen in stdout/stderr
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
ch.setFormatter(formatter)

# add the handlers to the logger
logger.addHandler(fh)
logger.addHandler(ch)

logger.debug("Debug log msg")
logger.info("Info log msg")
logger.warn("Warning log msg")

with open("log/factory.log", "r") as f:
    print f.read()

Logging module not writing to file

You can try running this snippet in your main file.

import logging 
logging.basicConfig(
    level=logging.INFO, 
    format='%(asctime)s [%(levelname)s] - %(message)s',
    filename='filename.txt')  # pass explicit filename here 
logger = logging.get_logger()  # get the root logger
logger.warning('This should go in the file.')
print logger.handlers   # you should have one FileHandler object

I can't get Python logging module to write to the log file from different module

I've rewritten your scripts a little so that this code can stand alone. If I changed this too much let me know and I can revisit it. These two files are an example of having it log to file. Note my comments:

##  main.py

import logging
from process import process_data
import os

def main():
    # Give this logger a name
    logger = logging.getLogger("Example")
    logger.setLevel(logging.DEBUG)

    formatter = logging.Formatter('%(asctime)s:%(name)s:%(message)s')
    # I changed this to the same directory, for convenience 
    templogfile = os.path.join(os.getcwd(), 'TST_HA_Debug.log')
    file_handler = logging.FileHandler(templogfile)
    file_handler.setLevel(logging.DEBUG)
    file_handler.setFormatter(formatter)

    stream_handler = logging.StreamHandler()
    stream_handler.setFormatter(formatter)

    logger.addHandler(file_handler)
    logger.addHandler(stream_handler)

    logging.getLogger("Example").debug('Logging has started') # This still gets written to the file

    process_data() # call process_data in process.py

if __name__ == '__main__':
    main()

##  process.py

import logging

def process_data(data=None):

    # make sure to grab the correct logger
    logger = logging.getLogger("Example")

    logger.debug('This message is logged by process') # This does get logged to file now

   #do some stuff with data here and log some msgs

    return

Why does this work? Because the module-level functions use the default root logger, which is not the one you've configured. For more details on this see these docs. There is a similar question that goes more into depth here.

By getting the configured logger before you start logging, you are able to log to the right configuration. Hope this helps!

---

Related Topics