How to Tail a Log File in Python

How can I tail a log file in Python?

So, this is coming quite late, but I ran into the same problem again, and there's a much better solution now. Just use pygtail:

Pygtail reads log file lines that have not been read. It will even
handle log files that have been rotated. Based on logcheck's logtail2
(http://logcheck.org)

Is there a way in python to tail a file and work with the line-output

I can't help with the regex because you did not post an example of how the log file looks like. I can help with the tail of the file, since that's quite universal.

data = []
with open(logfile,'rt',encoding='utf-8')as infile:
    for i,e in enumerate(infile):
        data.append(e.strip())
    if i < 11:
        for line in data:
            print(line)
    else:
        for line in data[:-10]:
            print(line)

In your case you would save the "print" into a container and then run your regex against it.

Python script to tail a log file

I recommend replacing time.sleep with asyncio.sleep

How to do log tailing in realtime efficiently and in an unblocking manner in tornado python

Use tornado.process.Subprocess instead of subprocess.Popen (and its STREAM option instead of PIPE). This lets you read from the subprocess asynchronously:

async def tail(self, file):
    self.__process = Subprocess(["tail", "-n", "1", "-f", file], stdout=Subprocess.STREAM)
    while True:
        line = await self.__process.stdout.read_until(b"\n")
        do_something(line)

def start_tail(self):
    IOLoop.current().spawn_callback(self.tail, self.__log_path)

Get last n lines of a file, similar to tail

The code I ended up using. I think this is the best so far:

def tail(f, n, offset=None):
    """Reads a n lines from f with an offset of offset lines.  The return
    value is a tuple in the form ``(lines, has_more)`` where `has_more` is
    an indicator that is `True` if there are more lines in the file.
    """
    avg_line_length = 74
    to_read = n + (offset or 0)

    while 1:
        try:
            f.seek(-(avg_line_length * to_read), 2)
        except IOError:
            # woops.  apparently file is smaller than what we want
            # to step back, go to the beginning instead
            f.seek(0)
        pos = f.tell()
        lines = f.read().splitlines()
        if len(lines) >= to_read or pos == 0:
            return lines[-to_read:offset and -offset or None], \
                   len(lines) > to_read or pos > 0
        avg_line_length *= 1.3

Create a log file

You can use the logging module to accomplish this.

At the very easiest level, it will be set up like this:

logging.basicConfig(filename="logfilename.log", level=logging.INFO)

There are a number of different levels that you can use to write to the file, such as:

logging.info('your text goes here')
logging.error('your text goes here')
logging.debug('your text goes here')

You can use these lines anywhere that you want to log to the file. If you want to replace the console printing with logging all together, just replace the print lines with logging.info(.......)

For more info on the topic, such as more configurable options (such as timestamps), check the docs (python 3): https://docs.python.org/3/library/logging.html

---

Related Topics