Get Last N Lines of a File, Similar to Tail

Get last n lines of a file, similar to tail

The code I ended up using. I think this is the best so far:

def tail(f, n, offset=None):
    """Reads a n lines from f with an offset of offset lines.  The return
    value is a tuple in the form ``(lines, has_more)`` where `has_more` is
    an indicator that is `True` if there are more lines in the file.
    """
    avg_line_length = 74
    to_read = n + (offset or 0)

    while 1:
        try:
            f.seek(-(avg_line_length * to_read), 2)
        except IOError:
            # woops.  apparently file is smaller than what we want
            # to step back, go to the beginning instead
            f.seek(0)
        pos = f.tell()
        lines = f.read().splitlines()
        if len(lines) >= to_read or pos == 0:
            return lines[-to_read:offset and -offset or None], \
                   len(lines) > to_read or pos > 0
        avg_line_length *= 1.3

Get last n lines or bytes of a huge file in Windows (like Unix's tail). Avoid time consuming options

How about this (reads last 8 bytes for demo):

$fpath = "C:\10GBfile.dat"
$fs = [IO.File]::OpenRead($fpath)
$fs.Seek(-8, 'End') | Out-Null
for ($i = 0; $i -lt 8; $i++)
{
    $fs.ReadByte()
}

---

UPDATE. To interpret bytes as string (but be sure to select correct encoding - here UTF8 is used):

$N = 8
$fpath = "C:\10GBfile.dat"
$fs = [IO.File]::OpenRead($fpath)
$fs.Seek(-$N, [System.IO.SeekOrigin]::End) | Out-Null
$buffer = new-object Byte[] $N
$fs.Read($buffer, 0, $N) | Out-Null
$fs.Close()
[System.Text.Encoding]::UTF8.GetString($buffer)

---

UPDATE 2. To read last M lines, we'll be reading the file by portions until there are more than M newline char sequences in the result:

$M = 3
$fpath = "C:\10GBfile.dat"

$result = ""
$seq = "`r`n"
$buffer_size = 10
$buffer = new-object Byte[] $buffer_size

$fs = [IO.File]::OpenRead($fpath)
while (([regex]::Matches($result, $seq)).Count -lt $M)
{
    $fs.Seek(-($result.Length + $buffer_size), [System.IO.SeekOrigin]::End) | Out-Null
    $fs.Read($buffer, 0, $buffer_size) | Out-Null
    $result = [System.Text.Encoding]::UTF8.GetString($buffer) + $result
}
$fs.Close()

($result -split $seq) | Select -Last $M

Try playing with bigger $buffer_size - this ideally is equal to expected average line length to make fewer disk operations. Also pay attention to $seq - this could be \r\n or just \n.
This is very dirty code without any error handling and optimizations.

Most efficient way to search the last X lines of a file?

# Tail
from __future__ import with_statement

find_str = "FIREFOX"                    # String to find
fname = "g:/autoIt/ActiveWin.log_2"     # File to check

with open(fname, "r") as f:
    f.seek (0, 2)           # Seek @ EOF
    fsize = f.tell()        # Get Size
    f.seek (max (fsize-1024, 0), 0) # Set pos @ last n chars
    lines = f.readlines()       # Read to end

lines = lines[-10:]    # Get last 10 lines

# This returns True if any line is exactly find_str + "\n"
print find_str + "\n" in lines

# If you're searching for a substring
for line in lines:
    if find_str in line:
        print True
        break

Read last n lines of file (tail) without reading it line-by-line?

Something like this:

• Use seek() to get something like the last 4096 bytes of a file.
• See how many newlines you have in those bytes. If you have n or more, then you're done. If you have fewer, then read the previous 4096 bytes until you're done.

Not sure if there's a built-in way to do this.

Copying last n lines to a new file and then removing the n lines from original

Not all versions of head support negative line counts.
The default installed on macOS doesn't.

If you have coreutils installed (If you have Homebrew installed you can do this: brew install coreutils) you should be able to use ghead -n -3000.

Retrieve last 100 lines logs

You can use tail command as follows:

tail -100 <log file>   > newLogfile

Now last 100 lines will be present in newLogfile

EDIT:

More recent versions of tail as mentioned by twalberg use command:

tail -n 100 <log file>   > newLogfile

Last n rows of each files in a directory

A for loop is not the best, since it will not support all cases. Ex. if you have spaces in your filenames.

To avoid all issues, use:

find . -type f -name "*.log" -exec tail -5 {} \; -print

• .: is the current directory. If your log files are not in the current directory, you can replace the . by the directory containing the files.
• -name "*.log" can be modified to filter your files more precisely. Ex. file*.log.
• tail -5 {} will print the last 5 lines. Change the number as you require.
• the -print option will print the filenames of logs found. But you can omit it if you do not need that information in your display.

---

Related Topics