How to Read Last Lines (I.E. "Tail") from a File Using PHP

What is the best way to read last lines (i.e. tail ) from a file using PHP?

Methods overview

Searching on the internet, I came across different solutions. I can group them
in three approaches:

• naive ones that use file() PHP function;
• cheating ones that runs tail command on the system;
• mighty ones that happily jump around an opened file using fseek().

I ended up choosing (or writing) five solutions, a naive one, a cheating one
and three mighty ones.

1. The most concise naive solution,
   using built-in array functions.
2. The only possible solution based on tail command, which has
   a little big problem: it does not run if tail is not available, i.e. on
   non-Unix (Windows) or on restricted environments that don't allow system
   functions.
3. The solution in which single bytes are read from the end of file searching
   for (and counting) new-line characters, found here.
4. The multi-byte buffered solution optimized for large files, found
   here.
5. A slightly modified version of solution #4 in which buffer length is
   dynamic, decided according to the number of lines to retrieve.

All solutions work. In the sense that they return the expected result from
any file and for any number of lines we ask for (except for solution #1, that can
break PHP memory limits in case of large files, returning nothing). But which one
is better?

Performance tests

To answer the question I run tests. That's how these thing are done, isn't it?

I prepared a sample 100 KB file joining together different files found in
my /var/log directory. Then I wrote a PHP script that uses each one of the
five solutions to retrieve 1, 2, .., 10, 20, ... 100, 200, ..., 1000 lines
from the end of the file. Each single test is repeated ten times (that's
something like 5 × 28 × 10 = 1400 tests), measuring average elapsed
time in microseconds.

I run the script on my local development machine (Xubuntu 12.04,
PHP 5.3.10, 2.70 GHz dual core CPU, 2 GB RAM) using the PHP command line
interpreter. Here are the results:

Solution #1 and #2 seem to be the worse ones. Solution #3 is good only when we need to
read a few lines. Solutions #4 and #5 seem to be the best ones.
Note how dynamic buffer size can optimize the algorithm: execution time is a little
smaller for few lines, because of the reduced buffer.

Let's try with a bigger file. What if we have to read a 10 MB log file?

Now solution #1 is by far the worse one: in fact, loading the whole 10 MB file
into memory is not a great idea. I run the tests also on 1MB and 100MB file,
and it's practically the same situation.

And for tiny log files? That's the graph for a 10 KB file:

Solution #1 is the best one now! Loading a 10 KB into memory isn't a big deal
for PHP. Also #4 and #5 performs good. However this is an edge case: a 10 KB log
means something like 150/200 lines...

You can download all my test files, sources and results
here.

Final thoughts

Solution #5 is heavily recommended for the general use case: works great
with every file size and performs particularly good when reading a few lines.

Avoid solution #1 if you
should read files bigger than 10 KB.

Solution #2
and #3
aren't the best ones for each test I run: #2 never runs in less than
2ms, and #3 is heavily influenced by the number of
lines you ask (works quite good only with 1 or 2 lines).

Read last line from file

This should work:

$line = '';

$f = fopen('data.txt', 'r');
$cursor = -1;

fseek($f, $cursor, SEEK_END);
$char = fgetc($f);

/**
 * Trim trailing newline chars of the file
 */
while ($char === "\n" || $char === "\r") {
    fseek($f, $cursor--, SEEK_END);
    $char = fgetc($f);
}

/**
 * Read until the start of file or first newline char
 */
while ($char !== false && $char !== "\n" && $char !== "\r") {
    /**
     * Prepend the new char
     */
    $line = $char . $line;
    fseek($f, $cursor--, SEEK_END);
    $char = fgetc($f);
}

fclose($f);

echo $line;

Note that this solution will repeat the last character of the line unless your file ends in a newline. If your file does not end in a newline, you can change both instances of $cursor-- to --$cursor.

PHP Tail - Getting the last 10 lines only

using that library is an overkill, the simple tail command will work fine in ubuntu

tail -n 10 /you/file/full/path/here

if you want to get this from within a php script you can use

$string = exec( 'tail -n 10 /you/file/full/path/here');

How to read only 5 last line of the text file in PHP?

Untested code, but should work:

$file = file("filename.txt");
for ($i = max(0, count($file)-6); $i < count($file); $i++) {
  echo $file[$i] . "\n";
}

Calling max will handle the file being less than 6 lines.

How would I tail a dated log file?

You've a couple of errors, try this:

<?php
$filename = date('Y_m_d').'.ilog'; // misplaced quotes
$filedir = '/full/path/to/gamelogs/'; // relative path instead of full path
$output = shell_exec("tail -n250  {$filedir}{$filename}"); //exec removed
echo str_replace(PHP_EOL, '<br />', $output); // line OK
?>

PHP: Retrieving lines from the end of a large text file

If you are on a 'nix machine, you should be able to use shell escaping and the tool 'tail'.
It's been a while, but something like this:

$lastLines = `tail -n 500`;

notice the use of tick marks, which executes the string in BASH or similar and returns the results.

Read a file one line at a time in node.js?

Since Node.js v0.12 and as of Node.js v4.0.0, there is a stable readline core module. Here's the easiest way to read lines from a file, without any external modules:

const fs = require('fs');
const readline = require('readline');

async function processLineByLine() {
  const fileStream = fs.createReadStream('input.txt');

  const rl = readline.createInterface({
    input: fileStream,
    crlfDelay: Infinity
  });
  // Note: we use the crlfDelay option to recognize all instances of CR LF
  // ('\r\n') in input.txt as a single line break.

  for await (const line of rl) {
    // Each line in input.txt will be successively available here as `line`.
    console.log(`Line from file: ${line}`);
  }
}

processLineByLine();

Or alternatively:

var lineReader = require('readline').createInterface({
  input: require('fs').createReadStream('file.in')
});

lineReader.on('line', function (line) {
  console.log('Line from file:', line);
});

The last line is read correctly (as of Node v0.12 or later), even if there is no final \n.

UPDATE: this example has been added to Node's API official documentation.

Delete a line in multiple text files with the same line beginning but varying line ending using Python v3.5

Ok, based on suggestions by Avinash Raj, tdelaney, and Sampson Oliver, here on Stack Overflow, and another friend who helped privately, here is the solution that is now working:

import os
indir = '/Users/dhunter/GRID01/'  # input directory
for i in os.listdir(indir):  # for each "i" (iteration) within the indir variable directory...
    if i.lower().endswith('.gps'):  # if the filename of an iteration ends with .GPS, then...
        if not i.lower().endswith('.gpsnew.gps'):  # if the filename does not end with .gpsnew.gps, then...
            print(i + ' loaded')  # print the filename to CLI.
            with open (indir + i, 'r') as my_file:
                for line in my_file:
                    if not line.startswith('$GNGSA'):
                        if not line.startswith('$GNVTG'):
                            with open(indir + i + 'new.gps', 'a') as outputfile:
                                outputfile.write(line)
                                outputfile.write('\r\n')

(You'll see I had to add in another layer of if statement to stop it from using the output files from previous uses of the script "if not i.lower().endswith('.gpsnew.gps'):", but this line can easily be deleted for anyone who uses these instructions in future)

We switched the open mode on the third-last line to "a" for append, so that it would save all the right lines to the file, rather than overwriting each time.

We also added in the final line to add a line break at the end of each line.

Thanks everyone for their help, explanations, and suggestions. Hopefully this solution will be useful to someone in future. :)

---

Related Topics