How to Watch a File Write in PHP

How To watch a file write in PHP?

I don't believe that there's some magical way to do it. You just have to continuously poll the file size and output any new data. This is actually quite easy, and the only real thing to watch out for is that file sizes and other stat data is cached in php. The solution to this is to call clearstatcache() before outputting any data.

Here's a quick sample, that doesn't include any error handling:

function follow($file)
{
    $size = 0;
    while (true) {
        clearstatcache();
        $currentSize = filesize($file);
        if ($size == $currentSize) {
            usleep(100);
            continue;
        }

        $fh = fopen($file, "r");
        fseek($fh, $size);

        while ($d = fgets($fh)) {
            echo $d;
        }

        fclose($fh);
        $size = $currentSize;
    }
}

follow("file.txt");

PHP: How to read a file live that is constantly being written to

You need to loop with sleep:

$file='/home/user/youfile.txt';
$lastpos = 0;
while (true) {
    usleep(300000); //0.3 s
    clearstatcache(false, $file);
    $len = filesize($file);
    if ($len < $lastpos) {
        //file deleted or reset
        $lastpos = $len;
    }
    elseif ($len > $lastpos) {
        $f = fopen($file, "rb");
        if ($f === false)
            die();
        fseek($f, $lastpos);
        while (!feof($f)) {
            $buffer = fread($f, 4096);
            echo $buffer;
            flush();
        }
        $lastpos = ftell($f);
        fclose($f);
    }
}

(tested.. it works)

PHP output file on disk to browser

There is fpassthru() that should do exactly what you need. See the manual entry to read about the following example:

<?php

// open the file in a binary mode
$name = './img/ok.png';
$fp = fopen($name, 'rb');

// send the right headers
header("Content-Type: image/png");
header("Content-Length: " . filesize($name));

// dump the picture and stop the script
fpassthru($fp);
exit;

?>

See here for all of PHP's filesystem functions.

If it's a binary file you want to offer for download, you probably also want to send the right headers so the "Save as.." dialog pops up. See the 1st answer to this question for a good example on what headers to send.

What is the best way to read last lines (i.e. tail) from a file using PHP?

Methods overview

Searching on the internet, I came across different solutions. I can group them
in three approaches:

naive ones that use file() PHP function;
cheating ones that runs tail command on the system;
mighty ones that happily jump around an opened file using fseek().

I ended up choosing (or writing) five solutions, a naive one, a cheating one
and three mighty ones.

The most concise naive solution,
using built-in array functions.
The only possible solution based on tail command, which has
a little big problem: it does not run if tail is not available, i.e. on
non-Unix (Windows) or on restricted environments that don't allow system
functions.
The solution in which single bytes are read from the end of file searching
for (and counting) new-line characters, found here.
The multi-byte buffered solution optimized for large files, found
here.
A slightly modified version of solution #4 in which buffer length is
dynamic, decided according to the number of lines to retrieve.

All solutions work. In the sense that they return the expected result from
any file and for any number of lines we ask for (except for solution #1, that can
break PHP memory limits in case of large files, returning nothing). But which one
is better?

Performance tests

To answer the question I run tests. That's how these thing are done, isn't it?

I prepared a sample 100 KB file joining together different files found in
my /var/log directory. Then I wrote a PHP script that uses each one of the
five solutions to retrieve 1, 2, .., 10, 20, ... 100, 200, ..., 1000 lines
from the end of the file. Each single test is repeated ten times (that's
something like 5 × 28 × 10 = 1400 tests), measuring average elapsed
time in microseconds.

I run the script on my local development machine (Xubuntu 12.04,
PHP 5.3.10, 2.70 GHz dual core CPU, 2 GB RAM) using the PHP command line
interpreter. Here are the results:

Execution time on sample 100 KB log file

Solution #1 and #2 seem to be the worse ones. Solution #3 is good only when we need to
read a few lines. Solutions #4 and #5 seem to be the best ones.
Note how dynamic buffer size can optimize the algorithm: execution time is a little
smaller for few lines, because of the reduced buffer.

Let's try with a bigger file. What if we have to read a 10 MB log file?

Execution time on sample 10 MB log file

Now solution #1 is by far the worse one: in fact, loading the whole 10 MB file
into memory is not a great idea. I run the tests also on 1MB and 100MB file,
and it's practically the same situation.

And for tiny log files? That's the graph for a 10 KB file:

Execution time on sample 10 KB log file

Solution #1 is the best one now! Loading a 10 KB into memory isn't a big deal
for PHP. Also #4 and #5 performs good. However this is an edge case: a 10 KB log
means something like 150/200 lines...

You can download all my test files, sources and results
here.

Final thoughts

Solution #5 is heavily recommended for the general use case: works great
with every file size and performs particularly good when reading a few lines.

Avoid solution #1 if you
should read files bigger than 10 KB.

Solution #2
and #3
aren't the best ones for each test I run: #2 never runs in less than
2ms, and #3 is heavily influenced by the number of
lines you ask (works quite good only with 1 or 2 lines).

How to stream a media file using PHP?

I think you need to implement the Range header, so that the client can skip to a specific position in the file. You can probably find out what goes wrong by sniffing the request the player sends.

Streaming a text file, auto update

Well, I would not do this the Shrek's Donkey way:

"Are we there, yet?", 5 seconds later, "Are we there, yet?"

Are we there, yet?", 5 seconds later, "Are we there, yet?

What is the problem with this?

Bandwidth consumption, pure and simply.

What should I do?

The answer is: Server Push. Or something like Server Push, since is not possible to start things from the server with HTTP.

Instead of poll the server every 5 seconds if there is a new version of the file, why not let the server notify you just when it really has changed?

And how do I do this?

The answer is: with Ajax, but with a different approach.

The high level algorithm is:

Send an Ajax request to the server from the browser.
When the server receives the request, verify if the file has changed.
1. If it has changed, then return the new data.
2. If not, make the server sleep for a while and then go back to step 2.
If a long time has passed and there is no modification, the server could (or must) return a response, with no data bounded to it.

How can I check if the file has changed?

Instead of reading the file and comparing the data, a better approach is to send with the request the timestamp of when the file had it last change. You can do this with the filemtime funciton.

On the server, you verify if the file last modified time is bigger than the one coming from the request. If it is, then you read the file and send the response along with the new file modified time (step 2a). If not, go for step 2b and use the usleep function to make the server sleep for a while and save CPU.

To know if a long time has passed and there's no change, you can use the microtime function at the beginning of the script and make the difference of its value on every iteration. If there has been past much time, you'll send an empty response.

Making a draft, the server-side script would be like:

$startTime = microtime();
$filePath = '/path/to/file.txt';
$lastModifiedTime = $_GET['lastModifiedTyme']; // Supposing it comes from the query string

$currentModifiedTime = null;
while ($currentModifiedTime = filemtime($filePath) < $lastModifiedTime) {
    usleep(1000); // dorme por 1 seg.

    // If has passed more than 1 minute
    if ((microtime() - $startTime) > 60000) {
        header('HTTP/1.1 304 Not Modified'); // Or simply http_response_code(304) for PHP 5.4+
        exit;
    }
}

$data = file_get_contents($file);
$jsonResponse = json_encode(array(
    'data' => $data,
    'modifiedTime' => $currentModifiedTime
);

echo $jsonResponse;

On the client side, you'll have to re-run the request every time you receive a response. This could (and should) have a litte delay.

It sounds like a kind of an overhead, I know, but just for you to know that there are another ways of doing this.

Filestreaming to html5 in PHP without writing a file to the filesystem

For posterity, here's my final result, which seems to work well without needing temp files.

Not certain if it would make more sense to include the query into the class and just pass the $MediaFileID to the class, but this works as is, so for now I've left it.

The entire modified class is included with its original credit info:

function ExecutereadMediaSP($MediaFileID){

    try{

    $connection = ConnectToDB();

    ini_set('memory_limit', '-1');

    // logs basic info about media viewer.  Mainly for a basic hit counter.
    LogMediaRequest($connection, $MediaFileID);

    //Selects binary data from SQL Server based on MediaFileID.  Passes this to Stream Class.  May be able to make further improvements later.
    $sql = "SELECT ... data from FILESTREAM column based on file ID ...";   

    $rst = $connection->prepare($sql);
    $rst->execute();
    $rst->bindColumn(1, $filecontent, PDO::PARAM_LOB, 0, PDO::SQLSRV_ENCODING_BINARY);

    $row = $rst->fetch(PDO::FETCH_ASSOC);//sql can only return one row due to unique identifier

    //Stream file 
    $stream = new VideoStream($filecontent);
    $stream->start();

    //Clean up.
    $rst->closeCursor();
    unset($rst);
    $connection = null;

    } catch (Exception $e) {
        error_log("Error in getting video\n".$e->getMessage(),0);
    }

}

/**
 * VideoStream - PHP class that supports (adaptive) streaming of files
 *
 * @author Rana
 * modified by HazCod to use stream_get_contents and correct session shutoff
 * https://github.com/HazCod
 * @link http://codesamplez.com/programming/php-html5-video-streaming-tutorial
 */
class VideoStream
{
    private $path = "";
    private $stream = "";
    private $buffer = 102400;
    private $start  = -1;
    private $end    = -1;
    private $size   = 0;

    function __construct($filecontent) 
    {

        try{

        // Opens file handle resource to replace use of actual file in the file system.
        $file_handle = fopen('php://memory', 'r+', false, stream_context_create()); 

        // Writes data from SQL Query to file wrapper.
        fwrite($file_handle, $filecontent);

        // Moves pointer to beginning of file.
        fseek($file_handle, 0);

        // Gets info on the "file."  Required to get filesize.
        $fstat = array();

        // gather statistics
        $fstat = fstat($file_handle);

        //Set File Size for Stream Class.
        $this->size  = $fstat['size'];

        // Define Stream as "File."
        $this->stream = $file_handle;

        } catch (PDOException $e) {
            error_log("Error in getting video\n".$e->getMessage(),0);
        }

    }

    /**
     * Set proper header to serve the video content
     */
    private function setHeader()
    {   
        ob_get_clean();
        header("Content-Type: video/mp4");
        header("Cache-Control: max-age=2592000, public");
        header("Expires: ".gmdate('D, d M Y H:i:s', time()+2592000) . ' GMT');
      //  header("Last-Modified: ".gmdate('D, d M Y H:i:s', @filemtime($this->path)) . ' GMT' );
        $this->start = 0;

        $this->end   = $this->size - 1;
       // header("Accept-Ranges: 0-".$this->end);

        if (isset($_SERVER['HTTP_RANGE'])) {

            $c_start = $this->start;
            $c_end = $this->end;

            list(, $range) = explode('=', $_SERVER['HTTP_RANGE'], 2);
            if (strpos($range, ',') !== false) {
                header('HTTP/1.1 416 Requested Range Not Satisfiable');
                header("Content-Range: bytes $this->start-$this->end/$this->size");
                exit;
            }
            if ($range == '-') {
                $c_start = $this->size - substr($range, 1);
            }else{
                $range = explode('-', $range);
                $c_start = $range[0];

                $c_end = (isset($range[1]) && is_numeric($range[1])) ? $range[1] : $c_end;
            }
            $c_end = ($c_end > $this->end) ? $this->end : $c_end;
            if ($c_start > $c_end || $c_start > $this->size - 1 || $c_end >= $this->size) {
                header('HTTP/1.1 416 Requested Range Not Satisfiable');
                header("Content-Range: bytes $this->start-$this->end/$this->size");
                exit;
            }
            $this->start = $c_start;
            $this->end = $c_end;
            $length = $this->end - $this->start + 1;
            fseek($this->stream, $this->start);
            header('HTTP/1.1 206 Partial Content');
            header("Content-Length: ".$length);
            header("Content-Range: bytes $this->start-$this->end/".$this->size);
        }
        else
        {
            header("Content-Length: ".$this->size);
        }  

    }

    /**
     * close curretly opened stream
     */
    private function end()
    {
        fclose($this->stream);
        exit;
    }

    /**
     * perform the streaming of calculated range
     */
    private function stream()
    {
        $i = $this->start;
        set_time_limit(0);
        while(!feof($this->stream) && $i <= $this->end && connection_aborted() == 0) {
            $bytesToRead = $this->buffer;
            if(($i+$bytesToRead) > $this->end) {
                $bytesToRead = $this->end - $i + 1;
            }
            $data = stream_get_contents($this->stream, $bytesToRead);
            echo $data;
            flush();
            $i += $bytesToRead;
        }
    }

    /**
     * Start streaming video content
     */
    function start()
    {
        session_write_close(); //ensure our session is written away before streaming, else we cannot use it elsewhere
        $this->setHeader();
        $this->stream();
        $this->end();
    }
}

How to Watch a File Write in PHP