Copy Large Files (Over 2 Gb) in PHP

Copy large files (over 2 GB) in PHP

This should do it.

function chunked_copy($from, $to) {
    # 1 meg at a time, you can adjust this.
    $buffer_size = 1048576; 
    $ret = 0;
    $fin = fopen($from, "rb");
    $fout = fopen($to, "w");
    while(!feof($fin)) {
        $ret += fwrite($fout, fread($fin, $buffer_size));
    }
    fclose($fin);
    fclose($fout);
    return $ret; # return number of bytes written
}

How to copy very large files from URL to server via PHP?

I think the problem might be the 30 second time-out on many servers running PHP scripts.

PHP scripts running via cron or shell wont have that problem so perhaps you could find a way to do it that way.

Alternatively you could add set_time_limit([desired time]) to the start of your code.

Reading very large files in PHP

Are you sure that it's fopen that's failing and not your script's timeout setting? The default is usually around 30 seconds or so, and if your file is taking longer than that to read in, it may be tripping that up.

Another thing to consider may be the memory limit on your script - reading the file into an array may trip over this, so check your error log for memory warnings.

If neither of the above are your problem, you might look into using fgets to read the file in line-by-line, processing as you go.

$handle = fopen("/tmp/uploadfile.txt", "r") or die("Couldn't get handle");
if ($handle) {
    while (!feof($handle)) {
        $buffer = fgets($handle, 4096);
        // Process buffer here..
    }
    fclose($handle);
}

Edit

PHP doesn't seem to throw an error, it just returns false.

Is the path to $rawfile correct relative to where the script is running? Perhaps try setting an absolute path here for the filename.

How to upload large files above 500MB in PHP

Do you think if increasing upload size limit will solve the problem? what if uploading 2GB file, what's happening then? Do you take into consideration the memory usage of such a script?

Instead, what you need is chunked upload, see here : Handling plupload's chunked uploads on the server-side
and here : File uploads; How to utilize "chunking"?

How can I download files larger than 2Gb in PHP?

I fixed the problem in the end by eliminating the readfile() function and outputting the content of the file myself using fopen() and fread().

However the reason why readfile()' fails when dealing with such large files remains an open subject (I've read the documentation on php.net and I couldn't find any notes about such issue). So i'm still waiting for that experienced programmer to enlighten me :).

Very large uploads with PHP

How about a Java applet? That's how we had to do it at a company I previously worked for. I know applets suck, especially in this day and age with all our options available, but they really are the most versatile solution to desktop-like problems encountered in web development. Just something to consider.

---

Related Topics