Very Large Uploads with PHP

Very large uploads with PHP

How about a Java applet? That's how we had to do it at a company I previously worked for. I know applets suck, especially in this day and age with all our options available, but they really are the most versatile solution to desktop-like problems encountered in web development. Just something to consider.

How to upload large files above 500MB in PHP

Do you think if increasing upload size limit will solve the problem? what if uploading 2GB file, what's happening then? Do you take into consideration the memory usage of such a script?

Instead, what you need is chunked upload, see here : Handling plupload's chunked uploads on the server-side
and here : File uploads; How to utilize "chunking"?

Upload large files and Time out php uploads

Resolved with: https://www.plupload.com/

    <div id="uploader">
        <p>Your browser doesnt have Flash, Silverlight or HTML5 support.</p>
    </div>

    <link href="https://rawgithub.com/moxiecode/plupload/master/js/jquery.plupload.queue/css/jquery.plupload.queue.css" type="text/css" rel="stylesheet" media="screen">
    <script type="text/javascript" src="assets/uploads/jquery-ui.js"></script>
    <script type="text/javascript" src="assets/uploads/plupload.full.min.js"></script>
    <script type="text/javascript" src="assets/uploads/jquery.plupload.queue/jquery.plupload.queue.min.js"></script>
    <script type="text/javascript" src="assets/uploads/jquery.ui.plupload/jquery.ui.plupload.min.js"></script>
    <link type="text/css" rel="stylesheet" href="assets/uploads/jquery.ui.plupload/css/jquery.ui.plupload.css" media="screen" />

    <script type="text/javascript" src="assets/uploads/i18n/pt_BR.js"></script>

    <script type="text/javascript">
    $(function() {
        $("#uploader").plupload({
            // General settings
            runtimes : 'html5,flash,silverlight,html4',
            url : "controller/xml_upload.php",

            // Maximum file size
            max_file_size : '2048mb',

            chunk_size: '1mb',

            // Specify what files to browse for
            filters : [
                {title : "XML", extensions : "xml"}
            ],

            // Rename files by clicking on their titles
            rename: true,

            // Sort files
            sortable: true,

            // Enable ability to drag'n'drop files onto the widget (currently only HTML5 supports that)
            dragdrop: false,

            // Views to activate
            views: {
                list: false,
                thumbs: false, // Show thumbs
                active: 'thumbs'
            },

            // Flash settings
            flash_swf_url : '/videos_add1_up/Moxie.swf',

            // Silverlight settings
            silverlight_xap_url : '/videos_add1_up/Moxie.xap',

            multipart_params : {
                "xml_usuario" : "xml_user"
            }
        });
    });
    </script>

PHP

    header("Expires: Mon, 26 Jul 1997 05:00:00 GMT");
    header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
    header("Cache-Control: no-store, no-cache, must-revalidate");
    header("Cache-Control: post-check=0, pre-check=0", false);
    header("Pragma: no-cache");
    @set_time_limit(5 * 60);
    $targetDir = '../../assets/arquivos/xml/';
    $cleanupTargetDir = true; 
    $maxFileAge = 5 * 3600; 

    if (!file_exists($targetDir)) {
        @mkdir($targetDir);
    }

    if (isset($_REQUEST["name"])) {
        $fileName = $_REQUEST["name"];
    } elseif (!empty($_FILES)) {
        $fileName = $_FILES["file"]["name"];
    } else {
        $fileName = uniqid("file_");
    }

    include '../../_config.php';

    $filePath = $targetDir . DIRECTORY_SEPARATOR . $fileName;

    // Chunking might be enabled
    $chunk = isset($_REQUEST["chunk"]) ? intval($_REQUEST["chunk"]) : 0;
    $chunks = isset($_REQUEST["chunks"]) ? intval($_REQUEST["chunks"]) : 0;

    if ($cleanupTargetDir) {
        if (!is_dir($targetDir) || !$dir = opendir($targetDir)) {
            die('{"jsonrpc" : "2.0", "error" : {"code": 100, "message": "Failed to open temp directory."}, "id" : "id"}');
        }

        while (($file = readdir($dir)) !== false) {
            $tmpfilePath = $targetDir . DIRECTORY_SEPARATOR . $file;

            // If temp file is current file proceed to the next
            if ($tmpfilePath == "{$filePath}.part") {
                continue;
            }

            // Remove temp file if it is older than the max age and is not the current file
            if (preg_match('/\.part$/', $file) && (filemtime($tmpfilePath) < time() - $maxFileAge)) {
                @unlink($tmpfilePath);
            }
        }
        closedir($dir);
    }   

    if (!$out = @fopen("{$filePath}.part", $chunks ? "ab" : "wb")) {
        die('{"jsonrpc" : "2.0", "error" : {"code": 102, "message": "Failed to open output stream."}, "id" : "id"}');
    }

    if (!empty($_FILES)) {
        if ($_FILES["file"]["error"] || !is_uploaded_file($_FILES["file"]["tmp_name"])) {
            die('{"jsonrpc" : "2.0", "error" : {"code": 103, "message": "Failed to move uploaded file."}, "id" : "id"}');
        }

        // Read binary input stream and append it to temp file
        if (!$in = @fopen($_FILES["file"]["tmp_name"], "rb")) {
            die('{"jsonrpc" : "2.0", "error" : {"code": 101, "message": "Failed to open input stream."}, "id" : "id"}');
        }
    } else {    
        if (!$in = @fopen("php://input", "rb")) {
            die('{"jsonrpc" : "2.0", "error" : {"code": 101, "message": "Failed to open input stream."}, "id" : "id"}');
        }
    }

    while ($buff = fread($in, 4096)) {
        fwrite($out, $buff);
    }

    @fclose($out);
    @fclose($in);

    // Check if file has been uploaded
    if (!$chunks || $chunk == $chunks - 1) {
        // Strip the temp .part suffix off 
        rename("{$filePath}.part", $filePath);
    }

php uploading large files

instead uploading with standard form try uploading with xhr object (as you said) but using file chunk method to send file to server, in this way theorically you should have not upload limits. Try this upload jquery plugin that provides also php scripts:

http://code.google.com/p/ax-jquery-multiuploader/ REMOVE BECAUSE CANNOT MANTAIN

New link (free):
http://www.albanx.com/download.php?item_id=4

Documentation:
http://www.albanx.com/ajaxuploader/doc.php

No error when uploading a really big file in PHP?

Check your PHP ini file, which governs how large a file PHP will allow to be uploaded. These variables are important:

• max upload filesize (upload_max_filesize)
• max post data size (post_max_size)
• memory limit (memory_limit)

Any uploads outside these bounds will be ignored or errored-out, depending on your settings.

This section in the docs has the best summary: http://ca3.php.net/manual/en/features.file-upload.common-pitfalls.php

EDIT: Also note that most browsers won't send uploads over 2GB in size. This link is outdated, but gives an idea: http://www.motobit.com/help/scptutl/pa98.htm. Anyone have a better source of info on this?

There are also limits that can be imposed by the server, such as Apache: http://httpd.apache.org/docs/2.2/mod/core.html#limitrequestbody

To truly see what's going on, you should probably check your webserver logs, check the browser upload limit (if you're using firefox), and try seeing if print_r($_FILES) generates any useful error numbers. If all else fails, try the net traffic monitor in firebug. The key is to determine if the request is even going to the server, and if it is what the request (including headers) looks like. Once you've gotten that far in the chain, then you can go back and see how PHP is handling the upload.

---

Related Topics