How to Set a Maximum Size Limit to PHP Curl Downloads

How to set a maximum size limit to PHP cURL downloads?

I have another answer that addresses the situation even better to leave here for posterity.

CURLOPT_WRITEFUNCTION is good for this but CURLOPT_PROGRESSFUNCTION is the best.

// We need progress updates to break the connection mid-way
curl_setopt($cURL_Handle, CURLOPT_BUFFERSIZE, 128); // more progress info
curl_setopt($cURL_Handle, CURLOPT_NOPROGRESS, false);
curl_setopt($cURL_Handle, CURLOPT_PROGRESSFUNCTION, function(
    $DownloadSize, $Downloaded, $UploadSize, $Uploaded
){
    // If $Downloaded exceeds 1KB, returning non-0 breaks the connection!
    return ($Downloaded > (1 * 1024)) ? 1 : 0;
});

Keep in mind that even if the PHP.net states^ for CURLOPT_PROGRESSFUNCTION:

A callback accepting five parameters.

My local tests have featured only four (4) parameters as the 1st (handle) is not present.

PHP curl_multi - how to limit the download size of file

This works:

<?php

$url = 'https://example.com/file';

$ch = curl_init($url);

$bytes = 0;

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_NOHEADER, 1);
curl_setopt($ch, CURLOPT_WRITEFUNCTION, function($ch, $data) use(&$bytes) {
    static $size = 0;

    //echo $data;

    $size += strlen($data);

    $bytes = $size;

    if ($size > 1024) {
        return -1;
    }
    return strlen($data);
});

$res = curl_exec($ch);

echo "Got $bytes bytes\n";

The concept is to use CURLOPT_WRITEFUNCTION to receive the data from the response body and increment a static counter local to the function. Once the number of bytes exceeds 1024, return -1 to abort the transfer. A value is shared between the callback and the program so you can check the value of $bytes after the transfer to see if it was greater than your target size or not.

Can you set a max file size to download using PHP's built in Curl functions?

There is no way to do this with PHP's built in curl functions, without making a separate request to the webserver hosting the file.

php cURL progress function size limit

CURLOPT_PROGRESSFUNCTION function argument type are double, so you are probably casting these value to int somewhere in your code. and probably since your os is 32bit based architecture you are overflowing the int size limit. I would suggest you revise your code to avoid casting to int big double numbers.

PHP cUrl - Unable to fetch large files

what do you think this line does? curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); - it tells php to catch all stdout output of curl_exec, and save it all in memory at once, before doing anything else, that's both a very slow approach (because you don't start writing to disk before your download is 100% complete, and unless you're running on SSDs, disks are slow), and extremely memory hungry approach (because you store the entire file in memory at once), neither of those things are desirable. instead, do $fp=fopen(basename($url),'wb');curl_setopt($ch,CURLOPT_FILE,$fp); - now curl will write the content directly to the disk, thus being much faster (writing it to disk as it's being downloaded) AND just use a small amount of ram, no matter how big the download file is.

• also note, if you're going to run large amount of slow downloads simultaneously, PHP-behind-a-webserver is simply a bad tool for the job, usually the amount of concurrent php processes you can run is very limited, and block your entire website from loading when all of them are busy, and php aborts if the client disconnect for some reason (see ignore_user_abort()), and many webservers will timeout if the script takes too long (see nginx proxy_read_timeout for example), and php often even kill itself for timeout reasons (see set_time_limit()) .. if that's the case, consider writing the downloader in another language (for example, Go's goroutines should be able to do a massive amount of concurrent slow downloads with little resource usage, unlike PHP)

Is there any size limit to post a file using curl?

Upload limits are a security feature. Without them, a rogue program or attacker could feed your server with a continuous stream of data until your hard disk is full, thus rendering the whole server unusable.

From the security standpoint it isn't particularly useful to restrict outgoing data and, as far as I know, neither the Curl library nor PHP itself impose any limit.

Your symptoms suggest the problem is on the destination server. Since you appear to have access to it (you mention getting empty $_POST) I suggest you verify upload limits there. That's something you can do (and often change) yourself, you don't have to ask the server administrator. Main involved directives include:

• post_max_size
• upload_max_filesize
• max_file_uploads
• max_input_time

You can inspect them with phpinfo() or ini_get() and you can change them the usual way.

How many maximum urls can I download at one time using curl

No limits but you must consider the connection of internet on your server, bandwidth, memory leaks, CPU and etc

---

Related Topics