How to Increase Memory Limit for PHP Over 2Gb

How to increase memory limit for PHP over 2GB?

Have you tried using the value in MB ?

php_value memory_limit 2048M

Also try editing this value in php.ini not Apache.

PHP out of memory though I have MUCH MUCH more allocated

It turns out that this site has Drupal's path_memory module enabled, and the path I was trying to hit was configured to have a maximum of 256M available at admin/config/system/path-memory. Tricky.

Increasing PHP memory_limit. At what point does it become insane?

The configuration for the memory_limit of PHP running as an Apache module to server webpages has to take into consideration how many Apache process you can have at the same time on the machine -- see the MaxClients configuration option for Apache.

If MaxClients is 100 and you have 2,000 MB of RAM, a very quick calculation will show that you should not use more than 20 MB *(because 20 MB * 100 clients = 2 GB or RAM, ie the total amount of memory your server has)* for the memory_limit value.

And this is without considering that there are probably other things running on the same server, like MySQL, the system itself, ... And that Apache is probably already using some memory for itself.

Or course, this is also a "worst case scenario", that considers that each PHP page is using the maximum amount of memory it can.

In your case, if you need such a big amount of memory for only one job, I would not increase the memory_limit for PḦP running as an Apache module.

Instead, I would launch that job from command-line (or via a cron job), and specify a higher memory_limit specificaly in this one and only case.

This can be done with the -d option of php, like :

$ php -d memory_limit=1GB temp.php
string(3) "1GB"

Considering, in this case, that temp.php only contains :

var_dump(ini_get('memory_limit'));

In my opinion, this is way safer than increasing the memory_limit for the PHP module for Apache -- and it's what I usually do when I have a large dataset, or some really heavy stuff I cannot optimize or paginate.

If you need to define several values for the PHP CLI execution, you can also tell it to use another configuration file, instead of the default php.ini, with the -c option :

php -c /etc/phpcli.ini temp.php

That way, you have :

• /etc/php.ini for Apache, with low memory_limit, low max_execution_time, ...
• and /etc/phpcli.ini for batches run from command-line, with virtually no limit

This ensures your batches will be able to run -- and you'll still have security for your website (memory_limit and max_execution_time being security measures)

Still, if you have the time to optimize your script, you should ; for instance, in that kind of situation where you have to deal with lots of data, pagination is a must-have ;-)

PHP x86 Memory Limit

Theoretically, you should be able to allocate up to 2GB of memory with an x86 process on Windows. However, there are other constraints on how much you can allocate. Critically: memory fragmentation in your available address space. Since base64_encode allocates its needed memory all at once, you'll need enough contiguous address space to encode the entire string. You're more likely to have enough with a 64-bit address space, which is why the 64-bit version of PHP runs just fine.

For a little more information in this answer: Java maximum memory on Windows XP

Addendum: In my testing, I was able to allocate only slightly more than yours, but smaller address allocations let me get closer to that 2GB limit.

Addendum 2: If you absolutely need to use the 32-bit version and are encoding to a file, you can work around this somewhat by encoding in some multiple of 3 bytes as then you can concatenate them.

// Example, from memory string to file.
$current_position = 0;
$length = strlen($file);
$outputfile = fopen('testoutput.txt', 'w');
while($current_position < $length) {
  fwrite($outputfile, base64_encode(substr($file, $current_position, 60000)));
  $current_position += 60000;
}
fclose($outputfile);

Or going all the way, from file to file, which allocates very little memory:

$inputfile = fopen('test.mp4', 'rb');
$outputfile = fopen('testoutput2.txt', 'w');
while( ($str = fread($inputfile, 61440)) !== false ) {
  if($str === '') {
    break;
  }
  fwrite($outputfile, base64_encode($str));
  $current_position += 61440;
}
fclose($outputfile);

Composer update memory limit

When you run composer update, the OS will look into the configured paths and try to locate an executable file with that name.

When running php composer update, the composer string is treated as a parameter to PHP, which is not searched in any paths. You have to provide the full path in order to run it.

Running which composer will tell you where the OS finds the composer executable, and then you simply use the full path in the PHP command:

$>which composer
/usr/local/bin/composer

$>php -d memory_limit=512M /usr/local/bin/composer update
...

Note that 512MB might be too few. My perception is that it will happily take 1GB or more, depending on the number of dependencies you use and the variety of versions that you theoretically allow, i.e. if you allow Symfony ~2.3, then you make Composer deal with a lot more possible versions compared to using ~2.7.

Also note that running Composer on the production machine is not the best idea. You would have to have access to Github, maybe provide access credentials, have VCS tools installed, and you will easily break your site if any of the remote hosting servers is offline during your update. It is a better idea to use Composer on a deployment system that does all the preparation, and then moves all the files onto the production server.

Update

It's the year 2020 now, and the way Composer manages its memory has changed quite a bit. The most important thing is that Composer will increase the memory limit by itself if it encounters a limit set too low. This however immediately triggers the problem of running out of memory on machines that have too few memory installed. You can make Composer use less memory by setting the environment variable like COMPOSER_MEMORY_LIMIT=512M, but this will create problems if Composer would need more memory to correctly operate.

My main point remains true: Do not run Composer on machines that have too few memory installed. You potentially need 1.5 GB of free memory to be able to update everything.

---

Related Topics