Cache Resources Exhausted Imagemagick

Error at creating GIF Image with ImageMagick: too many exceptions and blinking problem

1. 100 frames/second is overkill. You can run with 10 frames/sec and divide the image count by 10 (or at least the standard 25 frames/sec and divide by 4).
2. *.png is expanded and sorted alphabetically by your shell,so if you want frames in their number sequence, pad the names with 0's:

for n in {1..3372} ; ; do mv image$n.png image$(printf "%04d" $n).png ; done

3. convert (and other IM commands) uses a memory cache and has several self-enforced limits (that you can list with IM's identify command):

>>> identify -list resource
Resource limits:
  Width: 16KP
  Height: 16KP
  List length: 18.446744EP
  Area: 128MP
  Memory: 256MiB
  Map: 512MiB
  Disk: 1GiB
  File: 768
  Thread: 8
  Throttle: 0
  Time: unlimited

• You can boost the limit for one run:

convert -limit memory 1000  ...

• For more permanent changes you can also edit /etc/ImageMagick-6/policy.xml and look for lines such as <policy domain="resource" name="memory" value="256MiB"/> and increase the necessary ones.

Why does Appengine image rotation give cache resources error?

i.rotate(125)
gives the error, but
i.rotate(90) works, as does 270. I only want 90-degree rotations so that will do.

ImageMagick convert OOMing inside Singularity container

The problem turned out to be with ImageMagick's system-wide policy.xml that was installed in my container. Updating that file with more generous "memory" and "disk" values fixed this problem.

You can find the location of your system's policy.xml file using the command convert -list policy (Hat tip to Kurt Pfeifle's answer which clued me in on this). For me it was at /etc/ImageMagick-6/policy.xml. You can edit that file
(if you have root access). In the end I decided just to delete the file since I don't want my system from restricting my use at all inside the container.

You can set limits via the command line, say convert -limit memory 2GiB ... or environment variables (See Kurt Pfeifle's answer for details). However, this method does not allow expanding larger than system-wide limits set in policy.xml because this policy file is meant to be a way for system administrators to forcibly limit users. Therefore, the only way to remedy this is to update/remove the system-wide policy.

How to prevent image bombs with ImageMagick?

Setting the 'Resource Area' limit only sets the size at which images are not held in memory, and instead are paged to disk. If you want to use that setting to actually restrict the maximum size image that can be openend, you also need to set the 'Resource Disk' limit.

The code below correctly gives a memory allocation error for the image bombs taken from here.

try {
    Imagick::setResourceLimit(Imagick::RESOURCETYPE_AREA, 2000 * 2000);
    Imagick::setResourceLimit(Imagick::RESOURCETYPE_DISK, 2000 * 2000);

    $imagick = new Imagick("./picture-100M-6000x6000.png");
    $imagick->modulateImage(100, 50, 120);
    $imagick->writeImage("./output.png");

    echo "Complete";
}
catch(\Exception $e) {
    echo "Exception: ".$e->getMessage()."\n";
}

Output is:

Exception: Memory allocation failed `./picture-100M-6000x6000.png' @ error/png.c/MagickPNGErrorHandler/1630

If you want to set the width and height resource, and have a version of ImageMagick >= 6.9.0-1 you should be able to using the values directly of WidthResource = 9, HeightResource = 10

//Set max image width of 2000
Imagick::setResourceLimit(9, 2000);
//Set max image height of 1000
Imagick::setResourceLimit(10, 1000);

These don't have to be set programmatically, you can set them through the policy.xml file installed with ImageMagick. ImageMagick reads that file and uses those settings if none are set in a program - which may be a more convenient way of setting them, as you can change them per machine.

This makes it impossible for me to handle it properly.

It makes it impossible for you to handle it in the same process. You can handle it just fine by running the image processing in a background task.

Personally I think anyway that uses Imagick in a server that is accessed directly by web-browsers is nuts. It is far safer to run it in as a background task (managed by something like http://supervisord.org/) and communicating with that background task via a queue of jobs that need to be processed.

Not only does that solve the 'bad images can bring down my website' problem, it also makes it far easier to monitor resource usage, or shift the image processing to a machine with a faster CPU than a web-front end server needs.

Source - I'm the maintainer of the Imagick extension and I recently added this to the Imagick readme:

Security

The PHP extension Imagick works by calling the ImageMagick library.
Although the ImageMagick developers take good care in avoiding bugs it
is inevitable that some bugs will be present in the code. ImageMagick
also uses a lot of third party libraries to open, read and manipulate
files. The writers of these libraries also take care when writing
their code. However everyone makes mistakes and there will inevitably
be some bugs present.

Because ImageMagick is used to process images it is feasibly possible
for hackers to create images that contain invalid data to attempt to
exploit these bugs. Because of this we recommend the following:

1) Do not run Imagick in a server that is directly accessible from
outside your network. It is better to either use it as a background
task using something like SupervisorD or to run it in a separate
server that is not directly access on the internet.

Doing this will make it difficult for hackers to exploit a bug, even
if one should exist in the libraries that ImageMagick is using.

2) Run it as a very low privileged process. As much as possible the
files and system resources accessible to the PHP script that Imagick
is being called from should be locked down.

3) Check the result of the image processing is a valid image file
before displaying it to the user. In the extremely unlikely event that
a hacker is able to pipe arbitrary files to the output of Imagick,
checking that it is an image file, and not the source code of your
application that is being sent, is a sensible precaution.

---

Related Topics