Compare Two Images the Python/Linux Way

Compare two images the python/linux way

There is a OSS project that uses WebDriver to take screen shots and then compares the images to see if there are any issues (http://code.google.com/p/fighting-layout-bugs/)). It does it by openning the file into a stream and then comparing every bit.

You may be able to do something similar with PIL.

EDIT:

After more research I found

h1 = Image.open("image1").histogram()
h2 = Image.open("image2").histogram()

rms = math.sqrt(reduce(operator.add,
    map(lambda a,b: (a-b)**2, h1, h2))/len(h1))

on http://snipplr.com/view/757/compare-two-pil-images-in-python/ and http://effbot.org/zone/pil-comparing-images.htm

Comparing two images/pictures, and mark the difference

Regarding your first question:

import ImageDraw
draw = ImageDraw.Draw(im2)
draw.rectangle(diff)
im2.show()

Regarding your second question:

The error states, that sys.argv does not contain enough values to be assigned to file1 and file2. You need to pass the names of the two files you want to compare to you python script (the variable sys.arv contains the name of your script and all the command line parameters):

python name_of_your_script.py file1.jpg file2.jpg

How can I quantify difference between two images?

General idea

Option 1: Load both images as arrays (scipy.misc.imread) and calculate an element-wise (pixel-by-pixel) difference. Calculate the norm of the difference.

Option 2: Load both images. Calculate some feature vector for each of them (like a histogram). Calculate distance between feature vectors rather than images.

However, there are some decisions to make first.

Questions

You should answer these questions first:

Are images of the same shape and dimension?
If not, you may need to resize or crop them. PIL library will help to do it in Python.
If they are taken with the same settings and the same device, they are probably the same.
Are images well-aligned?
If not, you may want to run cross-correlation first, to find the best alignment first. SciPy has functions to do it.
If the camera and the scene are still, the images are likely to be well-aligned.
Is exposure of the images always the same? (Is lightness/contrast the same?)
If not, you may want to normalize images.
But be careful, in some situations this may do more wrong than good. For example, a single bright pixel on a dark background will make the normalized image very different.
Is color information important?
If you want to notice color changes, you will have a vector of color values per point, rather than a scalar value as in gray-scale image. You need more attention when writing such code.
Are there distinct edges in the image? Are they likely to move?
If yes, you can apply edge detection algorithm first (e.g. calculate gradient with Sobel or Prewitt transform, apply some threshold), then compare edges on the first image to edges on the second.
Is there noise in the image?
All sensors pollute the image with some amount of noise. Low-cost sensors have more noise. You may wish to apply some noise reduction before you compare images. Blur is the most simple (but not the best) approach here.
What kind of changes do you want to notice?
This may affect the choice of norm to use for the difference between images.
Consider using Manhattan norm (the sum of the absolute values) or zero norm (the number of elements not equal to zero) to measure how much the image has changed. The former will tell you how much the image is off, the latter will tell only how many pixels differ.

Example

I assume your images are well-aligned, the same size and shape, possibly with different exposure. For simplicity, I convert them to grayscale even if they are color (RGB) images.

You will need these imports:

import sys

from scipy.misc import imread
from scipy.linalg import norm
from scipy import sum, average

Main function, read two images, convert to grayscale, compare and print results:

def main():
    file1, file2 = sys.argv[1:1+2]
    # read images as 2D arrays (convert to grayscale for simplicity)
    img1 = to_grayscale(imread(file1).astype(float))
    img2 = to_grayscale(imread(file2).astype(float))
    # compare
    n_m, n_0 = compare_images(img1, img2)
    print "Manhattan norm:", n_m, "/ per pixel:", n_m/img1.size
    print "Zero norm:", n_0, "/ per pixel:", n_0*1.0/img1.size

How to compare. img1 and img2 are 2D SciPy arrays here:

def compare_images(img1, img2):
    # normalize to compensate for exposure difference, this may be unnecessary
    # consider disabling it
    img1 = normalize(img1)
    img2 = normalize(img2)
    # calculate the difference and its norms
    diff = img1 - img2  # elementwise for scipy arrays
    m_norm = sum(abs(diff))  # Manhattan norm
    z_norm = norm(diff.ravel(), 0)  # Zero norm
    return (m_norm, z_norm)

If the file is a color image, imread returns a 3D array, average RGB channels (the last array axis) to obtain intensity. No need to do it for grayscale images (e.g. .pgm):

def to_grayscale(arr):
    "If arr is a color image (3D array), convert it to grayscale (2D array)."
    if len(arr.shape) == 3:
        return average(arr, -1)  # average over the last axis (color channels)
    else:
        return arr

Normalization is trivial, you may choose to normalize to [0,1] instead of [0,255]. arr is a SciPy array here, so all operations are element-wise:

def normalize(arr):
    rng = arr.max()-arr.min()
    amin = arr.min()
    return (arr-amin)*255/rng

Run the main function:

if __name__ == "__main__":
    main()

Now you can put this all in a script and run against two images. If we compare image to itself, there is no difference:

$ python compare.py one.jpg one.jpg
Manhattan norm: 0.0 / per pixel: 0.0
Zero norm: 0 / per pixel: 0.0

If we blur the image and compare to the original, there is some difference:

$ python compare.py one.jpg one-blurred.jpg 
Manhattan norm: 92605183.67 / per pixel: 13.4210411116
Zero norm: 6900000 / per pixel: 1.0

P.S. Entire compare.py script.

Update: relevant techniques

As the question is about a video sequence, where frames are likely to be almost the same, and you look for something unusual, I'd like to mention some alternative approaches which may be relevant:

background subtraction and segmentation (to detect foreground objects)
sparse optical flow (to detect motion)
comparing histograms or some other statistics instead of images

I strongly recommend taking a look at “Learning OpenCV” book, Chapters 9 (Image parts and segmentation) and 10 (Tracking and motion). The former teaches to use Background subtraction method, the latter gives some info on optical flow methods. All methods are implemented in OpenCV library. If you use Python, I suggest to use OpenCV ≥ 2.3, and its cv2 Python module.

The most simple version of the background subtraction:

learn the average value μ and standard deviation σ for every pixel of the background
compare current pixel values to the range of (μ-2σ,μ+2σ) or (μ-σ,μ+σ)

More advanced versions make take into account time series for every pixel and handle non-static scenes (like moving trees or grass).

The idea of optical flow is to take two or more frames, and assign velocity vector to every pixel (dense optical flow) or to some of them (sparse optical flow). To estimate sparse optical flow, you may use Lucas-Kanade method (it is also implemented in OpenCV). Obviously, if there is a lot of flow (high average over max values of the velocity field), then something is moving in the frame, and subsequent images are more different.

Comparing histograms may help to detect sudden changes between consecutive frames. This approach was used in Courbon et al, 2010:

Similarity of consecutive frames. The distance between two consecutive frames is measured. If it is too high, it means that the second frame is corrupted and thus the image is eliminated. The Kullback–Leibler distance, or mutual entropy, on the histograms of the two frames:

$d(p,q) = \sum_i p(i) \log (p(i)/q(i))$

where p and q are the histograms of the frames is used. The threshold is fixed on 0.2.

Image comparison algorithm

A similar question was asked a year ago and has numerous responses, including one regarding pixelizing the images, which I was going to suggest as at least a pre-qualification step (as it would exclude very non-similar images quite quickly).

There are also links there to still-earlier questions which have even more references and good answers.

Here's an implementation using some of the ideas with Scipy, using your above three images (saved as im1.jpg, im2.jpg, im3.jpg, respectively). The final output shows im1 compared with itself, as a baseline, and then each image compared with the others.

>>> import scipy as sp
>>> from scipy.misc import imread
>>> from scipy.signal.signaltools import correlate2d as c2d
>>>
>>> def get(i):
...     # get JPG image as Scipy array, RGB (3 layer)
...     data = imread('im%s.jpg' % i)
...     # convert to grey-scale using W3C luminance calc
...     data = sp.inner(data, [299, 587, 114]) / 1000.0
...     # normalize per http://en.wikipedia.org/wiki/Cross-correlation
...     return (data - data.mean()) / data.std()
...
>>> im1 = get(1)
>>> im2 = get(2)
>>> im3 = get(3)
>>> im1.shape
(105, 401)
>>> im2.shape
(109, 373)
>>> im3.shape
(121, 457)
>>> c11 = c2d(im1, im1, mode='same')  # baseline
>>> c12 = c2d(im1, im2, mode='same')
>>> c13 = c2d(im1, im3, mode='same')
>>> c23 = c2d(im2, im3, mode='same')
>>> c11.max(), c12.max(), c13.max(), c23.max()
(42105.00000000259, 39898.103896795357, 16482.883608327804, 15873.465425120798)

So note that im1 compared with itself gives a score of 42105, im2 compared with im1 is not far off that, but im3 compared with either of the others gives well under half that value. You'd have to experiment with other images to see how well this might perform and how you might improve it.

Run time is long... several minutes on my machine. I would try some pre-filtering to avoid wasting time comparing very dissimilar images, maybe with the "compare jpg file size" trick mentioned in responses to the other question, or with pixelization. The fact that you have images of different sizes complicates things, but you didn't give enough information about the extent of butchering one might expect, so it's hard to give a specific answer that takes that into account.

Use imagequick to compare images

Here are some examples to help you understand how image comparison works.

The -metric AE tells you the Absolute Error, which is the number of pixels that differ - so it will be zero if all pixels are identical.

1. Compare two images that are exact copies of each other in every respect

convert -size 128x128 xc:red red.png                    # Make red image
cp red.png perfectCopy.png                              # Make perfect copy
compare -metric AE red.png perfectCopy.png result.png   # Count differing pixels
0                                                       # There are none - identical

According to standard Unix tools (md5, diff, tmp), files are binary identical and md5 checksums identical:

md5 red.png perfectCopy.png 
MD5 (red.png) = 39236e0e0dfb70da0e9bcbfbcf7b8181
MD5 (perfectCopy.png) = 39236e0e0dfb70da0e9bcbfbcf7b8181

ImageMagick hashes over pixels only (not including metadata) are identical:

identify -format "%#:%f\n" red.gif perfectCopy.png 
1157038985fec4573888bc7e74a5a728e3aa5cbc49e18253c934295162a9aeea:red.gif
1157038985fec4573888bc7e74a5a728e3aa5cbc49e18253c934295162a9aeea:perfectCopy.png

2. Compare two images with identical appearance but different metadata

convert -size 128x128 xc:red red.png                    # Make red image
sleep 2
convert -size 128x128 xc:red redDifferentDate.png       # Make red image with different date
compare -metric AE red.png redDifferentDate.png result.png
0                                                       # No difference

But, according to standard Unix tools (diff,md5,sum), the files are different - because the date is in there.

md5 red.png redDifferentDate.png 
MD5 (red.png) = 004088f6d275f431cedb74bc0209bbc5
MD5 (redDifferentDate.png) = d7d36f56e1940251f9804bd795ef4157

But ImageMagick knows images better, and its calculated hashes (checksum) over pixel data only (not including metadata) are the same:

identify -format "%#:%f\n" red.gif redDifferentDate.png 
1157038985fec4573888bc7e74a5a728e3aa5cbc49e18253c934295162a9aeea:red.gif
1157038985fec4573888bc7e74a5a728e3aa5cbc49e18253c934295162a9aeea:redDifferentDate.png

3. Compare two images with identical pixels, but totally different sizes and formats

convert -size 128x128 xc:red red.png                    # Make red PNG
convert -size 128x128 xc:red red.gif                    # Make red  GIF
compare -metric AE red.png red.gif result.png           # Count differing pixels
0                                                       # No difference

But, files and md5 hashes differ:

diff red.png red.gif
Binary files red.png and red.gif differ

md5 red.png red.gif 
MD5 (red.png) = aed0840c2c99425c25bd782e7b409022
MD5 (red.gif) = 5869df00d7b3cab3495a6c402ba61ec9

Again, ImageMagick knows better and the hashes over pixel data only (not including metadata) are still the same:

identify -format "%#:%f\n" red.gif red.png 
1157038985fec4573888bc7e74a5a728e3aa5cbc49e18253c934295162a9aeea:red.gif
1157038985fec4573888bc7e74a5a728e3aa5cbc49e18253c934295162a9aeea:red.png

4. Compare two grossly different files

Obviously if we create two grossly different files each full of random noise, everyone agrees they are different:

convert -size 128x128 xc:gray +noise random random1.png   # Make random image
convert -size 128x128 xc:gray +noise random random2.png   # Make random image
compare -metric AE random[12].png result.png              # Count differing pixels
16384                                                     # Yep, a fair few differ!

There are other metrics available, such as MeanSquared, RootMeanSquared etc - you can list them using:

identify -list metric

Output

AE
Fuzz
MAE
MEPP
MSE
NCC
PAE
PHASH
PSNR
RMSE

Compare Two Images the Python/Linux Way