How to Search an Image for Subimages Using Linux Console

How to search an image for subimages using linux console?

Here's a little example so you can see how it works...

First, our needle image

Now make a haystack, green and blue - very stylish :-)

convert -size 256x256 gradient:lime-blue haystack.png

Now hide two needles in the haystack, one at a time, nothing fancy:

convert haystack.png needle.png -geometry +30+5 -composite haystack.png 
convert haystack.png needle.png -geometry +100+150 -composite haystack.png 

Now search for the needles in the haystack, two output files will be produced, locations-0.png and locations-1.png

compare -metric RMSE -subimage-search haystack.png needle.png locations.png > /dev/null 2>&1

This is the second, more useful output file locations-1.png. It is black where IM is sure there is no match and progressively nearer to white the more certain ImageMagick is that there is a match.

Now look for locations where IM is 95+% certain there is a match and convert all pixels to text so we can search for the word white.

convert locations-1.png -threshold 95% txt: | grep white

The output is this, meaning ImageMagick has found the needles at 30,5 and 100,150 - exactly where we hid them! Told you it was Magic!

30,5: (255,255,255)  #FFFFFF  white
100,150: (255,255,255)  #FFFFFF  white

Here is the entire script so you can run it and play with it:

#!/bin/bash
convert -size 256x256 gradient:lime-blue haystack.png                      # make our haystack
convert haystack.png needle.png -geometry +30+5 -composite haystack.png    # hide our needle near top-left
convert haystack.png needle.png -geometry +100+150 -composite haystack.png # hide a second needle lower down

# Now search for the needles in the haystack...
# ... two output files will be produced, "locations-0.png" and "locations-1.png"
compare -metric RMSE -subimage-search haystack.png needle.png locations.png > /dev/null 2>&1

# Now look for locations where IM is 95% certain there is a match
convert locations-1.png -threshold 95% txt: | grep white

find and copy all images in directory using terminal linux mint, trying to understand syntax

First, understand that the pipe "|" links commands piping the output of the first into the second as an argument. Your two shell codes both pipe output of the find command into other commands (grep and xargs). Let's look at those commands one after another:

First command: find

find is a program to "search for files in a directory hierarchy" (that is the explanation from find's man page). The syntax is (in this case)

find <search directory> <search pattern> <action>

In both cases the search directory is . (that is the current directory). Note that it does not just search the current directory but all its subdirectories as well (the directory hierarchy).

The search pattern accepts options -name (meaning it searches for files the name of which matches the pattern given as an argument to this option) or -iname (same as name but case insensitive) among others.

The action pattern may be -print0 (print the exact filename including its position in the given search directory, i.e. the relative or absolute path to the file) or -exec (execute the given command on the file(s), the command is to be ended with ";" and every instance of "{}" is replaced by the filename).

That is, the first shell code (first part, left of the pipe)

find . -iname \*.jpg -print0 

searches all files with ending ".jpg" in the current directory hierarchy and prints their paths and names. The second one (first part)

find . -name '*' -exec file {} \; 

finds all files in the current directory hierarchy and executes

file <filename>

on them. File is another command that determines and prints the file type (have a look at the man page for details, man file).

Second command: xargs

xargs is a command that "builds and exectues command lines from standard input" (man xargs), i.e. from the find output that is piped into xargs. The command that it builds and executes is in this case

cp -v {} /home/joachim/neu2"

Option -I{} defines the replacement string, i.e. every instance of {} in the command is to be replaced by the input it gets from file (that is, the filenames). Option -0 defines that input items are not terminated (seperated) by whitespace or newlines but only by a null character. This seems to be necessary when using and the standard way to deal with find output as xargs input.

The command that is built and executed is then of course the copy command with option -v (verbose) and it copies each of the filenames it gets from find to the directory.

Third command: grep

grep filters its input giving only those lines or strings that match a particular output pattern. Option -o tells grep to print only the matching string, not the entire line (see man grep), -P tells it to interpret the following pattern as a perl regexp pattern. In perl regex, ^ is the start of the line, .+ is any arbitrary string, this arbitrary should then be followed by a colon, a space, a number of alphanumeric characters (in perl regex denoted \w+) a space and the string "image". Essentially this grep command filters the file output to only output the filenames that are image files. (Read about perl regex's for instance here: http://www.comp.leeds.ac.uk/Perl/matching.html )

The command you actually wanted

Now what you want to do is (1) take the output of the second shell command (which lists the image files), (2) bring it into the appropriate form and (3) pipe it into the xargs command from the first shell command line (which then builds and executes the copy command you wanted). So this time we have a three (actually four) stage shell command with two pipes. Not a problem. We already have stages (1) and (3) (though in stage (3) we need to leave out the -0 option because the input is not find output any more; we need it to treat newlines as item seperators).

Stage (2) is still missing. I suggest using the cut command for this. cut changes strings py splitting them into different fields (seperated by a delimiter character in the original string) that can then be rearranged. I will choose ":" as the delimiter character (this ends the filename in the grep output, option -d':') and tell it to give us just the first field (option -f1, essentialls: print only the filename, not the part that comes after the ":"), i.e. stage (2) would then be

cut -d':' -f1

And the entire command you wanted will then be:

find . -name '*' -exec file {} \; | grep -o -P '^.+: \w+ image' | cut -d':' -f1 | xargs -I{} cp -v {} /home/joachim/neu2

Note that you can find all the man pages for instance here: http://www.linuxmanpages.com

How to find subimage using the PIL library?

import cv2  
import numpy as np  
image = cv2.imread("Large.png")  
template = cv2.imread("small.png")  
result = cv2.matchTemplate(image,template,cv2.TM_CCOEFF_NORMED)  
print np.unravel_index(result.argmax(),result.shape)

This works fine and in efficient way for me.

How to get the list of dependent child images in Docker?

Short answer:
Here is a python3 script that lists dependent docker images.

Long answer:
You can see the image id and parent id for all image created after the image in question with the following:

docker inspect --format='{{.Id}} {{.Parent}}' \
    $(docker images --filter since=f50f9524513f --quiet)

You should be able to look for images with parent id starting with f50f9524513f, then look for child images of those, etc.. But .Parent isn’t what you think., so in most cases you would need to specify docker images --all above to make that work, then you will get image ids for all intermediate layers as well.

Here's a more limited python3 script to parse the docker output and do the searching to generate the list of images:

#!/usr/bin/python3
import sys

def desc(image_ids, links):
    if links:
        link, *tail = links
        if len(link) > 1:
            image_id, parent_id = link
            checkid = lambda i: parent_id.startswith(i)
            if any(map(checkid, image_ids)):
                return desc(image_ids | {image_id}, tail)
        return desc(image_ids, tail)
    return image_ids


def gen_links(lines):
    parseid = lambda s: s.replace('sha256:', '')
    for line in reversed(list(lines)):
        yield list(map(parseid, line.split()))


if __name__ == '__main__':
    image_ids = {sys.argv[1]}
    links = gen_links(sys.stdin.readlines())
    trunc = lambda s: s[:12]
    print('\n'.join(map(trunc, desc(image_ids, links))))

If you save this as desc.py you could invoke it as follows:

docker images \
    | fgrep -f <(docker inspect --format='{{.Id}} {{.Parent}}' \
        $(docker images --all --quiet) \
        | python3 desc.py f50f9524513f )

Or just use the gist above, which does the same thing.

---

Related Topics