Operation on Every Pair of Element in a List

Operation on every pair of element in a list

Check out product() in the itertools module. It does exactly what you describe.

import itertools

my_list = [1,2,3,4]
for pair in itertools.product(my_list, repeat=2):
    foo(*pair)

This is equivalent to:

my_list = [1,2,3,4]
for x in my_list:
    for y in my_list:
        foo(x, y)

Edit: There are two very similar functions as well, permutations() and combinations(). To illustrate how they differ:

product() generates every possible pairing of elements, including all duplicates:

1,1  1,2  1,3  1,4
2,1  2,2  2,3  2,4
3,1  3,2  3,3  3,4
4,1  4,2  4,3  4,4

permutations() generates all unique orderings of each unique pair of elements, eliminating the x,x duplicates:

 .   1,2  1,3  1,4
2,1   .   2,3  2,4
3,1  3,2   .   3,4
4,1  4,2  4,3   .

Finally, combinations() only generates each unique pair of elements, in lexicographic order:

 .   1,2  1,3  1,4
 .    .   2,3  2,4
 .    .    .   3,4
 .    .    .    .

All three of these functions were introduced in Python 2.6.

Perform operation on every array element pair quickly

Use ` itertools.combinations

from itertools import combinations
a = [5,2,8,14,6,13]

print [sum(i) for i in list(combinations(a, 2))]

No need of list(). Thanks to @PeterWood

print [sum(i) for i in combinations(a, 2)]

Output:

[7, 13, 19, 11, 18, 10, 16, 8, 15, 22, 14, 21, 20, 27, 19]

Demo

Process every pair in a sequence

var query = transactions
             .SelectMany((val1,j) => transactions.Select((val2,i) => new {v1=val1, v2=val2, i=i, j=j}))
             .Where(x => x.i < x.j);

var result = query.Select(x=> x.v1.UniqueID == x.v2.UniqueID);

This performs the comparison the correct number of times. The result also includes the indexes i, j of the two elements that matched.

Java 8 streams: process every possible pair of elements from list

You can do this using the flatMap operation:

list.stream()
    .flatMap(i -> list.stream().filter(j -> !i.equals(j)).map(j -> i * 2 + j))
    .forEach(System.out::println);

This code is creating a Stream of the input list. It flat maps each element of the list with a stream created by the same list, where the current element was filtered out, and each element of this new list is the result of the i * 2 + j operation.

All the elements are then printed to the console.

Iterate over all pairs of consecutive items in a list

Just use zip

>>> l = [1, 7, 3, 5]
>>> for first, second in zip(l, l[1:]):
...     print first, second
...
1 7
7 3
3 5

If you use Python 2 (not suggested) you might consider using the izip function in itertools for very long lists where you don't want to create a new list.

import itertools

for first, second in itertools.izip(l, l[1:]):
    ...

The same operation for many pairs in Python

One solution is to transpose pair:

f_test(scalar, pair.T)
#array([ 3, 10, 13])

Or you could use list comprehension:

[f_test(scalar=scalar, pair=p) for p in pair]
#[3, 10, 13]

---

Timing results

Looks like the first method is way faster. For an array of length 100,000 the speed improvement is ~270X on my computer!

N = 100000
scalar = 0
pair = np.array([[np.random.randint(0,10), np.random.randint(0,10)] for i in range(N)])

# Using transpose
%%timeit
f_test(scalar, pair.T)
#1000 loops, best of 3: 229 µs per loop

# List comprehension
%%timeit
[f_test(scalar=scalar, pair=p) for p in pair]
#10 loops, best of 3: 62 ms per loop

How to apply function to consecutive pairs of values from a list?

If you reverse the order of input arguments of check_state from def check_state(now, prev): to def check_state(prev, now): then the problem of applying a function to consecutive pairs of values of your lists becomes quite easy. I came up with the following function:

import itertools

def apply_pairwise(values,
                   function):
    """
    Applies function to consecutive pairs of elements from an input list
    """
    def pairwise(iterable):
        """
        s -> (s0,s1), (s1,s2), (s2,s3), ...
        """
        a, b = itertools.tee(iterable)
        next(b, None)
        return zip(a, b)

    yield from itertools.chain([None],
                               itertools.starmap(function, pairwise(values)))

Examples of usage:

>>> btc = [200, 149, 98, 44]
>>> list(apply_pairwise(btc, check_state))
[None, 2, 3, 3]
>>> eth = [200, 320, 405, 460]
>>> list(apply_pairwise(eth, check_state))
[None, 1, 0, 0]

---

If you can't reverse the inputs:

If it's impossible to change the order of inputs, we could adopt our function a bit:

import itertools

def apply_pairwise(values,
                   function,
                   *,
                   reverse=False):
    """
    Applies function to consecutive pairs of elements from an input list
    """
    def pairwise(iterable):
        """
        s -> (s0,s1), (s1,s2), (s2,s3), ...
        or -> (s1,s0), (s2,s1), (s3,s2), ... if reverse
        """
        a, b = itertools.tee(iterable)
        next(b, None)

        if reverse:
            return zip(b, a)
        return zip(a, b)

    yield from itertools.chain([None],
                               itertools.starmap(function, pairwise(values)))

and you could use it like this:

>>> btc = [200, 149, 98, 44]
>>> list(apply_pairwise(btc, check_state, reverse=True))
[None, 2, 3, 3]
>>> eth = [200, 320, 405, 460]
>>> list(apply_pairwise(eth, check_state, reverse=True))
[None, 1, 0, 0]

---

Explanation:

In order to get consecutive pairs of elements, we could use a helper function from recipes of itertools:

def pairwise(iterable):
    "s -> (s0,s1), (s1,s2), (s2, s3), ..."
    a, b = tee(iterable)
    next(b, None)
    return zip(a, b)

It works like this:

>>> list(pairwise(range(5)))
[(0, 1), (1, 2), (2, 3), (3, 4)]

Now for each pair of elements we would like to apply your check_state function. Here itertools.starmap can be useful. It works like this:

>>> list(itertools.starmap(pow, [(2, 3), (2, 10), (10, 3), (3, 4)]))
[8, 1024, 1000, 81]

The only thing left is to prepend the values yielded by starmap by None. As starmap makes an iterator, we could use itertools.chain to combine the first None with the rest of the elements.

---

P.S.: Applying this to values of your tickers dict should be easy enough. I will leave it to you.

---

Related Topics