How to Do Multiple Arguments to Map Function Where One Remains the Same

How to do multiple arguments to map function where one remains the same

One option is a list comprehension:

[add(x, 2) for x in [1, 2, 3]]

More options:

a = [1, 2, 3]

import functools
map(functools.partial(add, y=2), a)

import itertools
map(add, a, itertools.repeat(2, len(a)))

Using map with function that has multiple arguments

You should use a lambda function, to see this works lets start by using a helper function to map f over some list.

map helper [1, 2, 3] where
  helper x = f x y z

In Haskell there are two syntax for functions so lets use the lambda syntax to define our helper function:

map helper [1, 2, 3] where
  helper = \x -> f x y z

using the lambda syntax we don't need to give our helper function an explicit name, it can just be an anonymous function we map over the input

map (\x -> f x y z) [1, 2, 3]

So now you can say

mapF y z = map (\x -> f x y z) [1,2,3]

But presumably you don't want x to be 1, 2 and 3, you want it to be a list
you pass as an argument to mapF. So you need to give that a different name:

mapF xs y z = map (\x -> f x y z) xs

It is Haskell convention to use s as a suffix for variables that hold lists or other containers. So if one value is x then a list of them is xs

Using map with multiple args

Regarding why map truncates in python3, this is simply because python3's map is actually itertools.imap. And the documentation says:

Like map() but stops when the shortest iterable is exhausted
instead of filling in None for shorter iterables. The reason for
the difference is that infinite iterator arguments are typically an
error for map() (because the output is fully evaluated) but
represent a common and useful way of supplying arguments to
imap().

The truncation allows you to do things like map(func, itertools.repeat(5), [1,2,3]) and iterate over the result without worries. With the old map that would be an infinite loop.

One of the most significant changes in python3 is that a lot of built-in functions now return generators instead of lists, including map and zip. This "increased lazyness" changed the way these functions should be used and thus the modified behaviour.

As to why one would ever use python2's multiple-iterables to map I don't know. Sure, it's a shortcut for something like (in python3):

list(itertools.starmap(function, itertools.zip_longest(*iterables)))

This might have some corner case usage, but I've never seen it used.
Probably most people don't even know that map accepts a sequence of iterables.
So, AFAIK there isn't any super-power deriving from the use of multiple arguments.

As to why map is in the language, that's because map was there long before list-comprehensions. Before list-comprehensions it was quite useful for building lists.
It wasn't remove for backward compatibility and because many people actually like it,
although Guido did want to remove it.

To learn more about the history of map, filter and reduce and other functional aspects, read: The History of Python: Origins of Python's "Functional" Features

Pass multiple arguments to map's func argument

using sections

you can use a section like this:

Prelude> let func x y = x / y
Prelude> map (`func` 2) [2,4,6,8]
[1.0,2.0,3.0,4.0]

see when you write a function in backticks you make it into an operator - then you can plug in values to the left or to the right to (for x or y here).

If you wanted to substitute for x then you could have used normal partial application as well:

Prelude> map (func 24) [2,4,6,8]
[12.0,6.0,4.0,3.0]

which is the same as

Prelude> map (24 `func`) [2,4,6,8]
[12.0,6.0,4.0,3.0]

using a lambda

of course if you don't like this at all you can always use lambda's:

Prelude> map (\x -> func x 2) [2,4,6,8]
[1.0,2.0,3.0,4.0]

using list-comprehension

or you can do the same as you did in Python (use a list comprehension):

Prelude> [ func x 2 | x <- [2,4,6,8] ]
[1.0,2.0,3.0,4.0]

ok I think right now I can think of no other (easy) ones ;)

using partial application with flip

flip and partial application (I think a bit less readable - but I find it kind of funny ;):

Prelude> map (flip f 2) [2,4,6,8]
[1.0,2.0,3.0,4.0]

remark / div

as you only dealt with Ints here you probably would want to use div instead of (/):

Prelude> map (`div` 2) [2,4,6,8]
[1,2,3,4]

How to use multiprocessing pool.map with multiple arguments

The answer to this is version- and situation-dependent. The most general answer for recent versions of Python (since 3.3) was first described below by J.F. Sebastian.¹ It uses the Pool.starmap method, which accepts a sequence of argument tuples. It then automatically unpacks the arguments from each tuple and passes them to the given function:

import multiprocessing
from itertools import product

def merge_names(a, b):
    return '{} & {}'.format(a, b)

if __name__ == '__main__':
    names = ['Brown', 'Wilson', 'Bartlett', 'Rivera', 'Molloy', 'Opie']
    with multiprocessing.Pool(processes=3) as pool:
        results = pool.starmap(merge_names, product(names, repeat=2))
    print(results)

# Output: ['Brown & Brown', 'Brown & Wilson', 'Brown & Bartlett', ...

For earlier versions of Python, you'll need to write a helper function to unpack the arguments explicitly. If you want to use with, you'll also need to write a wrapper to turn Pool into a context manager. (Thanks to muon for pointing this out.)

import multiprocessing
from itertools import product
from contextlib import contextmanager

def merge_names(a, b):
    return '{} & {}'.format(a, b)

def merge_names_unpack(args):
    return merge_names(*args)

@contextmanager
def poolcontext(*args, **kwargs):
    pool = multiprocessing.Pool(*args, **kwargs)
    yield pool
    pool.terminate()

if __name__ == '__main__':
    names = ['Brown', 'Wilson', 'Bartlett', 'Rivera', 'Molloy', 'Opie']
    with poolcontext(processes=3) as pool:
        results = pool.map(merge_names_unpack, product(names, repeat=2))
    print(results)

# Output: ['Brown & Brown', 'Brown & Wilson', 'Brown & Bartlett', ...

In simpler cases, with a fixed second argument, you can also use partial, but only in Python 2.7+.

import multiprocessing
from functools import partial
from contextlib import contextmanager

@contextmanager
def poolcontext(*args, **kwargs):
    pool = multiprocessing.Pool(*args, **kwargs)
    yield pool
    pool.terminate()

def merge_names(a, b):
    return '{} & {}'.format(a, b)

if __name__ == '__main__':
    names = ['Brown', 'Wilson', 'Bartlett', 'Rivera', 'Molloy', 'Opie']
    with poolcontext(processes=3) as pool:
        results = pool.map(partial(merge_names, b='Sons'), names)
    print(results)

# Output: ['Brown & Sons', 'Wilson & Sons', 'Bartlett & Sons', ...

^{1. Much of this was inspired by his answer, which should probably have been accepted instead. But since this one is stuck at the top, it seemed best to improve it for future readers.}

Pass named arguments in list comprehensions in map function

Use a generator expression instead of a map.

resized_imgs = tuple(resize_image(img, None, output_height) for img in all_img_steps)

Passing multiple parameters to pool.map() function in Python

You can use functools.partial for this (as you suspected):

from functools import partial

def target(lock, iterable_item):
    for item in iterable_item:
        # Do cool stuff
        if (... some condition here ...):
            lock.acquire()
            # Write to stdout or logfile, etc.
            lock.release()

def main():
    iterable = [1, 2, 3, 4, 5]
    pool = multiprocessing.Pool()
    l = multiprocessing.Lock()
    func = partial(target, l)
    pool.map(func, iterable)
    pool.close()
    pool.join()

Example:

def f(a, b, c):
    print("{} {} {}".format(a, b, c))

def main():
    iterable = [1, 2, 3, 4, 5]
    pool = multiprocessing.Pool()
    a = "hi"
    b = "there"
    func = partial(f, a, b)
    pool.map(func, iterable)
    pool.close()
    pool.join()

if __name__ == "__main__":
    main()

Output:

hi there 1
hi there 2
hi there 3
hi there 4
hi there 5

---

Related Topics