Why Does Using 'Arg=None' Fix Python's Mutable Default Argument Issue

Why does using `arg=None` fix Python's mutable default argument issue?

The default value of a_list (or any other default value, for that matter) is stored in the function's interiors once it has been initialized and thus can be modified in any way:

>>> def f(x=[]): return x
...
>>> f.func_defaults
([],)
>>> f.func_defaults[0] is f()
True

resp. for Python 3:

>>> def f(x=[]): return x
...
>>> f.__defaults__
([],)
>>> f.__defaults__[0] is f()
True

So the value in func_defaults is the same which is as well known inside function (and returned in my example in order to access it from outside.

In other words, what happens when calling f() is an implicit x = f.func_defaults[0]. If that object is modified subsequently, you'll keep that modification.

In contrast, an assignment inside the function gets always a new []. Any modification will last until the last reference to that [] has gone; on the next function call, a new [] is created.

In order words again, it is not true that [] gets the same object on every execution, but it is (in the case of default argument) only executed once and then preserved.

Function argument with mutables, avoid `if arg is None` construct

I think in most cases, the if arg is None: arg = ... syntax is the best solution. But if you are really bothered by it, here's a decorator which works for single-argument functions:

from functools import wraps

def default_argument(arg_factory):
    def decorator(f):
        @wraps(f)
        def wrapped(arg=None):
            if arg is None:
                arg = arg_factory()
            return f(arg)
        return wrapped
    return decorator

Usage examples below: you can either pass a reference to an existing function, or a lambda.

import datetime as dt

@default_argument(dt.datetime.now)
def dt_to_str(dtime):
    return dtime.strftime('%c')

print(dt_to_str())
# Mon Oct 25 00:16:03 2021

@default_argument(lambda: [])
def thing_with_list(lst):
    return lst + [123]

print(thing_with_list())
# [123]

Avoiding Default Argument Value is Mutable Warning (PyCharm)

If your only requirement is to bundle several elements together into one object, and that object doesn't need to be mutable, use a tuple instead of a list. Tuples provide all the same behavior of lists except for modification of their elements. To be exact, mutable elements can still be mutated, but elements of the tuple cannot be assigned, added, or removed, and the tuple itself is considered an immutable data structure. The syntax for constructing a tuple is the same as a list, just with parentheses instead of square brackets:

# Immutable container with immutable elements, perfectly fine to use as a default argument
tuple_o_fruits = ("Apple", "Banana", "Orange")

Python's lack of generic enforceable const-correctness is one of the few things that really irks me about the language. The standard solution is to use builtin immutable datatypes like tuple in place of mutable ones like list. The immutable form of set is frozenset, for example. Many common builtin types such as str, int, and float are only available in immutable form. bytes is an interesting case because the immutable form is better known than its mutable alternative, bytearray. The most prominent builtin mutable type that lacks an immutable alternative is dict (but it's not hard to make your own).

So, the real answer to "how do I avoid warnings about mutable default arguments" is, 9 times out of 10, to not use a mutable object, because immutable alternative are generally readily available. This should be sufficient for your case.

Why the mutable default argument fix syntax is so ugly, asks python newbie

Default arguments are evaluated at the time the def statement is executed, which is the probably the most reasonable approach: it is often what is wanted. If it wasn't the case, it could cause confusing results when the environment changes a little.

Differentiating with a magic local method or something like that is far from ideal. Python tries to make things pretty plain and there is no obvious, clear replacement for the current boilerplate that doesn't resort to messing with the rather consistent semantics Python currently has.

Least Astonishment and the Mutable Default Argument

Actually, this is not a design flaw, and it is not because of internals or performance. It comes simply from the fact that functions in Python are first-class objects, and not only a piece of code.

As soon as you think of it this way, then it completely makes sense: a function is an object being evaluated on its definition; default parameters are kind of "member data" and therefore their state may change from one call to the other - exactly as in any other object.

In any case, the effbot (Fredrik Lundh) has a very nice explanation of the reasons for this behavior in Default Parameter Values in Python.
I found it very clear, and I really suggest reading it for a better knowledge of how function objects work.

Fix mutable default arguments via metaclass

Your code as it stands is actually doing what you expect. It is passing a new copy of the default to the function when called. However, since you do nothing with this new value it is garbage collected and the memory is free for immediate reallocation on your very next call.

Thus, you keep getting the same id().

The fact that the id() for two objects at different points in time is the same does not indicate they are the same object.

To see this effect, alter your function so it does something with the value that will increase its reference count, such as:

class ImmutableDefaultArgumentsObject(ImmutableDefaultArgumentsBase):
    cache = []
    def function_a(self, mutable_default_arg=set()):
        print("function_b", mutable_default_arg, id(mutable_default_arg))
        self.cache.append(mutable_default_arg)

Now running your code will provide:

function_b set() 4362897448
function_b set() 4362896776
function_b set() 4362898344
function_b set() 4362899240
function_b set() 4362897672

Are there caveats of using mutable types as default parameters in functions in Python?

The main caveat is that someone reading your code later, unless they're familiar with this particular gotcha, might not understand that checked persists, or that the caller isn't actually intended to ever provide a value for it. Assuming you wanted to avoid generators, it would be more clear to write something like:

data = [3, 4, 1, 8, 5, 9, 2, 6, 7]
checked = []

def get_min():
    mn = min(x for x in data if x not in checked)
    checked.append(mn)
    return mn

---

Related Topics