Python Overwriting Variables in Nested Functions

Python overwriting variables in nested functions

In Python 3.x, you can use the nonlocal keyword:

def outer():
    string = ""
    def inner():
        nonlocal string
        string = "String was changed by a nested function!"
    inner()
    return string

In Python 2.x, you could use a list with a single element and overwrite that single element:

def outer():
    string = [""]
    def inner():
        string[0] = "String was changed by a nested function!"
    inner()
    return string[0]

How do I change nesting function's variable in the nested function

In Python 3.x, you can use the nonlocal declaration (in nested) to tell Python you mean to assign to the count variable in nesting.

In Python 2.x, you simply can't assign to count in nesting from nested. However, you can work around it by not assigning to the variable itself, but using a mutable container:

def nesting():
    count = [0]
    def nested():
        count[0] += 1

    for i in range(10):
        nested()
    print count[0]

Although for non-trivial cases, the usual Python approach is to wrap the data and functionality in a class, rather than using closures.

Does an equivalent of override exist for nested functions?

Here's one way of doing it, creating a new foo that "does the right thing" by hacking the function internals. ( As mentioned by @DSM ). Unfortunately we cant just jump into the foo function and mess with its internals, as they're mostly marked read only, so what we have to do is modify a copy we construct by hand.

# Here's the original function
def foo():
  def bar():
    print("    In bar orig")
  def baz():
    print("  Calling bar from baz")
    bar()
  print("Foo calling bar:")
  bar()
  print("Foo calling baz:")
  baz()

# Here's using it
foo()

# Now lets override the bar function

import types

# This is our replacement function
def my_bar():
  print("   Woo hoo I'm the bar override")

# This creates a new code object used by our new foo function 
# based on the old foo functions code object.
foocode = types.CodeType(
    foo.func_code.co_argcount,
    foo.func_code.co_nlocals,
    foo.func_code.co_stacksize,
    foo.func_code.co_flags,
    foo.func_code.co_code,
    # This tuple is a new version of foo.func_code.co_consts
    # NOTE: Don't get this wrong or you will crash python.
    ( 
       foo.func_code.co_consts[0],
       my_bar.func_code,
       foo.func_code.co_consts[2],
       foo.func_code.co_consts[3],
       foo.func_code.co_consts[4]
    ),
    foo.func_code.co_names,
    foo.func_code.co_varnames,
    foo.func_code.co_filename,
    foo.func_code.co_name,
    foo.func_code.co_firstlineno,
    foo.func_code.co_lnotab,
    foo.func_code.co_freevars,
    foo.func_code.co_cellvars )

# This is the new function we're replacing foo with
# using our new code.
foo = types.FunctionType( foocode , {})

# Now use it
foo()

I'm pretty sure its not going to catch all cases. But it works for the example (for me on an old python 2.5.1 )

Ugly bits that could do with some tidy up are:

1. The huge argument list being passed to CodeType
2. The ugly tuple constructed from co_consts overriding only one member. All the info is in co_consts to determine which to replace - so a smarter function could do this. I dug into the internals by hand using print( foo.func_code.co_consts ).

You can find some information about the CodeType and FunctionType by using the interpreter
command help( types.CodeType ).

UPDATE:
I thought this was too ugly so I built a helper function to make it prettier. With the helper you can write:

# Use our function to get a new version of foo with "bar" replaced by mybar    
foo = monkey_patch_fn( foo, "bar", my_bar )

# Check it works
foo()

Here's the implementation of monkey_patch_fn:

# Returns a copy of original_fn with its internal function
# called name replaced with new_fn.
def monkey_patch_fn( original_fn, name, new_fn ):

  #Little helper function to pick out the correct constant
  def fix_consts(x):
    if x==None: return None
    try:
      if x.co_name == name:
        return new_fn.func_code
    except AttributeError, e:
        pass
    return x

  original_code = original_fn.func_code
  new_consts = tuple( map( fix_consts, original_code.co_consts ) )
  code_type_args = [
     "co_argcount", "co_nlocals", "co_stacksize", "co_flags", "co_code",
     "co_consts", "co_names", "co_varnames", "co_filename", "co_name",
     "co_firstlineno", "co_lnotab", "co_freevars", "co_cellvars" ]

  new_code = types.CodeType(
     *[ ( getattr(original_code,x) if x!="co_consts" else new_consts )
        for x in code_type_args ] )
  return types.FunctionType( new_code, {} )

Overriding an inner function of a method in python

Either factoring _do_atomic_job() into a proper method, or maybe factoring it
into its own class seem like the best approach to take. Overriding an inner
function can't work, because you won't have access to the local variable of the
containing method.

You say that _do_atomic_job() takes a lot of parameters returns lots of values. Maybe you group some of these parameters into reasonable objects:

_do_atomic_job(start_x, start_y, end_x, end_y) # Separate coordinates
_do_atomic_job(start, end)                     # Better: start/end points
_do_atomic_job(rect)                           # Even better: rectangle

If you can't do that, and _do_atomic_job() is reasonably self-contained,
you could create helper classes AtomicJobParams and AtomicJobResult.
An example using namedtuples instead of classes:

AtomicJobParams = namedtuple('AtomicJobParams', ['a', 'b', 'c', 'd'])

jobparams = AtomicJobParams(a, b, c, d)
_do_atomic_job(jobparams)                # Returns AtomicJobResult

Finally, if the atomic job is self-contained, you can even factor it into its
own class AtomicJob.

class AtomicJob:
    def __init__(self, a, b, c, d):
        self.a = a
        self.b = b
        self.c = c
        self.d = d
        self._do_atomic_job()

    def _do_atomic_job(self):
        ...
        self.result_1 = 42
        self.result_2 = 23
        self.result_3 = 443

Overall, this seems more like a code factorization problem. Aim for rather lean
classes that delegate work to helpers where appropriate. Follow the single responsibility principle. If values belong together, bundle them up in a value class.

As David Miller (a prominent Linux kernel developer) recently said:

If you write interfaces with more than 4 or 5 function arguments, it's
possible that you and I cannot be friends.

---

Related Topics