Visibility of Global Variables in Imported Modules

Visibility of global variables in imported modules

Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)

There are different ways to solve this, depending on your actual use case.

Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:

import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()

If you really do want a global, but it's just there to be used by module1, set it in that module.

import module1
module1.a=3
module1.f()

On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:

import shared_stuff
import module1
shared_stuff.a = 3
module1.f()

… and, in module1.py:

import shared_stuff
def f():
    print shared_stuff.a

Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.

Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:

import builtins
import module1
builtins.a = 3
module1.f()

How to make main.py's global variable visible in imported module?

TL;DR : There's no clean way to do so, and actually for pretty good reasons. Also, chances are passing it as an argument will actually be as fast if not faster (arguments are local to the function and local lookups are faster than global ones).

More detailed answer:

The "pretty good reasons" are about maintainability and robustness. A function that only depends on it's inputs is easy to understand and test and has a predictable behaviour. A function that depends on a module-level global is already harder to understand and test and becomes much less predictable. A function depending on a global defined in the main script (yes, this can be done in a hackish way, and I'm NOT going to show this way here) is not only totally untestable and unpredictable, but also requires that the importing modules does define this global, which is brittle at best (and actually makes the module unusable in a different context). There's already plenty of litterature on the "globals are evil" topic so I won't explain further...

Now wrt/ the OP motivations, ie

I would like to avoid to pass millions of times (for very short
computation) the same variable as parameter.

I assume the OP's main concern here are performances...

In Python, passing an argument to a function has a rather low cost - nothing is copied, what is actually passed is (a reference to) the object itself (cf this reference article from Ned Batcheler on Python's names and values for more details). Also, the argument is a local variable of the function so the name lookup during function execution is the fastest possible - faster than resolving a module global, and much faster than resolving an attribute on a module global. So the clean and obvious solution (passing the objec as argument) will probably be as fast if not faster than the hackish one.

As a matter of fact, I did a quick&crude benchmark of both solutions - alas without the timeit module which would have yielded more accurate results but the very nature of the hack made it impossible to use timeit here (yes, hacks never make your life easier) - using both Python 2.7.6 and Python 3.4.3, and the average difference (using 500000 iterations in the main loop) was quite small and actually less than the average variability between two calls of the same code version. I suspect than with real-life code in the function with multiple lookups of the variable, the clean solution would tend to become faster due to local lookup vs global lookup costs.

The code I used for the benchmark:

Clean solution:

# main2.py
import datetime
start = datetime.datetime.now()

from lib2 import myfunc
GENERALPARAM = ["foo", "bar"]

if __name__ == "__main__":
    for i in range(500000):
        myfunc(i, GENERALPARAM)

    done = datetime.datetime.now()
    print("done in %s" % (done - start))

# lib2.py
def myfunc(i, p): 
    return "{} - {}".format(i, p)

The hackish solution:

# main.py
import datetime
start = datetime.datetime.now()

from lib import myfunc
GENERALPARAM = ["foo", "bar"]

if __name__ == "__main__":
    for i in range(500000):
        myfunc(i)

    done = datetime.datetime.now()
    print("done in %s" % (done - start))

# lib.py
import __main__ as main # this is the hack. DONT DO THIS AT HOME

def myfunc(i): 
    return "{} - {}".format(i, main.GENERALPARAM)

Global variable imported from a module does not update - why?

You can think of a package as a dict. Every function and variable in a package is listed as a key in that dict which you can view using globals().

When you import an object from another package, you copy a reference to the object into your own package under a name (usually the same, different if you import <var> as <name>).

By setting the value in the other package, you overwrite the key in that package with a new value, but you leave your reference pointing to the old one.

An analogy with dicts

We can demonstrate the process using dicts as an analogy:

# Our 'packages'
one = {'x': 'start'}
two = {}

# Equivalent of our import
two['x'] = one['x']   

# Variable updated in `one'
one['x'] = 'updated'

# ... and accessed in `two`
print(two['x'])   # Still 'start'

We would not expect the two dict to be updated just because we overwrote a value in one.

What you can do about it

You can modify the object as long as you don't break the pointer by overwriting the variable. For example if x was a dict, you could change a value inside the dict and the two variables would still point to the same object.

Alternatively you could attach a variables to the function like this:

def change(value):
    change.x = value

This does the same work by ensuring we are mutating the same object.

A better answer yet might be to wrap both items in an object if they need to travel together:

class Changer:
    x = 'start'

    @classmethod
    def change(cls, value):
        cls.x = value

At this point however, we could modify the value directly as an attribute:

Changer.x = 'updated'

Which might be the simplest.