Preserving Global State in a Flask Application

Preserving global state in a flask application

Based on your question, I think you're confused about the definition of "global".

In a stock Flask setup, you have a Flask server with multiple threads and potentially multiple processes handling requests. Suppose you had a stock global variable like "itemlist = []", and you wanted to keep adding to it in every request - say, every time someone made a POST request to an endpoint. This is totally possible in theory and practice. It's also a really bad idea.

The problem is that you can't easily control which threads and processes "win" - the list could up in a really wonky order, or get corrupted entirely. So now you need to talk about locks, mutexs, and other primitives. This is hard and annoying.

You should keep the webserver itself as stateless as possible. Each request should be totally independent and not share any state in the server. Instead, use a database or caching layer which will handle the state for you. This seems more complicated but is actually simpler in practice. Check out SQLite for example ; it's pretty simple.

To address the 'flask.g' object, that is a global object on a per request basis.

http://flask.pocoo.org/docs/api/#flask.g

It's "wiped clean" between requests and cannot be used to share state between them.

Keeping global state in Flask Application

The warnings about "not keeping a per-process global state" in a web backend app (you'll have the very same issue with Django or any wsgi app) only applies to state that you expect to be shared between requests AND processes.

If it's ok for you to have per-process state (for example db connection is typically a per-process state) then it's not an issue. wrt/ connections pooling, you could (or not) decide that having distinct pools per server process is ok.

For anything else - any state that needs to be shared amongst processes -, this is usually handled by some using some external database or cache process, so if you want to have one single connection pool for all your Flask processes you will have to use a distinct server process for maintaining the pool indeed.

Also note that:

multiple processes of the same application (which, as I understand, can be spawned in the case of large production flask servers)

Actually this has nothing to do with being "large". With a traditional "blocking" server, you can only handle concurrent requests by using either multithreading or multiprocessing. The unix philosophy traditionnally favors multiprocessing ("prefork" model) for various reasons, and Python's multithreading is bordering on useless anyway (at least in this context) so you don't have much choice if you hope to serve one more one single request at a time.

To make a long story short, consider that just any production setup for a wsgi app will run multiple processes in the background, period.

Are global variables thread-safe in Flask? How do I share data between requests?

You can't use global variables to hold this sort of data. Not only is it not thread safe, it's not process safe, and WSGI servers in production spawn multiple processes. Not only would your counts be wrong if you were using threads to handle requests, they would also vary depending on which process handled the request.

Use a data source outside of Flask to hold global data. A database, memcached, or redis are all appropriate separate storage areas, depending on your needs. If you need to load and access Python data, consider multiprocessing.Manager. You could also use the session for simple data that is per-user.

---

The development server may run in single thread and process. You won't see the behavior you describe since each request will be handled synchronously. Enable threads or processes and you will see it. app.run(threaded=True) or app.run(processes=10). (In 1.0 the server is threaded by default.)

---

Some WSGI servers may support gevent or another async worker. Global variables are still not thread safe because there's still no protection against most race conditions. You can still have a scenario where one worker gets a value, yields, another modifies it, yields, then the first worker also modifies it.

---

If you need to store some global data during a request, you may use Flask's g object. Another common case is some top-level object that manages database connections. The distinction for this type of "global" is that it's unique to each request, not used between requests, and there's something managing the set up and teardown of the resource.

Flask app, global variable over multiple files

Just use a regular Python module-level variable.

# app.py

g = "Example"

@app.route("/example")
def example_endpoint():
    return g

# main.py

import app

app.g = "Hello"

Quoting the same page you linked:

The application context is created and destroyed as necessary. When a Flask application begins handling a request, it pushes an application context and a request context. When the request ends it pops the request context then the application context. Typically, an application context will have the same lifetime as a request.

So your setting flask.g outside of a request context (in your main.py) doesn't carry your value to anywhere.

Preserving state in mod_wsgi Flask application

This I suppose will depend on the database (and ORM) you're using with. If you're using SQLAlchemy or flask-sqlalchemy, (which I highly suggest you to do), you will usually define a single global variable for the database connection. This is usually set up in an init_app()-function. If you look at the application set up code, you'll see that quite often the main flask application object is a global variable app. Very often you'll find the database object as a global variable db which is often imported together with the app to other parts of the application.

You should look at the application and request-context documentation and especially the locality of the context, which explains how this works. Basically the context is never shared between requests (and thus threads), so there should be no race conditions and no problems.

Also, the global keyword in your case is not necessary. The point is that you will never really change the content of the variable db. It's an object which controls the database connection and you only call the methods it offers. The global keyword is necessary when you want to mutate the contents of the variable, by assigning a new value to it.

So, I would refactor the setup() as follows (and preferrably put this an __init__.py so it's nicely importable)

from flask import Flask

def setup():
    app = Flask(__name__)
    db = get_db() #This is a local variable now
    # whatever else needs to be done
    return app, db

app, db = setup()

And then in the mod_wsgi.py

from myapp import app

def application(environ, start_response):
    return app(environ, start_response)   

In Python Flask: What are appropriate places to store data?

In order to store data for the lifetime of a single request, how should that be done?

The g object is designed for this. The documentation states:

Flask provides you with a special object that ensures it is only valid for the active request and that will return different values for each request.

Although the documentation refers to g as "global", that's not really accurate - "thread-global" would be better.

In order to store data for the lifetime of an application, how should that be done?

I think the answer to this question answers this as well (or better) than I could:
Preserving global state in a flask application

Flask could be used in a multi-process environment. Is it correct to assume that in such a mode of operation there will be multiple application-wide objects? (This would imply that all of these app or g objects then need to be initialized individually for the lifetime of each worker process.)

In a multi-process environment, each request is handled as a seperate thread, and g is intialized and destroyed on a per-request basis, so there will be as many concurrent g object as threads - though each thread can only see it's own. In most scenarios I suspect there should only ever one app object, an instance of the Flask() class created by the programmer, i.e. app = Flask(__name__) or similar.

Blueprints and Application Dispatching are two way of having "multiple" application objects, in as far as you have multiple applications running concurrently.

Creating state in a Flask webapplication

Your example should not be done with global state, it will not work for the reason you mentioned - requests might land into different processes that will have different global values.

You can handle storing global variables by using a key-value cache, such as memcached or Redis, or file-system based cache - check Flask-Chaching package and particular docs https://flask-caching.readthedocs.io/en/latest/#built-in-cache-backends

Unable to share variables between 2 flasks requests in a view function

The linked answer appears to be completely wrong. The g object is for storing global data from one function to another within the same request, and is not at all for sharing data between requests.

For that you need sessions:

from flask import Flask, session

@home.route('/save', methods=['GET'])
def save_ressource():
    an_object = {'key': 'value'}
    session['an_object'] = an_object
    return 'sucees'

@home.route('/read', methods=['GET'])
def read_ressource():
    an_object = session.get('an_object')
    if an_object:
        return 'sucess'
    else:
        return 'failure'

---

Related Topics