Why Can't Environmental Variables Set in Python Persist

Why can't environmental variables set in python persist?

You can't do it from python, but some clever bash tricks can do something similar. The basic reasoning is this: environment variables exist in a per-process memory space. When a new process is created with fork() it inherits its parent's environment variables. When you set an environment variable in your shell (e.g. bash) like this:

export VAR="foo"

What you're doing is telling bash to set the variable VAR in its process space to "foo". When you run a program, bash uses fork() and then exec() to run the program, so anything you run from bash inherits the bash environment variables.

Now, suppose you want to create a bash command that sets some environment variable DATA with content from a file in your current directory called ".data". First, you need to have a command to get the data out of the file:

cat .data

That prints the data. Now, we want to create a bash command to set that data in an environment variable:

export DATA=`cat .data`

That command takes the contents of .data and puts it in the environment variable DATA. Now, if you put that inside an alias command, you have a bash command that sets your environment variable:

alias set-data="export DATA=`cat .data`"

You can put that alias command inside the .bashrc or .bash_profile files in your home directory to have that command available in any new bash shell you start.

Environment variable gets removed immediately after it is set Python

You are only setting the environment variable for that specific running Python process. Once the process exits, the variable ceases to exist. It appears you cannot set a persistent environment variable from Python, as is described here:

Why can't environmental variables set in python persist?

It looks like you'll need to shell execute whatever variable you want to persist, as is described in the second answer to the above questions, like so:

import pipes
import random
r = random.randint(1,100)
print("export BLAHBLAH=%s" % (pipes.quote(str(r))))

Edit:

As per @SuperStormer's comment below--they are correct--this solution is Unix/Mac specifc. If you need to accomplish the same thing on Windows you would utilize setx like so (this would set the variable for only current user, you need to add a /m switch on the end to set for the local machine):

import pipes
import random
r = random.randint(1,100)
print("setx BLAHBLAH %s" % (pipes.quote(str(r))))

How do I make environment variable changes stick in Python?

Under Windows it's possible for you to make changes to environment variables persistent via the registry with this recipe, though it seems like overkill.

To echo Brian's question, what are you trying to accomplish? There is probably an easier way.

Change environment variables persistently with Python

I'm speaking for Linux here, not sure about Windows.

Environment variables don't work that way. They are a part of the process (which is what you modify by changing os.environ), and they will propagate to child processes of your process (and their children obviously). They are in-memory only, and there is no way to "set and persist" them directly.

There are however several configuration files which allow you to set the environment on a more granular basis. These are read by various processes, and can be system-wide, specific to a user, specific to a shell, to a particular type of process etc.

Some of them are:

• /etc/environment for system-wide variables
• /etc/profile for shells (and their children)
• Several other shell-specific files in /etc
• Various dot-files in a user's home directory such as .profile, .bashrc, .bash_profile, .tcshrc and so on. Read your shell's documentation.
• I believe that there are also various ways to configure environment variables launched from GUIs (e.g. from the gnome panel or something like that).

Most of the time you'll want to set environment variables for the current user only. If you only care about shells, append them to ~/.profile in this format:

NAME="VALUE"

Why can't environmental variables set in python persist?

You can't do it from python, but some clever bash tricks can do something similar. The basic reasoning is this: environment variables exist in a per-process memory space. When a new process is created with fork() it inherits its parent's environment variables. When you set an environment variable in your shell (e.g. bash) like this:

export VAR="foo"

What you're doing is telling bash to set the variable VAR in its process space to "foo". When you run a program, bash uses fork() and then exec() to run the program, so anything you run from bash inherits the bash environment variables.

Now, suppose you want to create a bash command that sets some environment variable DATA with content from a file in your current directory called ".data". First, you need to have a command to get the data out of the file:

cat .data

That prints the data. Now, we want to create a bash command to set that data in an environment variable:

export DATA=`cat .data`

That command takes the contents of .data and puts it in the environment variable DATA. Now, if you put that inside an alias command, you have a bash command that sets your environment variable:

alias set-data="export DATA=`cat .data`"

You can put that alias command inside the .bashrc or .bash_profile files in your home directory to have that command available in any new bash shell you start.

How to set environment variables with python?

Ok so based on the comment above, to help future users who might not be aware of this, you cannot set environment variables outside the scope of the current process with Python.

You can make python aware of some variables and change env variables for the scope of a process and its child processes. But you can not set values for env in the system itself or other processes (that are not children of the current process). For example if I set a env variable called HOST_URL it wont be actually accessible in the system environment.

I found three ways to actually set the variables by:

1. Running a bash script to set the env variable values
2. Use VSCode launch.json for setting the variables either with env or envFile
3. Define them through Docker file or docker-compose.yml if you are containerizing your app

Note:
If there are other options to address this please comment and I ll add them. I want this to be a helpfull post for new developers like myself. Shaming and non-helpful comments never helped anyone or improved anything. This is, or should be, a learning and knowledge exchange platform and it should be open to all programming questions Stack Overflow Isn’t Very Welcoming. It’s Time for That to Change

Change environment variables persistently with Python

I'm speaking for Linux here, not sure about Windows.

Environment variables don't work that way. They are a part of the process (which is what you modify by changing os.environ), and they will propagate to child processes of your process (and their children obviously). They are in-memory only, and there is no way to "set and persist" them directly.

There are however several configuration files which allow you to set the environment on a more granular basis. These are read by various processes, and can be system-wide, specific to a user, specific to a shell, to a particular type of process etc.

Some of them are:

• /etc/environment for system-wide variables
• /etc/profile for shells (and their children)
• Several other shell-specific files in /etc
• Various dot-files in a user's home directory such as .profile, .bashrc, .bash_profile, .tcshrc and so on. Read your shell's documentation.
• I believe that there are also various ways to configure environment variables launched from GUIs (e.g. from the gnome panel or something like that).

Most of the time you'll want to set environment variables for the current user only. If you only care about shells, append them to ~/.profile in this format:

NAME="VALUE"

Unable to print environmental variable in python which is set manually by the user to use that variable in later part of code

You need to make sure your environment variable is persistent following a restart of your shell otherwise your new environment variable will not be accessible later on

ekavala@elx75030xhv:/var/tmp$ export NEWVAR='alan'
ekavala@elx75030xhv:/var/tmp$ python test.py 
alan
*closes shell and reopens*
ekavala@elx75030xhv:/var/tmp$ python test.py 
Please set the environment variable NEWVAR

Update your $HOME/.bashrc or /etc/environment with the variable instead of just doing a setx or export

Note: If you update /etc/environment you will need to reboot your computer to have the environment variables set in your shell

How to set environment variables of parent shell in Python?

This isn't possible.

Child processes inherit their environments from their parents rather than share them. Therefore any modifications you make to your environment will be reflected only in the child (python) process. Practically, you're just overwriting the dictionary the os module has created based on your environment of your shell, not the actual environment variables of your shell.

https://askubuntu.com/questions/389683/how-we-can-change-linux-environment-variable-in-python

Why can't environmental variables set in python persist?

---

Related Topics