Subprocess Wildcard Usage

subprocess wildcard usage

You need to supply shell=True to execute the command through a shell interpreter.
If you do that however, you can no longer supply a list as the first argument, because the arguments will get quoted then. Instead, specify the raw commandline as you want it to be passed to the shell:

 proc = subprocess.Popen('ls *.bc', shell=True,
                                    stdout=subprocess.PIPE,
                                    stderr=subprocess.PIPE)

subprocess popen argument with wildcard

Wildcard expansion is something bash takes care of while you're in the shell. It's not something built into Linux/Unix to be able to expand wildcards or any of that syntax. So you need to be explicit about it and do the expansion by hand.

There is an alternative, which is actually letting the shell do all the work, via shell=True. It has its drawbacks, as documented in the question. Quoting:

This is a good thing, see the warning block in the "Frequently Used Arguments" section, of the subprocess docs. It mainly discusses security implications, but can also helps avoid silly programming errors (as there are no magic shell characters to worry about)

My main complaint with shell=True is it usually implies there is a better way to go about the problem - with your example, you should use the glob module...

Wildcard not working in subprocess call using shlex

For replacing the * with what it means, you either need the shell or you need the glob module. So the easiest way would be shell=True (if the command is constant, I do not see any security holes).

Another approach would be

#!/usr/bin/python
import subprocess
import shlex
import glob

cmd = 'sudo rm -rf /work/TEST/*'
arg = shlex.split(cmd)
arg = arg[:-1] + glob.glob(arg[-1])

# This should work now
p = subprocess.Popen(arg)

or, if you would nevertheless append the path by yourself,

cmd = 'sudo rm -rf'
basearg = shlex.split(cmd)
arg = basearg + glob.glob(path+"/*")

Using a glob to generate arguments with subprocess.run()

Unless you specify shell=True (and as a first approximation, you should never specify shell=True), the arguments you provide are passed as is, with no shell expansions, word-splitting or dequoting. So the filename you pass as an argument is precisely /home/fricadelle/Artist - Album (2008)/*.flac, which is not the name of any file. (That's why you don't need to add backslashes before the spaces and parentheses. If you specified shell=True -- and I repeat, you really should avoid that -- then you would need to include backslashes so that the shell doesn't split the name into several different words.)

When you type
flac_files = "/home/fricadelle/Artist - Album (2008)/*.flac unquoted in a shell, the shell will try to expand that to a list of all the files whose names match then pattern, and will then pass that list as separate arguments. Since subprocess.run doesn't do this, you will have to do it yourself, which you would normally do with glob.glob. For example,

from subprocess import run
from glob import glob
flac_files = "/home/fricadelle/Artist - Album (2008)/*.flac"
run(['metaflac', '--add-replay-gain'] + glob(flac_files))

Note: unlike the shell, glob.glob will return an empty list if the pattern matches no files. You really should check for this error rather than invoke metaflac with no filename options.

Calling 'mv' from Python Popen with wildcard

Use a string instead of an array:

params = "mv /full_path_to_folder_source/*.nib /full_path_to_folder_target/"

When you specify arguments via the array form, the argument '/full_path_to_folder_source/*.nib' is passed to mv. You want to force bash to expand the argument, but Popen won't pass each argument through the shell.

error when using python subprocess with g++

shell=True won't expand wildcards when arguments are put in a list. Quickfix: use a string:

subprocess.run('g++ ./rmc-output/*.cpp -w -o executable', check=True,
           stderr=PIPE, stdout=PIPE, shell=True)

quick but dirty, so a better solution:

• drop shell=True (avoid whenever possible, security issues, lazy command lines...)
• use glob to compute files using python, not the shell

like this:

import glob
subprocess.run(['g++'] + glob.glob('./rmc-output/*.cpp') +['-w', '-o', 'executable'], check=True,
           stderr=PIPE, stdout=PIPE)

note that Windows versions of g++ have internal wildcard expansion to make up for the fact that Windows "shell" hasn't. Would have worked on Windows probably.

Shell expansion in subprocess?

Use the glob package:

import subprocess    
from glob import glob
subprocess.call(["mock"] + glob("*.src.rpm"))

---

Related Topics