Executing a Subprocess Fails

Executing a subprocess fails

To execute a batch file in Windows:

from subprocess import Popen
p = Popen("batchfile.bat", cwd=r"c:\directory\containing\batchfile")
stdout, stderr = p.communicate()

If you don't want to execute the batch file, but rather execute the command in your question directly from Python, you need to experiment a bit with the first argument to Popen.

First of all, the first argument can either be a string or a sequence.

So you either write:

p = Popen(r'"C:\Program Files\Systems\Emb Work 5.4\common\bin\run" "C:\Program Files\Systems\Emb Work 5.4\arm\bin\mpr.dll" ... ...', cwd=r"...")

or

p = Popen([r"C:\Program Files\Systems\Emb Work 5.4\common\bin\run", r"C:\Program Files\Systems\Emb Work 5.4\arm\bin\mpr.dll", ...], cwd=r"...")
# ... notice how you don't need to quote the elements containing spaces

According to the documentation:

On Windows: the Popen class uses CreateProcess() to execute the child program, which operates on strings. If args is a sequence, it will be converted to a string using the list2cmdline() method. Please note that not all MS Windows applications interpret the command line the same way: list2cmdline() is designed for applications using the same rules as the MS C runtime.

So if you use a sequence, it will be converted to a string. I would probably try with a sequence first, since then you won't have to quote all the elements that contain spaces (list2cmdline() does that for you).

For troubleshooting, I recommend you pass your sequence to subprocess.list2cmdline() and check the output.

Edit:

Here's what I'd do if I were you:

a) Create a simple Python script (testparams.py) like this:

import subprocess
params = [r"C:\Program Files\Systems\Emb Work 5.4\common\bin\run.exe", ...]
print subprocess.list2cmdline(params)

b) Run the script from the command line (python testparams.py), copy and paste the output to another command line, press enter and see what happens.

c) If it does not work, edit the python file and repeat until it works.

subprocess.run simple scenario fails

The quoting is wrong, you should have '-f', '1', etc

How to handle subprocess.run() error/exception

There is no way running cd in a subprocess is useful. The subprocess will change its own directory and then immediately exit, leaving no observable change in the parent process or anywhere else.

For the same reason, there is no binary command named cd on most systems; the cd command is a shell built-in.

Generally, if you run subprocess.run() without the check=True keyword argument, any error within the subprocess will simply be ignored. So if /bin/cd or a similar command existed, you could run

# purely theoretical, and utterly useless
subprocess.run(['cd', UserInput])

and simply not know whether it did anything or not.

If you do supply check=True, the exception you need to trap is CalledProcessError:

try:
    # pointless code as such; see explanation above
    subprocess.run(['cd', UserInput], check=True)
except subprocess.CalledProcessError:
    print('Directory name %s misspelled, or you lack the permissions' % UserInput)

But even more fundamentally, allowing users to prod the system by running arbitrary unchecked input in a subprocess is a horrible idea. (Allowing users to run arbitrary shell script with shell=True is a monumentally, catastrophically horrible idea, so let's not even go there. Maybe see Actual meaning of shell=True in subprocess)

A somewhat more secure approach is to run the subprocess with a cwd= keyword argument.

# also vaguely pointless
subprocess.run(['true'], cwd=UserInput)

In this case, you can expect a regular FileNotFoundError if the directory does not exist, or a PermissionError if you lack the privileges.

You should probably still add check=True and be prepared to handle any resulting exception, unless you specifically don't care whether the subprocess succeeded. (There are actually cases where this makes sense, like when you grep for something but are fine with it not finding any matches, which raises an error if you use check=True.)

Perhaps see also Running Bash commands in Python

Running Python script through subprocess fails

Executing ./util.py in a terminal means "Look in the current working directory for a file named util.py and run it." The working directory is the directory from where you run the command. This means that your python script cannot see util.py if you run it from a different directory.

If you are sure that file1.py and util.py always lie in the same directory, you could use __file__ and os.path.dirname to prefix it with the directory of file1.py:

file1.py:

#!/usr/bin/env python
import os
import subprocess

current_dir = os.path.dirname(__file__)
filename = os.path.join(current_dir, "util.py")
out = subprocess.Popen([filename], shell=True)
print("done")

How to overcome Type error with subprocess call python3

The issue is that you are trying to split the stdout.PIPE from the subprocess—a bytes object—using a regular string.

You can either convert the output to a regular string before splitting it (probably what you want):

output = str(rootfs.communicate()[0]).strip().split('\n')

or you can split it using a bytes object:

output = rootfs.communicate()[0].strip().split(b'\n')

Note: you will need to do the same for varfs and tmPfs.

subprocess.run() fails at the second iteration

I solved it myself using the plumbum module.

My codes remained the same, they are all good.

However, I modified the subprocess.run() command, or many of its variants I tried, to be smi = local.cmd.mpirun, then smi("-n", "1", "../smilei", particular_namelist_name) and I can run this at every iteration of the loop!

subprocess.run failure vs subprocess.check_call success

The correct execution is
subprocess.run(['start', 'npm', 'run', 'features:chrome:server'], check=True)

subprocess.check_call('start npm run features:chrome:server', shell=True) works because you invoked the shell. Further information you could find here.

subprocess.run("start npm run features:chrome:server", check=True, shell=True) should also work.

Python script fails execution on subprocess.run() call only when called from context menu

I managed to "solve" the problem by the round about hackish way of creating a batch file that calls the python script command, but this does seem a bit of a hack, and I would think there would be a better way.

Running a subprocess fails when not using pytest no-capture

$ pip install ptyprocess==0.6.0

# test.py

import unittest

from ptyprocess import PtyProcess

class TestCase(unittest.TestCase):

    def test(self):
        process = PtyProcess.spawn(
            ["sh", "./has-stdin.sh"]
        )
        process.wait()
        output = process.read().decode("utf-8")
        if process.exitstatus:
            raise Exception(output.strip("\r\n"))
        assert output.strip("\r\n") == 'no'

---

✅ Runs successfully with -s:

$ pytest test.py -s
platform linux -- Python 3.7.7, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /Users/maikel/docker/library/postgresql
collected 1 item

test.py .

=========================== 1 passed in 0.60s ===========================

✅ Runs successfully without -s:

$ pytest test.py
platform linux -- Python 3.7.7, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /Users/maikel/docker/library/postgresql
collected 1 item

test.py .

=========================== 1 passed in 0.60s ===========================

---

Caveats

• It runs quite a bit slower than subprocess (0.60s vs 0.02s in this example)
• There is no distinction between stdout and stderr

---

The situation is explained by this diagram:

Source: https://github.com/pexpect/ptyprocess/blob/3931cd4/docs/index.rst

---

Related Topics