Run interactive Bash with popen and a dedicated TTY Python
This is the solution that worked for me at the end (as suggested by qarma) :
libc = ctypes.CDLL('libc.so.6')
master, slave = pty.openpty()
p = subprocess.Popen(["/bin/bash", "-i"], preexec_fn=libc.setsid, stdin=slave, stdout=slave, stderr=slave)
os.close(slave)
... do stuff here ...
x = os.read(master, 1026)
print x
Run interactive Bash in dumb terminal using Python subprocess.Popen and pty
That's exactly the side effects of not putting the tty in raw mode. Usually a program (like expect) which handles pty would put the outer tty in raw mode.
Your Python script's tty (or pty) echos what you input and the new pty echos for the 2nd time. You can disable ECHO on the new pty. For example:
$ python3 using-pty.py
bash-5.1$ echo hello
echo hello
hello
bash-5.1$ stty -echo
stty -echo
bash-5.1$ echo hello # <-- no double echo any more
hello
bash-5.1$ exit
exitYour Python script's tty is not in raw mode so when you press ctrl-d Python would not get the literal ctrl-d (
'\004'
). Instead, Python would reach EOF andread()
returns an empty string. So to make the spawned shell exit you canuser_input = os.read(sys.stdin.fileno(), 10240)
if not user_input:
# explicitly send ctrl-d to the spawned process
os.write(master_fd, b'\04')
else:
os.write(master_fd, user_input)Similarly, the Python's tty is not in raw mode so when you press ctrl-c, it'll not get the literal ctrl-c (
'\003'
). Instead it's killed. As a workaround you can catchSIGINT
.def handle_sigint(signum, stack):
global master_fd
# send ctrl-c
os.write(master_fd, b'\03')
signal.signal(signal.SIGINT, handle_sigint)
Interacting with bash from python
Try with this example:
import subprocess
proc = subprocess.Popen(['/bin/bash'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
stdout = proc.communicate('ls -lash')
print stdout
You have to read more about stdin, stdout and stderr. This looks like good lecture: http://www.doughellmann.com/PyMOTW/subprocess/
EDIT:
Another example:
>>> process = subprocess.Popen(['/bin/bash'], shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
>>> process.stdin.write('echo it works!\n')
>>> process.stdout.readline()
'it works!\n'
>>> process.stdin.write('date\n')
>>> process.stdout.readline()
'wto, 13 mar 2012, 17:25:35 CET\n'
>>>
Filter out command that needs a terminal in Python subprocess module
What you expect is a function that receives command as input, and returns meaningful output by running the command.
Since the command is arbitrary, requirement for tty is just one of many bad cases may happen (other includes running a infinite loop), your function should only concern about its running period, in other words, a command is “bad” or not should be determined by if it ends in a limited time or not, and since subprocess
is asynchronous by nature, you can just run the command and handle it in a higher vision.
Demo code to play, you can change the cmd
value to see how it performs differently:
#!/usr/bin/env python
# coding: utf-8
import time
import subprocess
from subprocess import PIPE
#cmd = ['ls']
#cmd = ['sleep', '3']
cmd = ['vim', '-u', '/dev/null']
print 'call cmd'
p = subprocess.Popen(cmd, shell=True,
stdin=PIPE, stderr=PIPE, stdout=PIPE)
print 'called', p
time_limit = 2
timer = 0
time_gap = 0.2
ended = False
while True:
time.sleep(time_gap)
returncode = p.poll()
print 'process status', returncode
timer += time_gap
if timer >= time_limit:
print 'timeout, kill process'
p.kill()
break
if returncode is not None:
ended = True
break
if ended:
print 'process ended by', returncode
print 'read'
out, err = p.communicate()
print 'out', repr(out)
print 'error', repr(err)
else:
print 'process failed'
Three points are notable in the above code:
We use
Popen
instead ofcheck_output
to run the command, unlikecheck_output
which will wait for the process to end,Popen
returns immediately, thus we can do further things to control the process.We implement a timer to check for the process's status, if it runs for too long, we killed it manually because we think a process is not meaningful if it could not end in a limited time. In this way your original problem will be solved, as
vim
will never end and it will definitely being killed as an “unmeaningful” command.After the timer helps us filter out bad commands, we can get stdout and stderr of the command by calling
communicate
method of thePopen
object, after that its your choice to determine what to return to the user.
Conclusion
tty simulation is not needed, we should run the subprocess asynchronously, then control it by a timer to determine whether it should be killed or not, for those ended normally, its safe and easy to get the output.
Popen waiting for child process even when the immediate child has terminated
You could provide start_new_session
analog for the C
subprocess:
#!/usr/bin/env python
import os
import sys
import platform
from subprocess import Popen, PIPE
# set system/version dependent "start_new_session" analogs
kwargs = {}
if platform.system() == 'Windows':
# from msdn [1]
CREATE_NEW_PROCESS_GROUP = 0x00000200 # note: could get it from subprocess
DETACHED_PROCESS = 0x00000008 # 0x8 | 0x200 == 0x208
kwargs.update(creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP)
elif sys.version_info < (3, 2): # assume posix
kwargs.update(preexec_fn=os.setsid)
else: # Python 3.2+ and Unix
kwargs.update(start_new_session=True)
p = Popen(["C"], stdin=PIPE, stdout=PIPE, stderr=PIPE, **kwargs)
assert not p.poll()
[1]: Process Creation Flags for CreateProcess()
Related Topics
How to Get Monotonic Time Durations in Python
Return Value of X = Os.System(..)
How to Execute a Program or Call a System Command
How Is Returning the Output of a Function Different from Printing It
Difference Between Modes A, A+, W, W+, and R+ in Built-In Open Function
Split Strings into Words With Multiple Word Boundary Delimiters
What's the Canonical Way to Check For Type in Python
Tkinter Understanding Mainloop
Modifying List While Iterating
Uninstall Python Built from Source
After Anaconda Installation, Conda Command Fails With "Importerror: No Module Named Conda.Cli"
Tkinter: Attributeerror: Nonetype Object Has No Attribute ≪Attribute Name≫
How to Make Function Decorators and Chain Them Together
How to Sort a Dictionary by Key
How to Flush the Output of the Print Function
Why Are Multiple Instances of Tk Discouraged
Difference Between Shallow Copy, Deepcopy and Normal Assignment Operation