Run Interactive Bash With Popen and a Dedicated Tty Python

Run interactive Bash with popen and a dedicated TTY Python

This is the solution that worked for me at the end (as suggested by qarma) :

libc = ctypes.CDLL('libc.so.6')

master, slave = pty.openpty()
p = subprocess.Popen(["/bin/bash", "-i"], preexec_fn=libc.setsid, stdin=slave, stdout=slave, stderr=slave)
os.close(slave)

... do stuff here ...

x = os.read(master, 1026)
print x

Run interactive Bash in dumb terminal using Python subprocess.Popen and pty

That's exactly the side effects of not putting the tty in raw mode. Usually a program (like expect) which handles pty would put the outer tty in raw mode.

  • Your Python script's tty (or pty) echos what you input and the new pty echos for the 2nd time. You can disable ECHO on the new pty. For example:

    $ python3 using-pty.py
    bash-5.1$ echo hello
    echo hello
    hello
    bash-5.1$ stty -echo
    stty -echo
    bash-5.1$ echo hello # <-- no double echo any more
    hello
    bash-5.1$ exit
    exit
  • Your Python script's tty is not in raw mode so when you press ctrl-d Python would not get the literal ctrl-d ('\004'). Instead, Python would reach EOF and read() returns an empty string. So to make the spawned shell exit you can

    user_input = os.read(sys.stdin.fileno(), 10240)
    if not user_input:
    # explicitly send ctrl-d to the spawned process
    os.write(master_fd, b'\04')
    else:
    os.write(master_fd, user_input)
  • Similarly, the Python's tty is not in raw mode so when you press ctrl-c, it'll not get the literal ctrl-c ('\003'). Instead it's killed. As a workaround you can catch SIGINT.

    def handle_sigint(signum, stack):
    global master_fd
    # send ctrl-c
    os.write(master_fd, b'\03')
    signal.signal(signal.SIGINT, handle_sigint)

Interacting with bash from python

Try with this example:

import subprocess

proc = subprocess.Popen(['/bin/bash'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
stdout = proc.communicate('ls -lash')

print stdout

You have to read more about stdin, stdout and stderr. This looks like good lecture: http://www.doughellmann.com/PyMOTW/subprocess/

EDIT:

Another example:

>>> process = subprocess.Popen(['/bin/bash'], shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
>>> process.stdin.write('echo it works!\n')
>>> process.stdout.readline()
'it works!\n'
>>> process.stdin.write('date\n')
>>> process.stdout.readline()
'wto, 13 mar 2012, 17:25:35 CET\n'
>>>

Filter out command that needs a terminal in Python subprocess module

What you expect is a function that receives command as input, and returns meaningful output by running the command.

Since the command is arbitrary, requirement for tty is just one of many bad cases may happen (other includes running a infinite loop), your function should only concern about its running period, in other words, a command is “bad” or not should be determined by if it ends in a limited time or not, and since subprocess is asynchronous by nature, you can just run the command and handle it in a higher vision.

Demo code to play, you can change the cmd value to see how it performs differently:

#!/usr/bin/env python
# coding: utf-8

import time
import subprocess
from subprocess import PIPE


#cmd = ['ls']
#cmd = ['sleep', '3']
cmd = ['vim', '-u', '/dev/null']

print 'call cmd'
p = subprocess.Popen(cmd, shell=True,
stdin=PIPE, stderr=PIPE, stdout=PIPE)
print 'called', p

time_limit = 2
timer = 0
time_gap = 0.2

ended = False
while True:
time.sleep(time_gap)

returncode = p.poll()
print 'process status', returncode

timer += time_gap
if timer >= time_limit:
print 'timeout, kill process'
p.kill()
break

if returncode is not None:
ended = True
break

if ended:
print 'process ended by', returncode

print 'read'
out, err = p.communicate()
print 'out', repr(out)
print 'error', repr(err)
else:
print 'process failed'

Three points are notable in the above code:

  1. We use Popen instead of check_output to run the command, unlike check_output which will wait for the process to end, Popen returns immediately, thus we can do further things to control the process.

  2. We implement a timer to check for the process's status, if it runs for too long, we killed it manually because we think a process is not meaningful if it could not end in a limited time. In this way your original problem will be solved, as vim will never end and it will definitely being killed as an “unmeaningful” command.

  3. After the timer helps us filter out bad commands, we can get stdout and stderr of the command by calling communicate method of the Popen object, after that its your choice to determine what to return to the user.

Conclusion

tty simulation is not needed, we should run the subprocess asynchronously, then control it by a timer to determine whether it should be killed or not, for those ended normally, its safe and easy to get the output.

Popen waiting for child process even when the immediate child has terminated

You could provide start_new_session analog for the C subprocess:

#!/usr/bin/env python
import os
import sys
import platform
from subprocess import Popen, PIPE

# set system/version dependent "start_new_session" analogs
kwargs = {}
if platform.system() == 'Windows':
# from msdn [1]
CREATE_NEW_PROCESS_GROUP = 0x00000200 # note: could get it from subprocess
DETACHED_PROCESS = 0x00000008 # 0x8 | 0x200 == 0x208
kwargs.update(creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP)
elif sys.version_info < (3, 2): # assume posix
kwargs.update(preexec_fn=os.setsid)
else: # Python 3.2+ and Unix
kwargs.update(start_new_session=True)

p = Popen(["C"], stdin=PIPE, stdout=PIPE, stderr=PIPE, **kwargs)
assert not p.poll()

[1]: Process Creation Flags for CreateProcess()



Related Topics



Leave a reply



Submit