Python subprocess and user interaction
Check out the subprocess manual. You have options with subprocess
to be able to redirect the stdin
, stdout
, and stderr
of the process you're calling to your own.
from subprocess import Popen, PIPE, STDOUT
p = Popen(['grep', 'f'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
grep_stdout = p.communicate(input='one\ntwo\nthree\nfour\nfive\nsix\n')[0]
print grep_stdout
You can also interact with a process line by line. Given this as prog.py
:
import sys
print 'what is your name?'
sys.stdout.flush()
name = raw_input()
print 'your name is ' + name
sys.stdout.flush()
You can interact with it line by line via:
>>> from subprocess import Popen, PIPE, STDOUT
>>> p = Popen(['python', 'prog.py'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
>>> p.stdout.readline().rstrip()
'what is your name'
>>> p.communicate('mike')[0].rstrip()
'your name is mike'
EDIT: In python3, it needs to be 'mike'.encode()
.
Interact with python subprocess once waits for user input
Try this:
import subprocess
process = subprocess.Popen("program.exe get -n WiiVNC", stdin=subprocess.PIPE, shell=True)
process.stdin.write(b"y\n")
process.stdin.flush()
stdout, stderr = process.communicate()
How to get input from user and pass it to an interactive command line program triggered by subprocess call in python?
subprocess.call
just waits for the process to finish and gives return code, no way to interact with it. if you instead use subprocess.Popen
that gives you the ability to communicate with the subprocess while it is running via stdin
and stdout
import subprocess, sys
program = subprocess.Popen("python3",
# give us a pipes to coommunicate
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
data = input("input to subprocess")
[out,err] = program.communicate((data+"\n").encode())
print(out.decode())
print(err.decode(), file=sys.stderr)
Doing a bit of input then some output then more input can get messy though since there reading from stdout is blocking so determining when the output has stopped for more input is tricky.
Interactive input/output using Python
Two solutions for this issue on Linux:
First one is to use a file to write the output to, and read from it simultaneously:
from subprocess import Popen, PIPE
fw = open("tmpout", "wb")
fr = open("tmpout", "r")
p = Popen("./a.out", stdin = PIPE, stdout = fw, stderr = fw, bufsize = 1)
p.stdin.write("1\n")
out = fr.read()
p.stdin.write("5\n")
out = fr.read()
fw.close()
fr.close()
Second, as J.F. Sebastian offered, is to make p.stdout and p.stderr pipes non-blocking using fnctl module:
import os
import fcntl
from subprocess import Popen, PIPE
def setNonBlocking(fd):
"""
Set the file description of the given file descriptor to non-blocking.
"""
flags = fcntl.fcntl(fd, fcntl.F_GETFL)
flags = flags | os.O_NONBLOCK
fcntl.fcntl(fd, fcntl.F_SETFL, flags)
p = Popen("./a.out", stdin = PIPE, stdout = PIPE, stderr = PIPE, bufsize = 1)
setNonBlocking(p.stdout)
setNonBlocking(p.stderr)
p.stdin.write("1\n")
while True:
try:
out1 = p.stdout.read()
except IOError:
continue
else:
break
out1 = p.stdout.read()
p.stdin.write("5\n")
while True:
try:
out2 = p.stdout.read()
except IOError:
continue
else:
break
Python subprocess allow for user interaction
It looks like you want to use pexpect
:
import pexpect
child = pexpect.spawn('student_program')
while child.expect('Enter (\w+)\r\n', pexpect.EOF) == 0:
if child.match[1] == 'p':
child.sendline('3.14159')
To pass interactive control of the program to the user, use child.interact()
.
Running interactive program from within python
You need to keep interacting with your subprocess - at the moment once you pick the output from your subprocess you're pretty much done as you close its STDOUT
stream.
Here is the most rudimentary way to continue user input -> process output cycle:
import subprocess
import sys
import time
if __name__ == "__main__": # a guard from unintended usage
input_buffer = sys.stdin # a buffer to get the user input from
output_buffer = sys.stdout # a buffer to write rasa's output to
proc = subprocess.Popen(["path/to/rasa", "arg1", "arg2", "etc."], # start the process
stdin=subprocess.PIPE, # pipe its STDIN so we can write to it
stdout=output_buffer, # pipe directly to the output_buffer
universal_newlines=True)
while True: # run a main loop
time.sleep(0.5) # give some time for `rasa` to forward its STDOUT
print("Input: ", end="", file=output_buffer, flush=True) # print the input prompt
print(input_buffer.readline(), file=proc.stdin, flush=True) # forward the user input
You can replace input_buffer
with a buffer coming from your remote user(s) and output_buffer
with a buffer that forwards the data to your user(s) and you'll get essentially what you're looking for - the sub-process will be getting the input directly from the user (input_buffer
) and print its output to the user (output_buffer
).
If you need to perform other tasks while all this is running in the background, just run everything under the if __name__ == "__main__":
guard in a separate thread, and I'd suggest adding a try..except
block to pick up KeyboardInterrupt
and exit gracefully.
But... soon enough you'll notice that it doesn't exactly work properly all the time - if it takes longer than half a second of wait for rasa
to print its STDOUT
and enter the wait for STDIN
stage, the outputs will start to mix. This problem is considerably more complex than you might expect. The main issue is that STDOUT
and STDIN
(and STDERR
) are separate buffers and you cannot know when a subprocess is actually expecting something on its STDIN
. This means that without a clear indication from the subprocess (like you have the \r\n[path]>
in Windows CMD prompt on its STDOUT
for example) you can only send data to the subprocesses STDIN
and hope it will be picked up.
Based on your screenshot, it doesn't really give a distinguishable STDIN
request prompt because the first prompt is ... :\n
and then it waits for STDIN
, but then once the command is sent it lists options without an indication of its end of STDOUT
stream (technically making the prompt just ...\n
but that would match any line preceding it as well). Maybe you can be clever and read the STDOUT
line by line, then on each new line measure how much time has passed since the sub-process wrote to it and once a threshold of inactivity is reached assume that rasa
expects input and prompt the user for it. Something like:
import subprocess
import sys
import threading
# we'll be using a separate thread and a timed event to request the user input
def timed_user_input(timer, wait, buffer_in, buffer_out, buffer_target):
while True: # user input loop
timer.wait(wait) # wait for the specified time...
if not timer.is_set(): # if the timer was not stopped/restarted...
print("Input: ", end="", file=buffer_out, flush=True) # print the input prompt
print(buffer_in.readline(), file=buffer_target, flush=True) # forward the input
timer.clear() # reset the 'timer' event
if __name__ == "__main__": # a guard from unintended usage
input_buffer = sys.stdin # a buffer to get the user input from
output_buffer = sys.stdout # a buffer to write rasa's output to
proc = subprocess.Popen(["path/to/rasa", "arg1", "arg2", "etc."], # start the process
stdin=subprocess.PIPE, # pipe its STDIN so we can write to it
stdout=subprocess.PIPE, # pipe its STDIN so we can process it
universal_newlines=True)
# lets build a timer which will fire off if we don't reset it
timer = threading.Event() # a simple Event timer
input_thread = threading.Thread(target=timed_user_input,
args=(timer, # pass the timer
1.0, # prompt after one second
input_buffer, output_buffer, proc.stdin))
input_thread.daemon = True # no need to keep the input thread blocking...
input_thread.start() # start the timer thread
# now we'll read the `rasa` STDOUT line by line, forward it to output_buffer and reset
# the timer each time a new line is encountered
for line in proc.stdout:
output_buffer.write(line) # forward the STDOUT line
output_buffer.flush() # flush the output buffer
timer.set() # reset the timer
You can use a similar technique to check for more complex 'expected user input' patterns. There is a whole module called pexpect
designed to deal with this type of tasks and I wholeheartedly recommend it if you're willing to give up some flexibility.
Now... all this being said, you are aware that Rasa
is built in Python, installs as a Python module and has a Python API, right? Since you're already using Python why would you call it as a subprocess and deal with all this STDOUT/STDIN
shenanigans when you can directly run it from your Python code? Just import it and interact with it directly, they even have a very simple example that does exactly what you're trying to do: Rasa Core with minimal Python.
Related Topics
How to Create a Decorator That Can Be Used Either with or Without Parameters
Beautiful Soup 4 Find_All Don't Find Links That Beautiful Soup 3 Finds
Check If a Given Key Already Exists in a Dictionary and Increment It
JSON.Loads Allows Duplicate Keys in a Dictionary, Overwriting the First Value
Return Multiple Columns from Pandas Apply()
Inserting the Same Value Multiple Times When Formatting a String
Pygame: Problems with Shooting in Space Invaders
How to Convert an H:Mm:Ss Time String to Seconds in Python
String Similarity Metrics in Python
Pygame Window Not Responding After Few Seconds
How to Improve My Paw Detection
How to Implement a Minimal Server for Ajax in Python
How to Find Which Columns Contain Any Nan Value in Pandas Dataframe
Matplotlib: Draw Grid Lines Behind Other Graph Elements
Python Pandas - Missing Required Dependencies ['Numpy'] 1