Detecting when a child process is waiting for input
Have you noticed that raw_input writes the prompt string into stderr if stdout is terminal (isatty); if stdout is not a terminal, then the prompt too is written to stdout, but stdout will be in fully buffered mode.
With stdout on a tty
write(1, "Hello.\n", 7) = 7
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
write(2, "Type your name: ", 16) = 16
fstat(0, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 3), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb114059000
read(0, "abc\n", 1024) = 4
write(1, "Nice to meet you, abc!\n", 23) = 23
With stdout not on a tty
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff8d9d3410) = -1 ENOTTY (Inappropriate ioctl for device)
# oops, python noticed that stdout is NOTTY.
fstat(0, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 3), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f29895f0000
read(0, "abc\n", 1024) = 4
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f29891c4bd0}, {0x451f62, [], SA_RESTORER, 0x7f29891c4bd0}, 8) = 0
write(1, "Hello.\nType your name: Nice to m"..., 46) = 46
# squeeze all output at the same time into stdout... pfft.
Thus all writes are squeezed into stdout all at the same time; and what is worse, after the input is read.
The real solution is thus to use the pty. However you are doing it wrong. For the pty to work, you must use the pty.fork() command, not subprocess. (This will be very tricky). I have some working code that goes like this:
import os
import tty
import pty
program = "python"
# command name in argv[0]
argv = [ "python", "foo.py" ]
pid, master_fd = pty.fork()
# we are in the child process
if pid == pty.CHILD:
# execute the program
os.execlp(program, *argv)
# else we are still in the parent, and pty.fork returned the pid of
# the child. Now you can read, write in master_fd, or use select:
# rfds, wfds, xfds = select.select([master_fd], [], [], timeout)
Notice that depending on the terminal mode set by the child program there might be different kinds of linefeeds coming out, etc.
Now about the "waiting for input" problem, that cannot be really helped as one can always write to a pseudoterminal; the characters will be put to wait in the buffer. Likewise, a pipe always allows one to write up to 4K or 32K or some other implementation defined amount, before blocking. One ugly way is to strace the program and notice whenever it enters the read system call, with fd = 0; the other would be to make a C module with a replacement "read()" system call and link it in before glibc for the dynamic linker (fails if the executable is statically linked or uses system calls directly with assembler...), and then would signal python whenever the read(0, ...) system call is executed. All in all, probably not worth the trouble exactly.
Detecting when a child process is waiting for stdin
class STDINHandle:
def __init__(self, read_handle, write_handle):
self.handled_write = False
self.working = Lock()
self.write_handle = write_handle
self.read_handle = read_handle
def check_child_reading(self):
with self.working:
# Reset the flag
self.handled_write = True
# Write a character that cmd will ignore
self.write_handle.write("\r")
thread = Thread(target=self.try_read)
thread.start()
sleep(0.1)
# We need to stop the other thread by giving it data to read
if self.handled_write:
# Writing only 1 "\r" fails for some reason.
# For good measure we write 10 "\r"s
self.write_handle.write("\r"*10)
return True
return False
def try_read(self):
data = self.read_handle.read(1)
self.handled_write = False
def write(self, text):
self.write_handle.write(text)
I did a bit of testing and I think cmd ignores "\r" characters. I couldn't find a case where cmd will interpret it as an actual character (like what happened when I did "\b"). Sending a "\r" character and testing if it stays in the pipe. If it does stay in the pipe that means that the child hasn't processed it. If we can't read it from the pipe that means that the child has processed it. But we have a problem - we need to stop the read if we can't read from stdin otherwise it will mess with the next write to stdin. To do that we write more "\r"s to the pipe.
Note: I might have to change the timing on the sleep(0.1) line.
Node detect child_process waiting to read from stdin
I've played around with Node standard I/O and streams for a bit until I eventually arrived at a solution. You can find it here: https://github.com/TomasHubelbauer/node-stdio
The gist of it is that we need to create a Readable stream and pipe it to the process' standard input. Then we need to listen for the process' standard output and parse it, detect the chunks of interest (prompts to the user) and each time we get one of those, make our Readable output our "reaction" to the prompt to the process' standard input.
Start the process:
const cp = child_process.exec('node test');
Prepare a Readable and pipe it to the process' standard input:
new stream.Readable({ read }).pipe(cp.stdin);
Provide the read implementation which will be called when the process asks for input:
/** @this {stream.Readable} */
async function read(/** @type {number} */ size) {
this.push(await promise + '\n');
}
Here the promise is used to block until we have an answer to the question the process asked through its standard output. this.push will add the answer to an internal queue of the Readable and eventually it will be sent to the standard input of the process.
An example of how to parse the input for a program prompt, derive an answer from the question, wait for the answer to be provided and then send it to the process is in the linked repository.
Check if a child PID is waiting for an Scanf
The concepts of processes and IO are generally disjunct. There are no signals that are exchanged between child and parent process, except for unix (kill) signals sent to the parent pid, that propagate to the children.
Waitpid just waits for termination of a child pid, returning its status code.
If you want to exchange data between parent and child you need to create a pipe (see man -s 2 pipe) between both processes, see the manual page for an example.
If you want to use scanf (input from stdin) in the child, you need to bind the pipefd[0] to the stdin file descriptor 0 (or STDIN_FILENO).
Now you can use select or poll in the parent process to check if the child is ready to read data sent by the parent to pipefd[1].
If you use printf or some other stdio.h method to write to the child (via STDOUT_FILENO for example), the IO on the parent might block anyway, even if select or poll told that the child is ready to receive the data, if the child reads too slow or stops reading too early and the output buffer is full (which has a default size of 4096 bytes, I think).
A unistd.h write call might return a value
nw = write(pipefd[1], buffer, nbytes);
nw < nbytes if the child did not read that many (nbytes) bytes of input.
So be carefull with the tripping hazards when doing asynchronous communication. Check the CSP (Communicating Sequential Processes) method as a different and more stable approach to communication, that uses synchronous communication, when you understood the asynchronous methods.
How to detect when subprocess asks for input in Windows
if we not want let to child process process user input, but simply kill it in this case, solution can be next:
- start child process with redirected stdin to pipe.
- pipe server end we create in asynchronous mode and main set pipe
buffer to 0 size - before start child - write 1 byte to this pipe.
- because pipe buffer is 0 size - operation not complete, until another
side not read this byte - after we write this 1 byte and operation in progress (pending) -
start child process. - finally begin wait what complete first: write operation or child process ?
- if write complete first - this mean, that child process begin read
from stdin - so kill it at this point
one possible implementation on c++:
struct ReadWriteContext : public OVERLAPPED
{
enum OpType : char { e_write, e_read } _op;
BOOLEAN _bCompleted;
ReadWriteContext(OpType op) : _op(op), _bCompleted(false)
{
}
};
VOID WINAPI OnReadWrite(DWORD dwErrorCode, DWORD dwNumberOfBytesTransfered, OVERLAPPED* lpOverlapped)
{
static_cast<ReadWriteContext*>(lpOverlapped)->_bCompleted = TRUE;
DbgPrint("%u:%x %p\n", static_cast<ReadWriteContext*>(lpOverlapped)->_op, dwErrorCode, dwNumberOfBytesTransfered);
}
void nul(PCWSTR lpApplicationName)
{
ReadWriteContext wc(ReadWriteContext::e_write), rc(ReadWriteContext::e_read);
static const WCHAR pipename[] = L"\\\\?\\pipe\\{221B9EC9-85E6-4b64-9B70-249026EFAEAF}";
if (HANDLE hPipe = CreateNamedPipeW(pipename, PIPE_ACCESS_DUPLEX|FILE_FLAG_OVERLAPPED,
PIPE_TYPE_BYTE|PIPE_READMODE_BYTE|PIPE_WAIT, 1, 0, 0, 0, 0))
{
static SECURITY_ATTRIBUTES sa = { sizeof(sa), 0, TRUE };
PROCESS_INFORMATION pi;
STARTUPINFOW si = { sizeof(si)};
si.dwFlags = STARTF_USESTDHANDLES;
si.hStdInput = CreateFileW(pipename, FILE_GENERIC_READ|FILE_GENERIC_WRITE, 0, &sa, OPEN_EXISTING, 0, 0);
if (INVALID_HANDLE_VALUE != si.hStdInput)
{
char buf[256];
if (WriteFileEx(hPipe, "\n", 1, &wc, OnReadWrite))
{
si.hStdError = si.hStdOutput = si.hStdInput;
if (CreateProcessW(lpApplicationName, 0, 0, 0, TRUE, CREATE_NO_WINDOW, 0, 0, &si, &pi))
{
CloseHandle(pi.hThread);
BOOLEAN bQuit = true;
goto __read;
do
{
bQuit = true;
switch (WaitForSingleObjectEx(pi.hProcess, INFINITE, TRUE))
{
case WAIT_OBJECT_0:
DbgPrint("child terminated\n");
break;
case WAIT_IO_COMPLETION:
if (wc._bCompleted)
{
DbgPrint("child read from hStdInput!\n");
TerminateProcess(pi.hProcess, 0);
}
else if (rc._bCompleted)
{
__read:
rc._bCompleted = false;
if (ReadFileEx(hPipe, buf, sizeof(buf), &rc, OnReadWrite))
{
bQuit = false;
}
}
break;
default:
__debugbreak();
}
} while (!bQuit);
CloseHandle(pi.hProcess);
}
}
CloseHandle(si.hStdInput);
// let execute pending apc
SleepEx(0, TRUE);
}
CloseHandle(hPipe);
}
}
another variant of code - use event completion, instead apc. however this not affect final result. this variant of code give absolute the same result as first:
void nul(PCWSTR lpApplicationName)
{
OVERLAPPED ovw = {}, ovr = {};
if (ovr.hEvent = CreateEvent(0, 0, 0, 0))
{
if (ovw.hEvent = CreateEvent(0, 0, 0, 0))
{
static const WCHAR pipename[] = L"\\\\?\\pipe\\{221B9EC9-85E6-4b64-9B70-249026EFAEAF}";
if (HANDLE hPipe = CreateNamedPipeW(pipename, PIPE_ACCESS_DUPLEX|FILE_FLAG_OVERLAPPED,
PIPE_TYPE_BYTE|PIPE_READMODE_BYTE|PIPE_WAIT, 1, 0, 0, 0, 0))
{
static SECURITY_ATTRIBUTES sa = { sizeof(sa), 0, TRUE };
PROCESS_INFORMATION pi;
STARTUPINFOW si = { sizeof(si)};
si.dwFlags = STARTF_USESTDHANDLES;
si.hStdInput = CreateFileW(pipename, FILE_GENERIC_READ|FILE_GENERIC_WRITE, 0, &sa, OPEN_EXISTING, 0, 0);
if (INVALID_HANDLE_VALUE != si.hStdInput)
{
char buf[256];
if (!WriteFile(hPipe, "\n", 1, 0, &ovw) && GetLastError() == ERROR_IO_PENDING)
{
si.hStdError = si.hStdOutput = si.hStdInput;
if (CreateProcessW(lpApplicationName, 0, 0, 0, TRUE, CREATE_NO_WINDOW, 0, 0, &si, &pi))
{
CloseHandle(pi.hThread);
BOOLEAN bQuit = true;
HANDLE h[] = { ovr.hEvent, ovw.hEvent, pi.hProcess };
goto __read;
do
{
bQuit = true;
switch (WaitForMultipleObjects(3, h, false, INFINITE))
{
case WAIT_OBJECT_0 + 0://read completed
__read:
if (ReadFile(hPipe, buf, sizeof(buf), 0, &ovr) || GetLastError() == ERROR_IO_PENDING)
{
bQuit = false;
}
break;
case WAIT_OBJECT_0 + 1://write completed
DbgPrint("child read from hStdInput!\n");
TerminateProcess(pi.hProcess, 0);
break;
case WAIT_OBJECT_0 + 2://process terminated
DbgPrint("child terminated\n");
break;
default:
__debugbreak();
}
} while (!bQuit);
CloseHandle(pi.hProcess);
}
}
CloseHandle(si.hStdInput);
}
CloseHandle(hPipe);
// all pending operation completed here.
}
CloseHandle(ovw.hEvent);
}
CloseHandle(ovr.hEvent);
}
}
Is there any way to determine if a nodejs childprocess wants input or is just sending feedback?
So I was wondering if it is possible and it seems not. I posted an issue for node which ended up beeing closed: https://github.com/nodejs/node/issues/16214
I am asking for a way to determine if the current process is waiting for an input.
There isn't one. I think you have wrong expectations about pipe I/O
because that's simply not how it works.Talking about expectations, check out expect. There is probably a
node.js port if you look around.I'll close this out because it's not implementable as a feature, and
as a question nodejs/help is the more appropriate place.
So if anyone has the same problem as I had you can simply write multiple lines into stdin and use that as predefined values. Keep in mind that will eventually break the stream if any input is broken or wrong in feature updates:
// new Spawn.
let spawn = require('child_process');
// new msqlsec process.
let msqlsec = spawn('mysql_secure_installation', ['']);
// Arguments as Array.
let inputArgs = ['password', 'n', 'y', 'y', 'y', 'y'];
// Set correct encodings for logging.
msqlsec.stdin.setEncoding('utf-8');
msqlsec.stdout.setEncoding('utf-8');
msqlsec.stderr.setEncoding('utf-8');
// Use defined input and write line for each of them.
for (let a = 0; a < inputArgs.length; a++) {
msqlsec.stdin.write(inputArgs[a] + '\n');
}
// Event Standard Out.
msqlsec.stdout.on('data', (data) => {
console.log(data.toString('utf8'));
});
// Event Standard Error.
msqlsec.stderr.on('data', (err) => {
// Logerror.
console.log(err);
});
// When job is finished (with or without error) it ends up here.
msqlsec.on('close', (code) => {
// Check if there were errors.
if (code !== 0) console.log('Exited with code: ' + code.toString());
// close input to writeable stream.
msqlsec.stdin.end();
});
For the sake of completeness if someone wants to fill the user input manually you can simply start the given process like this:
// new msqlsec process.
let msqlsec = spawn('mysql_secure_installation', [''], { stdio: 'inherit', shell: true });
Determine if QProcess is waiting for user input
I'm not sure it makes sense to check if the child process is waiting for user input -- first, because I don't believe there is any realistic way to do that, and second, because it's not necessary -- any data you write() to the QProcess will be buffered and read by the child process if/when it does try to read from stdin. (The child process will by definition be executing asynchronously relative to your own process, so any approach that relies on knowing what the child "is currently doing" is inherently suspect, since what the child process is currently doing can and will change without notice from one instant to the next, before your process has time to react)
What you can do (if you want to) is read the child process's stdout and/or stderr streams, and react to the child process's output. (e.g. if you know the child process will at some point print enter your decision now -> to stdout, you could read in the stdout-data from the QProcess's StandardOutput channel and react appropriately when you see that string)
Related Topics
How to Access the Query String in Flask Routes
How to Use Python2.7 Pip Instead of Default Pip
How I Open Remote Server Folder Using Python
How to Make a Cross-Module Variable
How to Pass a List as a Command-Line Argument with Argparse
How to Install Python Packages on Windows
Python Setup.Py Develop VS Install
Multiple Python Versions on the Same MAChine
Converting from a String to Boolean in Python
Threading in a Pyqt Application: Use Qt Threads or Python Threads
How to Automatically Install Required Packages from a Python Script as Necessary
Pil Installation Fails Missing:Stdarg.H
Python and Regular Expression with Unicode
Parsing Boolean Values with Argparse
Python's Equivalent of && (Logical-And) in an If-Statement
Python: Ignore 'Incorrect Padding' Error When Base64 Decoding