Get Live Output from Process

Get Live output from Process

Take a look at this page, it looks this is the solution for you: http://msdn.microsoft.com/en-us/library/system.diagnostics.process.beginoutputreadline.aspx and http://msdn.microsoft.com/en-us/library/system.diagnostics.process.standardoutput.aspx

[Edit]
This is a working example:

        Process p = new Process();
        p.StartInfo.RedirectStandardError = true;
        p.StartInfo.RedirectStandardOutput = true;
        p.StartInfo.UseShellExecute = false;
        p.StartInfo.CreateNoWindow = true;
        p.StartInfo.FileName = @"C:\Program Files (x86)\gnuwin32\bin\ls.exe";
        p.StartInfo.Arguments = "-R C:\\";

        p.OutputDataReceived += new DataReceivedEventHandler((s, e) => 
        { 
            Console.WriteLine(e.Data); 
        });
        p.ErrorDataReceived += new DataReceivedEventHandler((s, e) =>
        {
            Console.WriteLine(e.Data);
        });

        p.Start();
        p.BeginOutputReadLine();
        p.BeginErrorReadLine();

Btw, ls -R C:\ lists all files from the root of C: recursively. These are a lot of files, and I'm sure it isn't done when the first results show up in the screen.
There is a possibility 7zip holds the output before showing it. I'm not sure what params you give to the proces.

How to get an live output from runtime execution in java?

Your process.waitFor() call is a blocking call, and only unblocks when the process ends, preventing your streams from working, since the streams will be closed when the process has ended.

Read from the stream in a separate thread that you call the .waitFor() in, or read from the stream before calling .waitFor()

Runtime runtime = Runtime.getRuntime();
Process process = runtime.exec("cmd.exe /c kotlinc -script " + script.getAbsolutePath());
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line = "";
while ((line = bufferedReader.readLine()) != null){
    System.out.println(line);
}
int exitValue = process.waitFor();

Incidentally, I would use a ProcessBuilder to get the Process, and not Runtime.getRuntime()

How to get live output with subprocess in Python

There are three layers of buffering here, and you need to limit all three of them to guarantee you get live data:

1. Use the stdbuf command (on Linux) to wrap the subprocess execution (e.g. run ['stdbuf', '-oL'] + cmd instead of just cmd), or (if you have the ability to do so) alter the program itself to either explicitly change the buffering on stdout (e.g. using setvbuf for C/C++ code to switch stdout globally to line-buffered mode, rather than the default block buffering it uses when outputting to a non-tty) or to insert flush statements after critical output (e.g. fflush(stdout); for C/C++, fileobj.flush() for Python, etc.) the buffering of the program to line-oriented mode (or add fflushs); without that, everything is stuck in user-mode buffers of the sub-process.

2. Add bufsize=0 to the Popen arguments (probably not needed since you don't send anything to stdin, but harmless) so it unbuffers all piped handles. If the Popen is in text=True mode, switch to bufsize=1 (which is line-buffered, rather than unbuffered).

3. Add flush=True to the print arguments (if you're connected to a terminal, the line-buffering will flush it for you, so it's only if stdout is piped to a file that this will matter), or explicitly call sys.stdout.flush().

Between the three of these, you should be able to guarantee no data is stuck waiting in user-mode buffers; if at least one line has been output by the sub-process, it will reach you immediately, and any output triggered by it will also appear immediately. Item #1 is the hardest in most cases (when you can't use stdbuf, or the process reconfigures its own buffering internally and undoes the effect of stdbuf, and you can't modify the process executable to fix it); you have complete control over #2 and #3, but #1 may be outside your control.

Read live output of process in java

Using https://github.com/zeroturnaround/zt-exec

new ProcessExecutor().command("python", "script.py")
    .redirectOutput(new LogOutputStream() {
        @Override
        protected void processLine(String line) {
            ...
        }
    })
    .execute();

live output from subprocess command

TLDR for Python 3:

import subprocess
import sys

with open("test.log", "wb") as f:
    process = subprocess.Popen(your_command, stdout=subprocess.PIPE)
    for c in iter(lambda: process.stdout.read(1), b""):
        sys.stdout.buffer.write(c)
        f.buffer.write(c)

---

You have two ways of doing this, either by creating an iterator from the read or readline functions and do:

import subprocess
import sys

# replace "w" with "wb" for Python 3
with open("test.log", "w") as f:
    process = subprocess.Popen(your_command, stdout=subprocess.PIPE)
    # replace "" with b'' for Python 3
    for c in iter(lambda: process.stdout.read(1), ""):
        sys.stdout.write(c)
        f.write(c)

or

import subprocess
import sys

# replace "w" with "wb" for Python 3
with open("test.log", "w") as f:
    process = subprocess.Popen(your_command, stdout=subprocess.PIPE)
    # replace "" with b"" for Python 3
    for line in iter(process.stdout.readline, ""):
        sys.stdout.write(line)
        f.write(line)

Or you can create a reader and a writer file. Pass the writer to the Popen and read from the reader

import io
import time
import subprocess
import sys

filename = "test.log"
with io.open(filename, "wb") as writer, io.open(filename, "rb", 1) as reader:
    process = subprocess.Popen(command, stdout=writer)
    while process.poll() is None:
        sys.stdout.write(reader.read())
        time.sleep(0.5)
    # Read the remaining
    sys.stdout.write(reader.read())

This way you will have the data written in the test.log as well as on the standard output.

The only advantage of the file approach is that your code doesn't block. So you can do whatever you want in the meantime and read whenever you want from the reader in a non-blocking way. When you use PIPE, read and readline functions will block until either one character is written to the pipe or a line is written to the pipe respectively.

Live-output / stream from Python subprocess

Had some problems referencing the selected answer for streaming output from a test runner. The following worked better for me:

import subprocess
from time import sleep

def stream_process(process):
    go = process.poll() is None
    for line in process.stdout:
        print(line)
    return go

process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while stream_process(process):
    sleep(0.1)

---

Related Topics