Popen() Writes Output of Command Executed to Cout

popen() writes output of command executed to cout

Popen doesn't capture stderr only stdout. Redirecting stderr to stdout fixes the issue.

#include <string>
#include <iostream>
#include <cstdlib>
#include <cstdio>
#include <array>

int main()
{
std::string command("ls afskfksakfafkas 2>&1");

std::array<char, 128> buffer;
std::string result;

std::cout << "Opening reading pipe" << std::endl;
FILE* pipe = popen(command.c_str(), "r");
if (!pipe)
{
std::cerr << "Couldn't start command." << std::endl;
return 0;
}
while (fgets(buffer.data(), 128, pipe) != NULL) {
std::cout << "Reading..." << std::endl;
result += buffer.data();
}
auto returnCode = pclose(pipe);

std::cout << result << std::endl;
std::cout << returnCode << std::endl;

return 0;
}

Is output read from popen()ed FILE* complete before pclose()?

TL;DR executive summary: how do we know the content retrieved with the fread() is complete? — we've got an EOF.

You get an EOF when the child process closes its end of the pipe. This can happen when it calls close explicitly or exits. Nothing can come out of your end of the pipe after that. After getting an EOF you don't know whether the process has terminated, but you do know for sure that it will never write anything to the pipe.

By calling pclose you close your end of the pipe and wait for termination of the child. When pclose returns, you know that the child has terminated.

If you call pclose without getting an EOF, and the child tries to write stuff to its end of the pipe, it will fail (in fact it wil get a SIGPIPE and probably die).

There is absolutely no room for any chicken-and-egg situation here.

Constantly print Subprocess output while process is running

You can use iter to process lines as soon as the command outputs them: lines = iter(fd.readline, ""). Here's a full example showing a typical use case (thanks to @jfs for helping out):

from __future__ import print_function # Only Python 2.x
import subprocess

def execute(cmd):
popen = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True)
for stdout_line in iter(popen.stdout.readline, ""):
yield stdout_line
popen.stdout.close()
return_code = popen.wait()
if return_code:
raise subprocess.CalledProcessError(return_code, cmd)

# Example
for path in execute(["locate", "a"]):
print(path, end="")

Capture output from subprocess.Popen commands that execute further processes

Apologies, probably as expected, this was user error. There is an if/else that determines whether the output goes to a logfile or terminal that wasn't being triggered correctly. So the example code I originally posted wasn't actually being touched. I should have added a bunch of print statements to see this fact earlier... It is working as expected now.

How do I execute a command and get the output of the command within C++ using POSIX?

#include <cstdio>
#include <iostream>
#include <memory>
#include <stdexcept>
#include <string>
#include <array>

std::string exec(const char* cmd) {
std::array<char, 128> buffer;
std::string result;
std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
if (!pipe) {
throw std::runtime_error("popen() failed!");
}
while (fgets(buffer.data(), buffer.size(), pipe.get()) != nullptr) {
result += buffer.data();
}
return result;
}

Pre-C++11 version:

#include <iostream>
#include <stdexcept>
#include <stdio.h>
#include <string>

std::string exec(const char* cmd) {
char buffer[128];
std::string result = "";
FILE* pipe = popen(cmd, "r");
if (!pipe) throw std::runtime_error("popen() failed!");
try {
while (fgets(buffer, sizeof buffer, pipe) != NULL) {
result += buffer;
}
} catch (...) {
pclose(pipe);
throw;
}
pclose(pipe);
return result;
}

Replace popen and pclose with _popen and _pclose for Windows.

Store output of subprocess.Popen call in a string

In Python 2.7 or Python 3

Instead of making a Popen object directly, you can use the subprocess.check_output() function to store output of a command in a string:

from subprocess import check_output
out = check_output(["ntpq", "-p"])

In Python 2.4-2.6

Use the communicate method.

import subprocess
p = subprocess.Popen(["ntpq", "-p"], stdout=subprocess.PIPE)
out, err = p.communicate()

out is what you want.

Important note about the other answers

Note how I passed in the command. The "ntpq -p" example brings up another matter. Since Popen does not invoke the shell, you would use a list of the command and options—["ntpq", "-p"].

Getting realtime output using subprocess

I tried this, and for some reason while the code

for line in p.stdout:
...

buffers aggressively, the variant

while True:
line = p.stdout.readline()
if not line: break
...

does not. Apparently this is a known bug: http://bugs.python.org/issue3907 (The issue is now "Closed" as of Aug 29, 2018)

What interval does popen go by when running commands and how to capture output line by line?

If you use the GNU libc then you could use getline() instead of fgets. See here for a more detailed explanation.

Otherwise, use whatever facility your system provides to read a line from a FILE*. If you really can't find such a facility (which would be very surprising), then at worst you can always use this approach. fgets is also a good solution provided you account for the buffer size limit and do some more buffering by yourself if required (check the doc, everything is explained).

The thing you gotta remember is that popen just returns a FILE* so anything that will work on a standard file will also work in your case. Examples are plenty on the web.


Now for the "interval" part of your question, I'm afraid it makes little sense. To begin with, it depends on what interval does the external command print new data to the standard output. In any case, you'll be able to read it in your program as soon as the external command writes it.



Related Topics



Leave a reply



Submit