How to Flush Stdout of a Running Process

Force flushing of output to a file while bash script is still running

bash itself will never actually write any output to your log file. Instead, the commands it invokes as part of the script will each individually write output and flush whenever they feel like it. So your question is really how to force the commands within the bash script to flush, and that depends on what they are.

How to force a running program to flush the contents of its I/O buffers to disk with external means?

Can I just add some clarity? Obviously months have passed, and I imagine your program isn't running any more ... but there's some confusion here about buffering which still isn't clear.

As soon as you use the stdio library and FILE *, you will by default have a fairly small (implementation dependent, but typically some KB) buffer inside your program which is accumulating what you write, and flushing it to the OS when it's full, (or on file close). When you kill your process, it is this buffer that gets lost.

If the data has been flushed to the OS, then it is kept in a unix file buffer until the OS decides to persist it to disk (usually fairly soon), or someone runs the sync command. If you kill the power on your computer, then this buffer gets lost as well. You probably don't care about this scenario, because you probably aren't planning to yank the power! But this is what @EJP was talking about (re Stuff that is in the OS buffer/disk cache' won't be lost): your problem is the stdio cache, not the OS.

In an ideal world, you'd write your app so it fflushed (or std::flush()) at key points. In your example, you'd say:

    if (i == 0) {//This case is interesting!
        fprintf(filept, "Hello world\n");
        fflush(filept);
    }

which would cause the stdio buffer to flush to the OS. I imagine your real writer is more complex, and in that situation I would try to make the fflush happen "often but not too often". Too rare, and you lose data when you kill the process, too often and you lose the performance benefits of buffering if you are writing a lot.

In your described situation, where the program is already running and can't be stopped and rewritten, then your only hope, as you say, is to stop it in a debugger. The details of what you need to do depend on the implementation of the std lib, but you can usually look inside the FILE *filept object and start following pointers, messy though. @ivan_pozdeev's comment about executing std::flush() or fflush() within the debugger is helpful.

Flush output of child process

One way to (try to) deal with buffering is to set up a terminal-like environment for the process, a pseudo-terminal (pty). That is not easy to do in general but IPC::Run has that capability ready for easy use.

Here is the driver, run for testing using at facility so that it has no controlling terminal (or run it via cron)

use warnings;
use strict;
use feature 'say';

use IPC::Run qw(run);
    
my @cmd = qw(./t_term.pl input arguments); 
    
run \@cmd, '>pty>', sub { say "out: @_" };

#run \@cmd, '>', sub { say "out: @_" }   # no pty

With >pty> it sets up a pseudo-terminal for STDOUT of the program in @cmd (with > it's a pipe); also see <pty< and see more about redirection.
The anonymous sub {} gets called every time there is output from the child, so one can process it as it goes. There are other related options.

The program that is called (t_term.pl) only tests for a terminal

use warnings;
use strict;
use feature 'say';

say "Is STDOUT filehandle attached to a terminal: ",
    ( (-t STDOUT) ? "yes" : "no" );
sleep 2;
say "bye from $$";

The -t STDOUT (see filetest operators) is a suitable way to check for a terminal in this example. For more/other ways see this post.

The output shows that the called program (t_term.pl) does see a terminal on its STDOUT, even when a driver runs without one (using at, or out of a crontab). If the >pty> is changed to the usual redirection > (a pipe) then there is no terminal.

Whether this solves the buffering problem is clearly up to that program, and to whether it is enough to fool it with a terminal.

Another way around the problem is using unbuffer when possible, as in mob's answer.

How can I flush the output of the print function?

In Python 3, print can take an optional flush argument:

print("Hello, World!", flush=True)

In Python 2, after calling print, do:

import sys
sys.stdout.flush()

By default, print prints to sys.stdout (see the documentation for more about file objects).

How can I read process output that has not been flushed?

The problems of my question in my original post are already very good explained
in the other answers.
Console applications use a function named isatty() to detect
if their stdout handler is connected to a pipe or a real console. In case of a pipe
all output is buffered and flushed in chunks except if you directly call fflush().
In case of a real console the output is unbuffered and gets directly printed to the
console output.

In Linux you can use openpty() to create a pseudoterminal and create your process in it. As a
result the process will think it runs in a real terminal and uses unbuffered output.
Windows seems not to have
such an option.

After a lot of digging through winapi documentation I found that this is not true. Actually you can create
your own console screen buffer and use it for stdout of your process that will be unbuffered then.

Sadly this is not a very comfortable solution because there are no event handler and we need to poll for new data.
Also at the moment I'm not sure how to handle scrolling when this screen buffer is full.
But even if there are still some problems
left I think I have created a very useful (and interesting) starting point for those of you who ever wanted to fetch unbuffered (and unflushed)
windows console process output.

#include <windows.h>
#include <stdio.h>

int main(int argc, char* argv[])
{
    char cmdline[] = "application.exe"; // process command
    HANDLE scrBuff;                     // our virtual screen buffer
    CONSOLE_SCREEN_BUFFER_INFO scrBuffInfo; // state of the screen buffer
                                            // like actual cursor position
    COORD scrBuffSize = {80, 25};       // size in chars of our screen buffer
    SECURITY_ATTRIBUTES sa;             // security attributes
    PROCESS_INFORMATION procInfo;       // process information
    STARTUPINFO startInfo;              // process start parameters
    DWORD procExitCode;                 // state of process (still alive)
    DWORD NumberOfCharsWritten;         // output of fill screen buffer func
    COORD pos = {0, 0};                 // scr buff pos of data we have consumed
    bool quit = false;                  // flag for reading loop

    // 1) Create a screen buffer, set size and clear

    sa.nLength = sizeof(sa);
    scrBuff = CreateConsoleScreenBuffer( GENERIC_READ | GENERIC_WRITE,
                                         FILE_SHARE_READ | FILE_SHARE_WRITE,
                                         &sa, CONSOLE_TEXTMODE_BUFFER, NULL);
    SetConsoleScreenBufferSize(scrBuff, scrBuffSize);
    // clear the screen buffer
    FillConsoleOutputCharacter(scrBuff, '\0', scrBuffSize.X * scrBuffSize.Y,
                               pos, &NumberOfCharsWritten);

    // 2) Create and start a process
    //      [using our screen buffer as stdout]

    ZeroMemory(&procInfo, sizeof(PROCESS_INFORMATION));
    ZeroMemory(&startInfo, sizeof(STARTUPINFO));
    startInfo.cb = sizeof(STARTUPINFO);
    startInfo.hStdOutput = scrBuff;
    startInfo.hStdError = GetStdHandle(STD_ERROR_HANDLE);
    startInfo.hStdInput = GetStdHandle(STD_INPUT_HANDLE);
    startInfo.dwFlags |= STARTF_USESTDHANDLES;
    CreateProcess(NULL, cmdline, NULL, NULL, FALSE,
                  0, NULL, NULL, &startInfo, &procInfo);    
    CloseHandle(procInfo.hThread);

    // 3) Read from our screen buffer while process is alive

    while(!quit)
    {
        // check if process is still alive or we could quit reading
        GetExitCodeProcess(procInfo.hProcess, &procExitCode);
        if(procExitCode != STILL_ACTIVE) quit = true;

        // get actual state of screen buffer
        GetConsoleScreenBufferInfo(scrBuff, &scrBuffInfo);

        // check if screen buffer cursor moved since
        // last time means new output was written
        if (pos.X != scrBuffInfo.dwCursorPosition.X ||
            pos.Y != scrBuffInfo.dwCursorPosition.Y)            
        {
            // Get new content of screen buffer
            //  [ calc len from pos to cursor pos: 
            //    (curY - posY) * lineWidth + (curX - posX) ]
            DWORD len =  (scrBuffInfo.dwCursorPosition.Y - pos.Y)
                        * scrBuffInfo.dwSize.X 
                        +(scrBuffInfo.dwCursorPosition.X - pos.X);
            char buffer[len];
            ReadConsoleOutputCharacter(scrBuff, buffer, len, pos, &len);

            // Print new content
            // [ there is no newline, unused space is filled with '\0'
            //   so we read char by char and if it is '\0' we do 
            //   new line and forward to next real char ]
            for(int i = 0; i < len; i++)
            {
                if(buffer[i] != '\0') printf("%c",buffer[i]);
                else
                {
                    printf("\n");
                    while((i + 1) < len && buffer[i + 1] == '\0')i++;
                }
            }

            // Save new position of already consumed data
            pos = scrBuffInfo.dwCursorPosition;
        }
        // no new output so sleep a bit before next check
        else Sleep(100);
    }

    // 4) Cleanup and end

    CloseHandle(scrBuff);   
    CloseHandle(procInfo.hProcess);
    return 0;
}

Dart don't buffer process stdout (tell process that it is running from a terminal)

I'm answering my own question in case anyone finds this post.

I ended up writing the backend in nodejs with the help of this library (https://www.npmjs.com/package/node-pty)

But I found a similar library for dart as well (https://pub.dev/packages/pty)

---

Related Topics