Turn Off Buffering

Disable output buffering

From Magnus Lycka answer on a mailing list:

You can skip buffering for a whole
python process using python -u
(or #!/usr/bin/env python -u etc.) or by
setting the environment variable
PYTHONUNBUFFERED.

You could also replace sys.stdout with
some other stream like wrapper which
does a flush after every call.

class Unbuffered(object):
   def __init__(self, stream):
       self.stream = stream
   def write(self, data):
       self.stream.write(data)
       self.stream.flush()
   def writelines(self, datas):
       self.stream.writelines(datas)
       self.stream.flush()
   def __getattr__(self, attr):
       return getattr(self.stream, attr)

import sys
sys.stdout = Unbuffered(sys.stdout)
print 'Hello'

Turn off buffering

The problem, I believe is in grep buffering its output. It is doing that when you pipe tail -f | grep ... | some_other_prog. To get grep to flush once per line, use the --line-buffered option:

% tail -f data.txt | grep -e APL --line-buffered | test.py
APL

APL

APL

where test.py is:

import sys
for line in sys.stdin:
    print(line)

(Tested on linux, gnome-terminal.)

Turn off buffering in pipe

You can use the unbuffer command (which comes as part of the expect package), e.g.

unbuffer long_running_command | print_progress

unbuffer connects to long_running_command via a pseudoterminal (pty), which makes the system treat it as an interactive process, therefore not using the 4-kiB buffering in the pipeline that is the likely cause of the delay.

For longer pipelines, you may have to unbuffer each command (except the final one), e.g.

unbuffer x | unbuffer -p y | z

How to turn off buffering on write() system call?

Perhaps what you mean is that you want to disable Nagle's Algorithm in which case the solution is:

setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, (int[]){1}, sizeof(int));

Edit: Hmm, it looks like you want more than this, and I doubt what you want is possible without designing your own protocol on top of UDP.

Edit 2: You may be able to get an effect similar to what you want by limiting the send and receive buffer sizes. The server (sender) should do:

setsockopt(sock, SOL_SOCKET, SO_SNDBUF, (int[]){YOUR_BUF_LIMIT}, sizeof(int));

and the client (receiver) should do:

setsockopt(sock, SOL_SOCKET, SO_RCVBUF, (int[]){YOUR_BUF_LIMIT}, sizeof(int));

How to disable buffering on a stream?

For file streams, you can use pubsetbuf for that :

std::ifstream f;
f.rdbuf()->pubsetbuf(0, 0);
f.open("test");

Explanation

The C++ standard says the following about the effect of setbuf (and thus pubsetbuf) for file streams :

If setbuf(0,0) is called on a stream before any I/O has occurred on that stream, the stream
becomes unbuffered. Otherwise the results are implementation-defined. “Unbuffered” means that
pbase() and pptr() always return null and output to the file should appear as soon as possible.

The first sentence guarantees that the above code makes the stream unbuffered. Note that some compilers (eg. gcc) see opening a file as an I/O operation on the stream, so pubsetbuf should be called before opening the file (as above).

The last sentence, however, seems to imply that that would only be for output, and not for input. I'm not sure if that was an oversight, or whether that was intended. Consulting your compiler documentation might be useful. For gcc eg., both input and output are made unbuffered (ref. GNU C++ Library Manual - Stream Buffers).

Stop buffering of adb shell output

Use adb exec-out instead of adb shell command to force the new adb to use the raw mode and avoid stdout buffering.

For more info read comments in shell_service.cpp

How to disable output buffering in PHP

tl;dr version

Do two things:

1. Disable the userspace output buffer, either...

   • Globally, by either...

     • Turning off output_buffering in your php.ini, or

     • Turning off output_buffering in your Apache config using

       php_flag "output_buffering" Off
       

   • or for just the script you care about, by either...

     • calling ob_end_flush(), or
     • calling ob_end_clean()

2. Also, disable the server-level output buffer as much as you possibly can, by either:

   • calling ob_implicit_flush() at the start of your script, or
   • calling flush() after every echo statement or other statement that adds output to the response body

Longer version

Confusingly, there are two layers of buffering that may be relevant and the PHP documentation does a poor job of distinguishing between the two.

The output buffer

The first layer is usually referred to by the PHP docs as the 'output buffer'. This layer of buffering only affects output to the body of the HTTP response, not the headers. You can turn on output buffering with ob_start(), and turn it off with ob_end_flush() or ob_end_clean(). You can also have all your scripts automatically start with output buffering on using the output_buffering option in php.ini.

The default value of this option for production versions of php.ini is 4096, which means that the first 4096 bytes of output will be buffered in the output buffer, at which point it will be flushed and output buffering is turned off.

You can disable this layer of buffering globally by setting output_buffering to Off in your php.ini file (or using

php_flag "output_buffering" Off

in your Apache config, if you're using Apache). Alternatively, you can disable it for a single script by calling ob_end_clean() or ob_end_flush() at the start of the script.

The write buffer, and the webserver buffer

Beyond the output buffer is what the PHP manual refers to as the 'write buffer', plus any buffering system your web server has. If you're using PHP with Apache through mod_php, and are not using mod_gzip, you can call flush() to flush these; with other backends, it might work too, although the manual is cagey about giving assurances:

Description

void flush ( void )

Flushes the write buffers of PHP and whatever backend PHP is using (CGI, a web server, etc). This attempts to push current output all the way to the browser with a few caveats.

flush() may not be able to override the buffering scheme of your web server and it has no effect on any client-side buffering in the browser. It also doesn't affect PHP's userspace output buffering mechanism. This means you will have to call both ob_flush() and flush() to flush the ob output buffers if you are using those.

There are also a couple of ways you can make PHP automatically call flush() every time you echo anything (or do anything else that echoes output to the response body).

The first is to call ob_implicit_flush(). Note that this function is deceptively named; given its ob_ prefix, any reasonable person would expect that it would affect the 'output buffer', as do ob_start, ob_flush etc. However, this is not the case; ob_implicit_flush(), like flush(), affects the server-level output buffer and does not interact in any way with the output buffer controlled by the other ob_ functions.

The second is to globally enable implicit flushing by setting the implicit_flush flag to On in your php.ini. This is equivalent to calling ob_implicit_flush() at the start of every script. Note that the manual advises against this, cryptically citing "serious performance implications", some of which I explore in this tangentially related answer.

---

Related Topics