Setting the Internal Buffer Used by a Standard Stream (Pubsetbuf)

Internal buffer used by standard input stream (pubsetbuf)

I've solved it! Spot the difference.

template <typename char_type>
struct istreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> >
{
    istreambuf(char_type* buffer, std::streamsize buffer_length)
    {
        // Set the "put" pointer to the start of the buffer and record its length.
        //this->setp(buffer, buffer + buffer_length);

        // Set the "get" pointer to the start of the buffer, the next item, and record its length.
        this->setg(buffer, buffer, buffer + buffer_length);
    }
};

I needed to set the "get" pointer, not the "put" pointer. It works well now.

Setting the internal buffer used by a standard stream (pubsetbuf)

After some more research on this problem, and scrutiny of my code, I came across a post suggesting the use of a hand-coded std::streambuf class. The idea behind this code is to create a streambuf that initializes its internals to refer to the given buffer. The code is as follows.

#include <streambuf>

template <typename char_type>
struct ostreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> >
{
    ostreambuf(char_type* buffer, std::streamsize bufferLength)
    {
        // set the "put" pointer the start of the buffer and record it's length.
        setp(buffer, buffer + bufferLength);
    }
};

Now if you look at my original code, you will notice that I didn't really need a stringstream to begin with. All I really needed was a way to write to an external buffer using the IOStream library and std::ostream is a much better type to address this problem. Incidentally, I suspect this is how the array_sink type from Boost.IOStreams is implemented.

Here is the modified code that uses my ostreambuf type.

#include <ostream>
#include "ostreambuf.h"  // file including ostreambuf struct from above.

void FillBuffer(char* buffer, unsigned int size)
{
    ostreambuf<char> ostreamBuffer(buffer, size);
    std::ostream messageStream(&ostreamBuffer);

    messageStream << "Hello" << std::endl;
    messageStream << "World!" << std::endl;
}

stringstream->rdbuf()->pubsetbuf is not setting the buffer

Not should set contents. pubsetbuf calls virtual setbuf

basic_streambuf<charT,traits>* setbuf(charT* s, streamsize n);

15 Effects: implementation-defined, except that setbuf(0,0) has no effect.

16 Returns: this.

VS 2010. There is no overload of virtual method setbuf in basic_stringbuf, it uses default from basic_streambuf

virtual _Myt *__CLR_OR_THIS_CALL setbuf(_Elem *, streamsize)
    {   // offer buffer to external agent (do nothing)
    return (this);
    }

C++ How to actually use pubsetbuf in ofstream?

Following the suggestions of @pantarei and @lightnessracesinorbit, I'll write the answer. I apologize if I bent the rules.

---

According to the cppreference site, the order of setting pubsetbuf matters, because it needs to be set before opening any files, otherwwise it has no effect. So, this is the order of the code as it needs to be (for my case):

int sz {131072};          // buffer size
std::vector<char> buf;   // std::vector instead of C-style char
buf.resize(sz);
std::ofstream outf;      // declaration, only
outf.rdbuf()->pubsetbuf(&buf[0], sz);  // set buffer before...
outf.open("file.txt");                 // ...opening the file
// rest of the code

My files are, most often, below 100k, so a 128k buffer is just fine to avoid too many writes.

Visual Studio std::stringstream pubsetbuf does not work

putsetbuf only makes sense for fstream (technically, for std::basic_filebuf), where the buffer and the stream are two different things.

For stringstream (technically, std::basic_stringbuf) they are one and the same std::string.

If you need a stream that works on a string that's external to it, consider std::strstream: or boost::iostreams::array_sink

initializing a C++ std::istringstream from an in memory buffer?

Look at std::istrstream it has a constructor

 istrstream( char* pch, int nLength );

This class is sort of depreciated or at least you are normally told to use other classes.

The issue with strstream is that it is more complex to manage the memory of the char* buffer so in general you would prefer stringstream as it does the memory management for you. However in this case you are already managing the memory of the char* so the normal benefit is in this case a cost. In fact in this case strstream does exactly what you want with minimal overhead in code or speed. This is similar to the discussion of ostrsteram by Herb Sutter

c++ rdbuf()->pubsetbuf for a stream, using a char buffer. I have no data in it

I don't see zeros. Checked with gcc 4.5. If you use Visual Studio, you can find this link usefull:

Setting the internal buffer used by a standard stream (pubsetbuf)

rdbuf()->pubsetbuf() using a bidirectional fstream is applied only to writes

The behaviour of setbuf isn't very well specified: https://en.cppreference.com/w/cpp/io/basic_filebuf/setbuf

According to cppreference (which matches my experience) libstdc++ only uses the buffer if you call pubsetbuf before opening the file, visual studio only uses the buffer if passed after opening the file. Therefore for cross platform code which has a resonable chance (but no guarantee) of using your buffer you should do:

fstream stream;
stream.rdbuf()->pubsetbuf(file_buffer->data(), file_buffer->size());
stream.open(R"(C:\test\test_file.bin)", ios::in | ios::out | ios::binary);
stream.rdbuf()->pubsetbuf(file_buffer->data(), file_buffer->size());

Also note you don't need to actually supply a buffer to pubsetbuf, you can just pass a null pointer:

fstream stream;
stream.rdbuf()->pubsetbuf(nullptr, BUFFER_SIZE);
stream.open(R"(C:\test\test_file.bin)", ios::in | ios::out | ios::binary);
stream.rdbuf()->pubsetbuf(nullptr, BUFFER_SIZE);

If you want to target libstdc++ in the future it is also worth noting that your buffer size needs to be 1 larger than your desired size.

boost::iostreams gives you a little more direct control over buffer sizes.

---

Related Topics