Boost Asio Streambuf

Convert std::string to boost::asio::streambuf

The following code shows how to convert between std::string and boost::asio::streambuf, see it online at ideone:

#include <iostream>
#include <boost/asio/streambuf.hpp>
#include <boost/asio/buffer.hpp>

int main()
{
   /* Convert std::string --> boost::asio::streambuf */
   boost::asio::streambuf sbuf;
   std::iostream os(&sbuf);
   std::string message("Teststring");
   os << message;

   /* Convert boost::asio::streambuf --> std::string */
   std::string str((std::istreambuf_iterator<char>(&sbuf)),
                    std::istreambuf_iterator<char>());

   std::cout << str << std::endl;
}

Boost ASIO streambuf

The nomenclature for boost::asio::streambuf is similar to that of which is defined in the C++ standard, and used across various classes in the standard template library, wherein data is written to an output stream and data is read from an input stream. For example, one could use std::cout.put() to write to the output stream, and std::cin.get() to read from the input stream.

When manually controlling the streambuf input and output sequences, the general lifecycle of data is as follows:

Buffers get allocated with prepare() for the output sequence.
After data has been written into the output sequence's buffers, the data will be commit()ed. This committed data is removed from the output sequence and appended to the input sequence from which it can be read.
Data is read from the input sequence's buffers obtained via data().
Once data has been read, it can then be removed from the input sequence by consume().

When using Boost.Asio operations that operate on streambuf or stream objects that use a streambuf, such as std::ostream, the underlying input and output sequences will be properly managed. If a buffer is provided to an operation instead, such as passing passing prepare() to a read operation or data() to a write operation, then one must explicitly handle the commit() and consume().

Here is an annotated version of the example code which writes directly from an streambuf to a socket:

// The input and output sequence are empty.
boost::asio::streambuf b;
std::ostream os(&b);

// prepare() and write to the output sequence, then commit the written
// data to the input sequence.  The output sequence is empty and
// input sequence contains "Hello, World!\n".
os << "Hello, World!\n";

// Read from the input sequence, writing to the socket.  The input and
// output sequences remain unchanged.
size_t n = sock.send(b.data());

// Remove 'n' bytes from the input sequence. If the send operation sent
// the entire buffer, then the input sequence would be empty.
b.consume(n);

And here is the annotated example for reading from a socket directly into an streambuf. The annotations assume that the word "hello" has been received, but not yet read, on the socket:

boost::asio::streambuf b;

// prepare() 512 bytes for the output sequence.  The input sequence
// is empty.
auto bufs = b.prepare(512);

// Read from the socket, writing into the output sequence.  The
// input sequence is empty and the output sequence contains "hello".
size_t n = sock.receive(bufs);

// Remove 'n' (5) bytes from output sequence appending them to the
// input sequence.  The input sequence contains "hello" and the
// output sequence has 507 bytes.
b.commit(n);

// The input and output sequence remain unchanged.
std::istream is(&b);
std::string s;

// Read from the input sequence and consume the read data.  The string
// 's' contains "hello".  The input sequence is empty, the output
// sequence remains unchanged.
is >> s;

Note how in the above examples, the steam objects handled committed and consuming the streambuf's output and input sequences. However, when the buffers themselves were used (i.e. data() and prepare()), the code needed to explicitly handle commits and consumes.

How copy or reuse boost::asio::streambuf?

One can use boost::asio::buffer_copy() to copy the contents of Asio buffers. This can be convenient if, for example, one wishes to copy the contents of one streambuf to another streambuf.

boost::asio::streambuf source, target;
...
std::size_t bytes_copied = buffer_copy(
  target.prepare(source.size()), // target's output sequence
  source.data());                // source's input sequence
// Explicitly move target's output sequence to its input sequence.
target.commit(bytes_copied);

A similar approach can be used to copy from a streambuf into any type for which Asio supports mutable buffers. For example, copying content into a std::vector<char>:

boost::asio::streambuf source;
...
std::vector<char> target(source.size());
buffer_copy(boost::asio::buffer(target), source.data());

One notable exception is that Asio does not support returning a mutable buffer for std::string. However, one can still accomplish copying into std::string via iterators:

boost::asio::streambuf source;
...
std::string target{
  buffers_begin(source.data()),
  buffers_end(source.data())
};

Here is an example demonstrating copying contents from boost::asio::streambuf into various other types:

#include <iostream>
#include <string>
#include <vector>
#include <boost/asio.hpp>

int main()
{
  const std::string expected = "This is a demo";

  // Populate source's input sequence.
  boost::asio::streambuf source;
  std::ostream ostream(&source);
  ostream << expected;

  // streambuf example
  {
    boost::asio::streambuf target;
    target.commit(buffer_copy(
      target.prepare(source.size()), // target's output sequence
      source.data()));               // source's input sequence

    // Verify the contents are equal.
    assert(std::equal(
      buffers_begin(target.data()),
      buffers_end(target.data()),
      begin(expected)
    ));
  }

  // std::vector example
  {
    std::vector<char> target(source.size());
    buffer_copy(boost::asio::buffer(target), source.data());

    // Verify the contents are equal.
    assert(std::equal(begin(target), end(target), begin(expected)));
  }

  // std::string example
  {
    // Copy directly into std::string.  Asio does not support
    // returning a mutable buffer for std::string.
    std::string target{
      buffers_begin(source.data()),
      buffers_end(source.data())
    };

    // Verify the contents are equal.
    assert(std::equal(begin(target), end(target), begin(expected)));
  }
}

boost::asio::streambuf::consume - Injects garbage character

boost::asio::streambuf::consume overflows.

Use

buffer.consume(buffer.size());

Rather than

buffer.consume(std::numeric_limits<size_t>::max());

Reporting bug to boost mailing list.

Boost ASIO ForwardIterator for streambuf

Boost has buffers_begin() and buffers_end() which you can use on streambuf's data():

Live On Coliru

#include <boost/asio.hpp>
#include <boost/spirit/include/qi.hpp>

namespace a = boost::asio;
namespace qi = boost::spirit::qi;

#include <iostream>
int main() 
{
    a::streambuf input;
    std::ostream(&input) << "123, 4, 5; 78, 8, 9;\n888, 8, 8;";

    auto f = a::buffers_begin(input.data()), l = a::buffers_end(input.data());
    //std::copy(f, l, std::ostream_iterator<char>(std::cout));

    std::vector<std::vector<int> > parsed;
    bool ok = qi::phrase_parse(f, l, *(qi::int_ % ',' >> ';'), qi::space, parsed);

    if (ok) {
        std::cout << "parsed: \n";
        for (auto& row : parsed) {
            for (auto& v : row) { std::cout << v << " "; }
            std::cout << ";\n";
        }
    }
    else
        std::cout << "parse failed\n";

    if (f!=l)
        std::cout << "remaining unparsed input: '" << std::string(f,l) << "'\n";
}

Prints (as expected):

parsed: 
123 4 5 ;
78 8 9 ;
888 8 8 ;

Note don't forget to consume() the parsed portion of the input!