Using Boost::Iostreams::Tee_Device

Using boost::iostreams::tee_device?

You use the constructor-forwarding version of io::stream, which construct a tee-stream itself and forward all arguments to that. C++03 has only limited capabilities when it comes to forwarding arguments to functions (amount of overloads needed easily grow exponentially). It (io::stream) makes the following restrictions:

Each of these members constructs an instance of stream and associates it with an instance of the Device T constructed from the given lists of arguments. The T constructors involved must take all arguments by value or const reference.

Well, but the tee_device constructor says

Constructs an instance of tee_device based on the given pair of Sinks. Each function parameter is a non-const reference if the corresponding template argument is a stream or stream buffer type, and a const reference otherwise.

That won't work, of course. io::stream provides another constructor that takes a T as first argument. This works here (Compiles, at least. The assertion fails, though. I've not worked with boost::iostreams so i can't help with that)

namespace io = boost::iostreams;
typedef io::tee_device<std::stringstream, std::stringstream> TeeDevice;
typedef io::stream< TeeDevice > TeeStream;
std::stringstream ss1, ss2;
TeeDevice my_tee(ss1, ss2); 
TeeStream my_split(my_tee);
my_split << "Testing";
assert(ss1.str() == "Testing" && ss1.str() == ss2.str());

Edit: After calling flush() or streaming << std::flush, the assertion passes.

Any way to wrap construction of boost tee stream for automatic type deduction?

Compile fine in c++17 with "guaranty copy elision".

In c++11, you might use return {..}:

template <typename StreamT1, typename StreamT2>
boost::iostreams::stream<boost::iostreams::tee_device<StreamT1, StreamT2> >
make_tee(StreamT1 & t1, StreamT2 & t2)
{
    using boost::iostreams::stream;
    using boost::iostreams::tee;
    return {tee(t1,t2)};
}

Demo

C++ hello world Boost tee example program

Based on help from the question John linked:

#include <boost/iostreams/tee.hpp>
#include <boost/iostreams/stream.hpp>
#include <fstream>
#include <iostream>

using std::ostream;
using std::ofstream;
using std::cout;

namespace bio = boost::iostreams;
using bio::tee_device;
using bio::stream;

int main()
{
    typedef tee_device<ostream, ofstream> TeeDevice;
    typedef stream<TeeDevice> TeeStream;
    ofstream ofs("sample.txt");
    TeeDevice my_tee(cout, ofs); 
    TeeStream my_split(my_tee);
    my_split << "Hello, World!\n";
    my_split.flush();
    my_split.close();
}

dupplicating std::ofstream appended content

The Tee filter from Boost.Iostreams can split an output stream into two.

Here's an example inspired heavily by the one given by Johannes Schaub in his answer here.

#include <sstream>
#include <iostream>
#include <boost/iostreams/stream.hpp>
#include <boost/iostreams/tee.hpp>

int main()
{
    namespace io = boost::iostreams;
    typedef io::tee_device<std::ostream, std::stringstream> TeeDevice;
    typedef io::stream<TeeDevice> TeeStream;
    std::stringstream ss;
    TeeDevice teeDevice(std::cout, ss);
    TeeStream tee(teeDevice);
    tee << "Hello World\n" << std::flush;
    std::cout << "ss: " << ss.str() << "\n";
}

When I omit the flush manipulator, ss.str() returns an empty string. I don't know whether or not this is the expected behaviour.

Tee-ing input (cin) out to a log file (or clog)

Final Answer:

#ifndef TEE_ISTREAM_H_
#define TEE_ISTREAM_H_

/*****************************************************************************/

#include <boost/iostreams/tee.hpp>
#include <boost/iostreams/invert.hpp>
#include <boost/iostreams/device/file.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/tr1/memory.hpp>
#include <iostream>

/*****************************************************************************/

namespace bio = boost::iostreams;

/*****************************************************************************/

class tee_source : public bio::source {
public:
    tee_source( std::istream & in, const std::string & filename )
        : in_(in), log_file_(filename, std::ios::app), tee_(bio::tee(log_file_), 1)
    { }

    std::streamsize read(char* s, std::streamsize n)
    {
        return tee_.read(in_,s,n);
    }

private:
    std::istream &                               in_;
    bio::file                                    log_file_;
    bio::inverse< bio::tee_filter< bio::file > > tee_;
};

/*****************************************************************************/

typedef bio::filtering_istream                tee_istream_t;
typedef std::tr1::shared_ptr< tee_istream_t > tee_istream_ptr_t;

/*****************************************************************************/

inline tee_istream_ptr_t make_tee_istream( std::istream & in, const std::string & filename )
{
    return tee_istream_ptr_t( new tee_istream_t( tee_source( in , filename ), 0 ) );
}

/*****************************************************************************/

#endif

Can someone provide an example of seeking, reading, and writing a 4GB file using boost iostreams

Short answer

Just include

#include <boost/iostreams/seek.hpp>

and use the seek function as in

boost::iostreams::seek(device, offset, whence);

where

• device is a file, stream, streambuf or any object convertible to seekable;
• offset is a 64-bit offset of type stream_offset;
• whence is BOOST_IOS::beg, BOOST_IOS::cur or BOOST_IOS::end.

The return value of seek is of type std::streampos, and it can be converted to a stream_offset using the position_to_offset function.

Example

Here is an long, tedious and repetitive example, which shows how to open two files, seek to offstets >4GB, and copying data between them.

WARNING: This code will create very large files (several GB). Try this example on an OS/file system which supports sparse files. Linux is ok; I did not test it on other systems, such as Windows.

/*
 * WARNING: This creates very large files (several GB)
 * unless your OS/file system supports sparse files.
 */
#include <boost/iostreams/device/file.hpp>
#include <boost/iostreams/positioning.hpp>
#include <cstring>
#include <iostream>

using boost::iostreams::file_sink;
using boost::iostreams::file_source;
using boost::iostreams::position_to_offset;
using boost::iostreams::seek;
using boost::iostreams::stream_offset;

static const stream_offset GB = 1000*1000*1000;

void setup()
{
    file_sink out("file1", BOOST_IOS::binary);
    const char *greetings[] = {"Hello", "Boost", "World"};
    for (int i = 0; i < 3; i++) {
        out.write(greetings[i], 5);
        seek(out, 7*GB, BOOST_IOS::cur);
    }
}

void copy_file1_to_file2()
{
    file_source in("file1", BOOST_IOS::binary);
    file_sink out("file2", BOOST_IOS::binary);
    stream_offset off;

    off = position_to_offset(seek(in, -5, BOOST_IOS::end));
    std::cout << "in: seek " << off << std::endl;

    for (int i = 0; i < 3; i++) {
        char buf[6];
        std::memset(buf, '\0', sizeof buf);

        std::streamsize nr = in.read(buf, 5);
        std::streamsize nw = out.write(buf, 5);
        std::cout << "read: \"" << buf << "\"(" << nr << "), "
                  << "written: (" << nw << ")" << std::endl;

        off = position_to_offset(seek(in, -(7*GB + 10), BOOST_IOS::cur));
        std::cout << "in: seek " << off << std::endl;
        off = position_to_offset(seek(out, 7*GB, BOOST_IOS::cur));
        std::cout << "out: seek " << off << std::endl;
    }
}

int main()
{
    setup();
    copy_file1_to_file2();
}

Run-time error reading a .gz file using boost::iostreams and zlib

The gzip file format has an additional header around the zlib data, which zlib can't read.

So you want to use boost's gzip_decompressor instead of zlib_decompressor.

in.push(gzip_decompressor());

Note you'll need to include boost/iostreams/filter/gzip.h instead of boost/iostreams/filter/zlib.h.

Here's a working example of streaming a GZIP file:

#include <fstream>
#include <iostream>
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/filter/gzip.hpp>
#include <boost/iostreams/copy.hpp>

using namespace boost::iostreams;

int main()
{
    std::ifstream file("hello.gz", std::ios_base::in | std::ios_base::binary);
    filtering_streambuf < input > in;
    in.push(gzip_decompressor());
    in.push(file);
    boost::iostreams::copy(in, std::cout);
}

You'll find more information on specific boost::iostreams filters lurking here in boost's documentation: http://www.boost.org/doc/libs/1_46_1/libs/iostreams/doc/quick_reference.html#filters

I also feel I should point out that your code didn't compile with gcc: in the C++ standard library, the ifstream constructor takes a const char *, not a std::string. (I'm not sure about Microsoft's version).

---

Related Topics