C++ Custom Stream Manipulator That Changes Next Item on Stream

How to implement custom sticky manipulator that automatically adds separators?

The issue with the solution you posted is that it relies on customizing the way integers get formatted using facets. Unfortunately, I do not think there is a corresponding facility which would work for arbitrary types.

There is. You can use utilize the underlying buffer of the stream to get what you want. The buffer is where the character sequence is eventually gathered for maintenance. The following code makes a stream buffer that holds a reference to the object whose character sequence you wish to use. We set the std::ios_base::unitbuf format flag so that the stream is flushed on each output operation (so we can add the separator to the end).

By extension, it also allows you to uninstall the separator and makes sure that no memory is leaked in the process:

#include <iostream>
#include <string>

namespace custom
{
    struct sep_impl
    {
        sep_impl(std::string const& separator);
        std::string separator;
    };
    
    sep_impl sep(std::string const& str)
    {
        return sep_impl(str);
    }
    
    std::ostream& nosep(std::ostream& os);
}

int separatorEnabled()
                      { static int idx = std::ios_base::xalloc(); return idx; }
int getSeparator()    { static int idx = std::ios_base::xalloc(); return idx; }

struct custom_separator : std::streambuf
{
public:
    custom_separator(std::ostream& _stream) : stream(_stream)
    { }
    
    int_type overflow(int_type c)
    {
        return stream.rdbuf()->sputc(c);
    }
    
    int sync()
    {
        if (stream.iword(separatorEnabled()))
        {
            void*& p = stream.pword(getSeparator());
            stream << *static_cast<std::string*>(p);
            return 0;
        }
        return stream.rdbuf()->pubsync();
    }
private:
    std::ostream& stream;
};

void cleanup(std::ios_base::event evt, std::ios_base& str, int idx)
{
    if (str.iword(separatorEnabled()) && evt == std::ios_base::erase_event)
    {
        void*& p = str.pword(idx);
        delete static_cast<std::string*>(p);
        str.iword(separatorEnabled()) = false; 
    }
}

std::ostream& set_separator(std::ostream& os, const custom::sep_impl& manip)
{
    if (!os.bad())
    {
        os.pword(getSeparator()) = new std::string(manip.separator);
        os.register_callback(cleanup, getSeparator());
    }
    
    return os;
}

std::ostream& operator<<(std::ostream& os, const custom::sep_impl& manip)
{
    std::ostream* p = os.tie();
    if (p && !p->iword(separatorEnabled()))
    {
        set_separator(*p, manip);
        p->iword(separatorEnabled()) = true;
    }
    
    return os << std::unitbuf;
}

namespace custom
{
    sep_impl::sep_impl(std::string const& _sep) : separator(_sep) { }
    
    std::ostream& nosep(std::ostream& os)
    {
        cleanup(std::ios_base::erase_event, *os.tie(), getSeparator());
        os.tie(nullptr);
        return os << std::nounitbuf;
    }
    
    void install_separator(std::ostream& o1, std::ostream& o2)
    {
        static custom_separator csep(o2);
        o1.rdbuf(&csep);
        o1.tie(&o2);
    }
}

int main()
{
    std::ostream os(nullptr);
    custom::install_separator(os, std::cout);
    
    os << custom::sep(", ") << 4 << 2 << custom::nosep;
}

I'm sure there is also room for improvement, so if anyone has any suggestions they are very much appreciated.

Live Example

Custom stream manipulator for streaming integers in any base

You can do something like the following. I have commented the code to explain what each part is doing, but essentially its this:

• Create a "manipulator" struct which stores some data in the stream using xalloc and iword.
• Create a custom num_put facet which looks for your manipulator and applies the manipulation.

Here is the code...

Edit: Note that im not sure I handled the std::ios_base::internal flag correctly here - as I dont actually know what its for.

Edit 2: I found out what std::ios_base::internal is for, and updated the code to handle it.

Edit 3: Added a call to std::locacle::global to show how to make all the standard stream classes support the new stream manipulator by default, rather than having to imbue them.

#include <algorithm>
#include <cassert>
#include <climits>
#include <iomanip>
#include <iostream>
#include <locale>

namespace StreamManip {

// Define a base manipulator type, its what the built in stream manipulators
// do when they take parameters, only they return an opaque type.
struct BaseManip
{
    int mBase;

    BaseManip(int base) : mBase(base)
    {
        assert(base >= 2);
        assert(base <= 36);
    }

    static int getIWord()
    {
        // call xalloc once to get an index at which we can store data for this
        // manipulator.
        static int iw = std::ios_base::xalloc();
        return iw;
    }

    void apply(std::ostream& os) const
    {
        // store the base value in the manipulator.
        os.iword(getIWord()) = mBase;
    }
};

// We need this so we can apply our custom stream manipulator to the stream.
std::ostream& operator<<(std::ostream& os, const BaseManip& bm)
{
    bm.apply(os);
    return os;
}

// convience function, so we can do std::cout << base(16) << 100;
BaseManip base(int b)
{
    return BaseManip(b);
}

// A custom number output facet.  These are used by the std::locale code in
// streams.  The num_put facet handles the output of numberic values as characters
// in the stream.  Here we create one that knows about our custom manipulator.
struct BaseNumPut : std::num_put<char>
{
    // These absVal functions are needed as std::abs doesnt support 
    // unsigned types, but the templated doPutHelper works on signed and
    // unsigned types.
    unsigned long int absVal(unsigned long int a) const
    {
        return a;
    }

    unsigned long long int absVal(unsigned long long int a) const
    {
        return a;
    }

    template <class NumType>
    NumType absVal(NumType a) const
    {
        return std::abs(a);
    }

    template <class NumType>
    iter_type doPutHelper(iter_type out, std::ios_base& str, char_type fill, NumType val) const
    {
        // Read the value stored in our xalloc location.
        const int base = str.iword(BaseManip::getIWord());

        // we only want this manipulator to affect the next numeric value, so
        // reset its value.
        str.iword(BaseManip::getIWord()) = 0;

        // normal number output, use the built in putter.
        if (base == 0 || base == 10)
        {
            return std::num_put<char>::do_put(out, str, fill, val);
        }

        // We want to conver the base, so do it and output.
        // Base conversion code lifted from Nawaz's answer

        int digits[CHAR_BIT * sizeof(NumType)];
        int i = 0;
        NumType tempVal = absVal(val);

        while (tempVal != 0)
        {
            digits[i++] = tempVal % base;
            tempVal /= base;
        }

        // Get the format flags.
        const std::ios_base::fmtflags flags = str.flags();

        // Add the padding if needs by (i.e. they have used std::setw).
        // Only applies if we are right aligned, or none specified.
        if (flags & std::ios_base::right || 
            !(flags & std::ios_base::internal || flags & std::ios_base::left))
        {
            std::fill_n(out, str.width() - i, fill);
        }

        if (val < 0)
        {
            *out++ = '-';
        }

        // Handle the internal adjustment flag.
        if (flags & std::ios_base::internal)
        {
            std::fill_n(out, str.width() - i, fill);
        }

        char digitCharLc[] = "0123456789abcdefghijklmnopqrstuvwxyz";
        char digitCharUc[] = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";

        const char *digitChar = (str.flags() & std::ios_base::uppercase)
            ? digitCharUc
            : digitCharLc;

        while (i)
        {
            // out is an iterator that accepts characters
            *out++ = digitChar[digits[--i]];
        }

        // Add the padding if needs by (i.e. they have used std::setw).
        // Only applies if we are left aligned.
        if (str.flags() & std::ios_base::left)
        {
            std::fill_n(out, str.width() - i, fill);
        }

        // clear the width
        str.width(0);

        return out;
    }

    // Overrides for the virtual do_put member functions.

    iter_type do_put(iter_type out, std::ios_base& str, char_type fill, long val) const
    {
        return doPutHelper(out, str, fill, val);
    }

    iter_type do_put(iter_type out, std::ios_base& str, char_type fill, unsigned long val) const
    {
        return doPutHelper(out, str, fill, val);
    }
};

} // namespace StreamManip

int main()
{
    // Create a local the uses our custom num_put
    std::locale myLocale(std::locale(), new StreamManip::BaseNumPut());

    // Set our locacle to the global one used by default in all streams created 
    // from here on in.  Any streams created in this app will now support the
    // StreamManip::base modifier.
    std::locale::global(myLocale);

    // imbue std::cout, so it uses are custom local.
    std::cout.imbue(myLocale);
    std::cerr.imbue(myLocale);

    // Output some stuff.
    std::cout << std::setw(50) << StreamManip::base(2) << std::internal << -255 << std::endl;
    std::cout << StreamManip::base(4) << 255 << std::endl;
    std::cout << StreamManip::base(8) << 255 << std::endl;
    std::cout << StreamManip::base(10) << 255 << std::endl;
    std::cout << std::uppercase << StreamManip::base(16) << 255 << std::endl;

    return 0;
}

How to create a manipulator that would call a specific function in the next object in the stream?

There are two primary ways the standard uses to manipulate input & output operations.

1. Storing values in the stream state

You can store formatting state within streams by using std::ios_base::xalloc().

This gives you a long and void* value in each stream that you can access with iword() / pword() .

This is the same mechanism that standard io manipulators like std::hex, std::boolalpha use.

Note that if you change the stream state it'll stay that way until you change it again, e.g.:

std::cout << std::hex << 16; // will be outputted in hexadecimal
std::cout << 12; // will still be outputted in hexadecimal

std::cout << std::dec << 16; // will be outputted in decimal
std::cout << 12; // still decimal

You could e.g. implement it like this for your A class:

class A {
public:
    void write_text(std::ostream& os) const {
        os << "TEXT";
    }

    void write_binary(std::ostream& os) const {
        os << "BINARY";
    }
};

// this gives us the unique index we need for pword() / iword()
inline int getAFormatIndex() {
    static int idx = std::ios_base::xalloc();
    return idx;
}

std::ostream& operator<<(std::ostream& os, A const& a) {
    std::ostream::sentry s{os};
    if(!s) return os;
    
    if(os.iword(getAFormatIndex()) == 0)
        a.write_text(os);
    else
        a.write_binary(os);

    return os;
}

struct text_t {};
struct binary_t {};

inline constexpr text_t text;
inline constexpr binary_t binary;

// change to text mode
std::ostream& operator<<(std::ostream& os, text_t const&) {
    os.iword(getAFormatIndex()) = 0;
    return os;
}

// change to binary mode
std::ostream& operator<<(std::ostream& os, binary_t const&) {
    os.iword(getAFormatIndex()) = 1;
    return os;
}

• The operator<< for A checks which format type is currently stored in the stream (0 for text, 1 for binary) and calls the corresponding method
• text & binary are the io manipulators that change the stream state when applied to a stream.

Example Usage:

A a;
std::cout << text << a;
std::cout << binary << a;
std::cout << a; // still in binary format

godbolt example

2. Wrapper function

Another kind of io manipulators you'll also encounter in the standard library are wrappers that change the input / output of a single element.

Examples of this would be std::quoted, std::get_money, std::put_money, etc...

Those functions only change the format for a single operation, in contrast to the above method that changes the format of all following input / output operations.
Example:

std::cout << std::put_money(12.34); // will be formatted as monetary value
std::cout << 12.34; // normal double output
std::cout << std::quoted("foo"); // -> "foo"
std::cout << "foo"; // -> foo

You could e.g. implement it like this for your A class:

class A {
public:
    void write_text(std::ostream& os) const {
        os << "TEXT";
    }

    void write_binary(std::ostream& os) const {
        os << "BINARY";
    }
};

std::ostream& operator<<(std::ostream& os, A const& a) {
    std::ostream::sentry s{os};
    if(!s) return os;

    a.write_text(os);

    return os;
}

struct binary_impl { A const& a; };

std::ostream& operator<<(std::ostream& os, binary_impl const& b) {
    std::ostream::sentry s{os};
    if(!s) return os;

    b.a.write_binary(os);

    return os;
}

binary_impl binary(A const& a) {
    return { a };
}

// text is the default, so we need no wrapper
A const& text(A const& a) {
    return a;
}

• We essentially use a wrapper object (binary_impl) that implements a different operator<< for A objects.

Example Usage:

A a;
std::cout << text(a);
std::cout << binary(a);
std::cout << a; // default is text format

godbolt example

---

The methods listed above are only the ones the standard library itself uses (and therefore probably the most recognized ones).

You can of course also create your own custom method for it, e.g. by using member methods that return objects that will serialize the object in a specific way:

class A {
public:
    void write_text(std::ostream& os) const {
        os << "TEXT";
    }

    void write_binary(std::ostream& os) const {
        os << "BINARY";
    }

    struct as_text_t { A const& a; };
    struct as_binary_t { A const& a; };

    as_text_t as_text() const {
        return { *this };
    }

    as_binary_t as_binary() const {
        return { *this };
    }
};

std::ostream& operator<<(std::ostream& os, A::as_text_t const& el) {
    std::ostream::sentry s{os};
    if(!s) return os;

    el.a.write_text(os);

    return os;
}

std::ostream& operator<<(std::ostream& os, A::as_binary_t const& el) {
    std::ostream::sentry s{os};
    if(!s) return os;

    el.a.write_binary(os);

    return os;
}

Usage:

A a;
std::cout << a.as_text();
std::cout << a.as_binary();

godbolt example

Custom stream manipulators

I found out that using std::ios_base::iword to store the requested character encoding was the best solution for the problem at hand:

#include <iostream>

/*!
\brief Unicode encoding
*/
enum EUnicodeEnc
{
    /** UTF-8 character encoding */
    EUnicodeEnc_UTF8 = 1,

    /** UTF-16 character encoding */
    EUnicodeEnc_UTF16 = 2,

    /** UTF-32 character encoding */
    EUnicodeEnc_UTF32 = 3
};

/** Allocate the \c std::ios_base::iword storage for use with \c SourceStreamEncoding object instances */
int SourceStreamEncoding::sourceEnc_xalloc = std::ios_base::xalloc();

/*!
\brief Stream I/O manipulator changes the source character encoding to UTF-8
*/
std::ios_base& FromUtf8(std::ios_base& os) {
    os.iword(SourceStreamEncoding::sourceEnc_xalloc) = EUnicodeEnc_UTF8;
    return os;
}

/*!
\brief Stream I/O manipulator changes the source character encoding to UTF-16
*/
std::ios_base& FromUtf16(std::ios_base& os) {
    os.iword(SourceStreamEncoding::sourceEnc_xalloc) = EUnicodeEnc_UTF16;
    return os;
}

/*!
\brief Stream I/O manipulator changes the source character encoding to UTF-32
*/
std::ios_base& FromUtf32(std::ios_base& os) {
    os.iword(SourceStreamEncoding::sourceEnc_xalloc) = EUnicodeEnc_UTF32;
    return os;
}

/*!
\brief Overrides \c std::ostream::flush()
\details Converts the buffer to the correct character encoding then flushes buffer
after writing its content to a storage device
*/
std::ostream &CFileManagerOStream::flush()
{
    switch (os.iword(SourceStreamEncoding::sourceEnc_xalloc))
    {
        case EUnicodeEnc_UTF8:
            characterEncoder.FromUTF8(...);
        break;
        case EUnicodeEnc_UTF16:
            characterEncoder.FromUTF16(...);
        break;
        case EUnicodeEnc_UTF32:
            characterEncoder.FromUTF32(...);
        break;
    }
    return (*this);
}

// Now I can do as follows:
int main()
{
    CFileManagerOStream outFile("MultipleUtf8Strings.dat"); // Custom std::ostream
    ...
#ifdef _WINDOWS
    CTcpStreamUtf16 largeBlobUtf16Stream;
    ...
    outFile << FromUtf16 << largeBlobUtf16Stream;
#else
    CTcpStreamUtf32 largeBlobUtf32Stream;
    ...
    outFile << FromUtf32 << largeBlobUtf32Stream;
#endif
}

Additionally, I've added the following manipulator that takes a single paramater:

class FromEnc
{
    public:
        explicit FromEnc(int i) : i_(i) {}
        int i_;
    private:
        template <class charT, class Traits>
        friend std::basic_ostream<charT, Traits>& operator<<(std::basic_ostream<charT, Traits>& os, const FromEnc& w) {
            os.iword(SourceStreamEncoding::sourceEnc_xalloc) = w.i_;
            return os;
        }
};

, so now I can also do as follows:

outFile << FromEnc(EUnicodeEnc_UTF16) << largeBlobUtf16Stream;

Custom manipulator for C++ iostream

It's particularly difficult to add a manipulator to a C++ stream, as one has no control of how the manipulator is used. One can imbue a new locale into a stream, which has a facet installed that controls how numbers are printed - but not how strings are output. And then the problem would still be how to store the quoting state safely into the stream.

Strings are output using an operator defined in the std namespace. If you want to change the way those are printed, yet keeping the look of manipulators, you can create a proxy class:

namespace quoting {
struct quoting_proxy {
    explicit quoting_proxy(std::ostream & os):os(os){}

    template<typename Rhs>
    friend std::ostream & operator<<(quoting_proxy const& q, 
                                     Rhs const& rhs) {
        return q.os << rhs;
    }

    friend std::ostream & operator<<(quoting_proxy const& q, 
                                     std::string const& rhs) {
        return q.os << "'" << rhs << "'";
    }

    friend std::ostream & operator<<(quoting_proxy const& q, 
                                     char const* rhs) {
        return q.os << "'" << rhs << "'";
    }
private:
    std::ostream & os;
};

struct quoting_creator { } quote;
quoting_proxy operator<<(std::ostream & os, quoting_creator) {
    return quoting_proxy(os);
}
}

int main() {
    std::cout << quoting::quote << "hello" << std::endl; 
}

Which would be suitable to be used for ostream. If you want to generalize, you can make it a template too and also accept basic_stream instead of plain string. It has different behaviors to standard manipulators in some cases. Because it works by returning the proxy object, it will not work for cases like

std::cout << quoting::quote; 
std::cout << "hello";

How do the stream manipulators work?

The standard defines the following operator<< overload in the basic_ostream class template:

basic_ostream<charT,traits>& operator<<(
    basic_ostream<charT,traits>& (*pf) (basic_ostream<charT,traits>&) );

Effects: None. Does not behave as a formatted output function (as described in 27.6.2.5.1).

Returns: pf(*this).

The parameter is a pointer to a function taking and returning a reference to a std::ostream.

This means that you can "stream" a function with this signature to an ostream object and it has the effect of calling that function on the stream. If you use the name of a function in an expression then it is (usually) converted to a pointer to that function.

std::hex is an std::ios_base manipulator defined as follows.

   ios_base& hex(ios_base& str);

Effects: Calls str.setf(ios_base::hex, ios_base::basefield).

Returns: str.

This means that streaming hex to an ostream will set the output base formatting flags to output numbers in hexadecimal. The manipulator doesn't output anything itself.

---

Related Topics