How Do the Stream Manipulators Work

How do the stream manipulators work?

The standard defines the following operator<< overload in the basic_ostream class template:

basic_ostream<charT,traits>& operator<<(
    basic_ostream<charT,traits>& (*pf) (basic_ostream<charT,traits>&) );

Effects: None. Does not behave as a formatted output function (as described in 27.6.2.5.1).

Returns: pf(*this).

The parameter is a pointer to a function taking and returning a reference to a std::ostream.

This means that you can "stream" a function with this signature to an ostream object and it has the effect of calling that function on the stream. If you use the name of a function in an expression then it is (usually) converted to a pointer to that function.

std::hex is an std::ios_base manipulator defined as follows.

   ios_base& hex(ios_base& str);

Effects: Calls str.setf(ios_base::hex, ios_base::basefield).

Returns: str.

This means that streaming hex to an ostream will set the output base formatting flags to output numbers in hexadecimal. The manipulator doesn't output anything itself.

How can stream manipulators in C++ be functions?

Isn't endl a variable holding \n?

No, it is not. std::endl is a function defined in global namespace

  template<typename _CharT, typename _Traits>
    inline basic_ostream<_CharT, _Traits>& 
    endl(basic_ostream<_CharT, _Traits>& __os)
    { return flush(__os.put(__os.widen('\n'))); }

In expression std::cout << endl_or_something right hand side of << is an argument of a call to operator<< (first argument is std::ostream implicitly). So endl_or_something should be an int, double or other type that can be converted to one of the possible arguments of operator<<. There is an overloaded version of this operator that takes pointers to functions ( functions which take reference to std::ostream and return reference to std::ostream):

  // [27.6.2.5] formatted output
  // [27.6.2.5.3]  basic_ostream::operator<<
  //@{
  /**
   *  @brief  Interface for manipulators.
   *
   *  Manipulators such as @c std::endl and @c std::hex use these
   *  functions in constructs like "std::cout << std::endl".  For more
   *  information, see the iomanip header.
  */
  __ostream_type&
  operator<<(__ostream_type& (*__pf)(__ostream_type&))
  {
// _GLIBCXX_RESOLVE_LIB_DEFECTS
// DR 60. What is a formatted input function?
// The inserters for manipulators are *not* formatted output functions.
return __pf(*this);
  }

Since std::endl signature matches, it can be used in expression

std::cout << "Hello!" << std::endl;

or equivalently

std::cout << "Hello!";
std::endl( std::cout);

Note however that this manipulator is often mistakenly used when a simple newline is desired, leading to poor buffering performance. In such cases use just "\n".

---

Why this isn't allowed?:

cout << "Hello!" << endl();

std::endl takes one argument, std::ostream. You can see that it can be called with:

return __pf(*this);

means

return std::endl( *this); // std::endl( std::cout);

There is no version of std::endl that takes no parameters so it could be called with

std::endl()

In expression

std::cout << std::endl;

it denotes an argument to operator<<, it is passed as a pointer to function and then called in body of operator<<.

How does the setw stream manipulator work?

That's because you need to add a \n at the setw(18). And this applies to any setw.

Sample code:

# include <iostream>
# include <iomanip>
int main()
{
std::cout<<"\t\t"<<"a\n";
std::cout<<std::setw(18)<<"a\n"; // And you add the \n here
return 0;
}

Output:

        a
        a

And another solution is:

# include <iostream>
# include <iomanip>
int main()
{
std::cout<<"\t\t"<<"a\n";
std::cout<<std::setw(18)<<"a "; // And you add the a space here
return 0;
}

And the output will be the same.

---

The reason behind why should we put the \n or a space is because:

It's because it justifies the whole two-character string "a"\n, not only the a. If you print the newline separately (... << 'a' << '\n') then you would get the same "error". You could also have solved it by using almost any space character, like ... << "a "; You might want to print both with and without the newline in the same program to see the difference. - Some Programmer Dude

Custom stream manipulators

I found out that using std::ios_base::iword to store the requested character encoding was the best solution for the problem at hand:

#include <iostream>

/*!
\brief Unicode encoding
*/
enum EUnicodeEnc
{
    /** UTF-8 character encoding */
    EUnicodeEnc_UTF8 = 1,

    /** UTF-16 character encoding */
    EUnicodeEnc_UTF16 = 2,

    /** UTF-32 character encoding */
    EUnicodeEnc_UTF32 = 3
};

/** Allocate the \c std::ios_base::iword storage for use with \c SourceStreamEncoding object instances */
int SourceStreamEncoding::sourceEnc_xalloc = std::ios_base::xalloc();

/*!
\brief Stream I/O manipulator changes the source character encoding to UTF-8
*/
std::ios_base& FromUtf8(std::ios_base& os) {
    os.iword(SourceStreamEncoding::sourceEnc_xalloc) = EUnicodeEnc_UTF8;
    return os;
}

/*!
\brief Stream I/O manipulator changes the source character encoding to UTF-16
*/
std::ios_base& FromUtf16(std::ios_base& os) {
    os.iword(SourceStreamEncoding::sourceEnc_xalloc) = EUnicodeEnc_UTF16;
    return os;
}

/*!
\brief Stream I/O manipulator changes the source character encoding to UTF-32
*/
std::ios_base& FromUtf32(std::ios_base& os) {
    os.iword(SourceStreamEncoding::sourceEnc_xalloc) = EUnicodeEnc_UTF32;
    return os;
}

/*!
\brief Overrides \c std::ostream::flush()
\details Converts the buffer to the correct character encoding then flushes buffer
after writing its content to a storage device
*/
std::ostream &CFileManagerOStream::flush()
{
    switch (os.iword(SourceStreamEncoding::sourceEnc_xalloc))
    {
        case EUnicodeEnc_UTF8:
            characterEncoder.FromUTF8(...);
        break;
        case EUnicodeEnc_UTF16:
            characterEncoder.FromUTF16(...);
        break;
        case EUnicodeEnc_UTF32:
            characterEncoder.FromUTF32(...);
        break;
    }
    return (*this);
}

// Now I can do as follows:
int main()
{
    CFileManagerOStream outFile("MultipleUtf8Strings.dat"); // Custom std::ostream
    ...
#ifdef _WINDOWS
    CTcpStreamUtf16 largeBlobUtf16Stream;
    ...
    outFile << FromUtf16 << largeBlobUtf16Stream;
#else
    CTcpStreamUtf32 largeBlobUtf32Stream;
    ...
    outFile << FromUtf32 << largeBlobUtf32Stream;
#endif
}

Additionally, I've added the following manipulator that takes a single paramater:

class FromEnc
{
    public:
        explicit FromEnc(int i) : i_(i) {}
        int i_;
    private:
        template <class charT, class Traits>
        friend std::basic_ostream<charT, Traits>& operator<<(std::basic_ostream<charT, Traits>& os, const FromEnc& w) {
            os.iword(SourceStreamEncoding::sourceEnc_xalloc) = w.i_;
            return os;
        }
};

, so now I can also do as follows:

outFile << FromEnc(EUnicodeEnc_UTF16) << largeBlobUtf16Stream;

C++ custom stream manipulator that changes next item on stream

First, you have to store some state into each stream. You can do that with the function iword and an index you pass to it, given by xalloc:

inline int geti() { 
    static int i = ios_base::xalloc();
    return i;
}

ostream& add_one(ostream& os) { os.iword(geti()) = 1; return os; } 
ostream& add_none(ostream& os) { os.iword(geti()) = 0; return os; }

Having that in place, you can already retrieve some state in all streams. Now, you just have to hook into the respective output operation. Numeric output is done by a facet, because it potentially is locale dependent. So you can do

struct my_num_put : num_put<char> {
    iter_type 
    do_put(iter_type s, ios_base& f, char_type fill, long v) const { 
        return num_put<char>::do_put(s, f, fill, v + f.iword(geti())); 
    } 

    iter_type 
    do_put(iter_type s, ios_base& f, char_type fill, unsigned long v) const { 
        return num_put<char>::do_put(s, f, fill, v + f.iword(geti())); 
    } 
}; 

Now, you can test the stuff.

int main() {
    // outputs: 11121011
    cout.imbue(locale(locale(),new my_num_put));
    cout << add_one << 10 << 11 
         << add_none << 10 << 11;
}

If you want that only the next number is incremented, just set the word to 0 again after each call to do_put.

Chaining C++ stream manipulators into single variable

A stream manipulator is just a function that a stream calls on itself through one of the operator << overloads (10-12 in the link). You just have to declare such a function (or something convertible to a suitable function pointer):

constexpr auto m = [](std::ostream &s) -> std::ostream& {
    return s << std::setw(5) << std::scientific << std::left;
};
std::cout << m << 12.3 << '\n';

See it live on Wandbox

Parametrized custom stream manipulators - why overload operator ?

The overload you're looking for is only defined for function pointers.

basic_ostream& operator<<(
   std::basic_ostream<CharT,Traits>& (*func)(std::basic_ostream<CharT,Traits>&) );

Your print_my_data class is a callable object (a functor, in C++ terms). But it is not a function pointer. On the other hand, endl is a function, and hence has a function pointer (in fact, it's one of the few functions in the C++ standard library which do have an address)

A not-unreasonable argument could be made that the overload should look like

basic_ostream& operator<<(
   std::function<std::basic_ostream<CharT,Traits>&(std::basic_ostream<CharT,Traits>&)> func);

But alas, std::function wasn't around when the I/O manipulation operators were written. Neither were concepts, for that matter. And using SFINAE to declare

template <typename F>
basic_ostream& operator<<(
   F func);

would have just opened an entire Pandora's box worth of messy details that the standards committee didn't want to deal with.

Writing a manipulator for a custom stream class

Your manipulator should be declared as a function which accepts just one argument of type ostream&. However, if you make it a member function, you know there is an implicit this argument being passed to the function as well.

Thus, you should rather declare your manipulator as a free, non-member function, making it friend of your class so that it can access its private member ib:

class IndentStream : public ostream {
public:
    IndentStream(ostream &os) : ib(os.rdbuf()), ostream(&ib){};

    ostream& indent(ostream& stream) {
        ib.indent();
        return stream;
    }

    friend ostream& deindent(ostream& stream);
//  ^^^^^^

private:
    indentbuf ib;
};

ostream& deindent(ostream& stream)
{
    IndentStream* pIndentStream = dynamic_cast<IndentStream*>(&stream);
    if (pIndentStream != nullptr)
    {
        pIndentStream->ib.deindent();
    }

    return stream;
}

int main()
{
    IndentStream is(cout);
    is << "31 hexadecimal: " << hex << 31 << endl;
    is << "31 hexadecimal: " << hex << 31 << deindent << endl;
    is << "31 hexadecimal: " << hex << 31 << endl;
    return 0;
}

Alternatively, if you really want your function to be a member, you could make it static:

class IndentStream : public ostream {
public:
    IndentStream(ostream &os) : ib(os.rdbuf()), ostream(&ib){};

    ostream& indent(ostream& stream) {
        ib.indent();
        return stream;
    }

    static ostream& deindent(ostream& stream)
    {
        IndentStream* pIndentStream = dynamic_cast<IndentStream*>(&stream);
        if (pIndentStream != nullptr)
        {
            pIndentStream->ib.deindent();
        }

        return stream;
    }

private:
    indentbuf ib;
};

However, this would force you to use a qualified name to refer to it:

int main()
{
    IndentStream is(cout);
    is << "31 hexadecimal: " << hex << 31 << endl;
    is << "31 hexadecimal: " << hex << 31 << IndentStream::deindent << endl;
    //                                       ^^^^^^^^^^^^^^
    is << "31 hexadecimal: " << hex << 31 << endl;
    return 0;
}

---

Related Topics