How the New Range-Based for Loop in C++17 Helps Ranges Ts

How the new range-based for loop in C++17 helps Ranges TS?

C++11/14 range-for was overconstrained...

The WG21 paper for this is P0184R0 which has the following motivation:

The existing range-based for loop is over-constrained. The end
iterator is never incremented, decremented, or dereferenced. Requiring
it to be an iterator serves no practical purpose.

As you can see from the Standardese that you posted, the end iterator of a range is only used in the loop-condition __begin != __end;. Hence end only needs to be equality comparable to begin, and it does not need to be dereferenceable or incrementable.

...which distorts operator== for delimited iterators.

So what disadvantage does this have? Well, if you have a sentinel-delimited range (C-string, line of text, etc.), then you have to shoehorn the loop-condition into the iterator's operator==, essentially like this

#include <iostream>

template <char Delim = 0>
struct StringIterator
{
    char const* ptr = nullptr;   

    friend auto operator==(StringIterator lhs, StringIterator rhs) {
        return lhs.ptr ? (rhs.ptr || (*lhs.ptr == Delim)) : (!rhs.ptr || (*rhs.ptr == Delim));
    }

    friend auto operator!=(StringIterator lhs, StringIterator rhs) {
        return !(lhs == rhs);
    }

    auto& operator*()  {        return *ptr;  }
    auto& operator++() { ++ptr; return *this; }
};

template <char Delim = 0>
class StringRange
{
    StringIterator<Delim> it;
public:
    StringRange(char const* ptr) : it{ptr} {}
    auto begin() { return it;                      }
    auto end()   { return StringIterator<Delim>{}; }
};

int main()
{
    // "Hello World", no exclamation mark
    for (auto const& c : StringRange<'!'>{"Hello World!"})
        std::cout << c;
}

Live Example with g++ -std=c++14, (assembly using gcc.godbolt.org)

The above operator== for StringIterator<> is symmetric in its arguments and does not rely on whether the range-for is begin != end or end != begin (otherwise you could cheat and cut the code in half).

For simple iteration patterns, the compiler is able to optimize the convoluted logic inside operator==. Indeed, for the above example, the operator== is reduced to a single comparison. But will this continue to work for long pipelines of ranges and filters? Who knows. It is likely to require heroic optimization levels.

C++17 will relax the constraints which will simplify delimited ranges...

So where exactly does the simplification manifest itself? In operator==, which now has extra overloads taking an iterator/sentinel pair (in both orders, for symmetry). So the run time logic becomes compile time logic.

#include <iostream>

template <char Delim = 0>
struct StringSentinel {};

struct StringIterator
{
    char const* ptr = nullptr;   

    template <char Delim>
    friend auto operator==(StringIterator lhs, StringSentinel<Delim> rhs) {
        return *lhs.ptr == Delim;
    }

    template <char Delim>
    friend auto operator==(StringSentinel<Delim> lhs, StringIterator rhs) {
        return rhs == lhs;
    }

    template <char Delim>
    friend auto operator!=(StringIterator lhs, StringSentinel<Delim> rhs) {
        return !(lhs == rhs);
    }

    template <char Delim>
    friend auto operator!=(StringSentinel<Delim> lhs, StringIterator rhs) {
        return !(lhs == rhs);
    }

    auto& operator*()  {        return *ptr;  }
    auto& operator++() { ++ptr; return *this; }
};

template <char Delim = 0>
class StringRange
{
    StringIterator it;
public:
    StringRange(char const* ptr) : it{ptr} {}
    auto begin() { return it;                      }
    auto end()   { return StringSentinel<Delim>{}; }
};

int main()
{
    // "Hello World", no exclamation mark
    for (auto const& c : StringRange<'!'>{"Hello World!"})
        std::cout << c;
}

Live Example using g++ -std=c++1z (assembly using gcc.godbolt.org, which is almost identical to the previous example).

...and will in fact support fully general, primitive "D-style" ranges.

WG21 paper N4382 has the following suggestion:

C.6 Range Facade and Adaptor Utilities [future.facade]

1 Until it
becomes trivial for users to create their own iterator types, the full
potential of iterators will remain unrealized. The range abstraction
makes that achievable. With the right library components, it should be
possible for users to define a range with a minimal interface (e.g.,
current, done, and next members), and have iterator types
automatically generated. Such a range facade class template is left as
future work.

Essentially, this is equal to D-style ranges (where these primitives are called empty, front and popFront). A delimited string range with only these primitives would look something like this:

template <char Delim = 0>
class PrimitiveStringRange
{
    char const* ptr;
public:    
    PrimitiveStringRange(char const* c) : ptr{c} {}
    auto& current()    { return *ptr;          }
    auto  done() const { return *ptr == Delim; }
    auto  next()       { ++ptr;                }
};

If one does not know the underlying representation of a primitive range, how to extract iterators from it? How to adapt this to a range that can be used with range-for? Here's one way (see also the series of blog posts by @EricNiebler) and the comments from @T.C.:

#include <iostream>

// adapt any primitive range with current/done/next to Iterator/Sentinel pair with begin/end
template <class Derived>
struct RangeAdaptor : private Derived
{      
    using Derived::Derived;

    struct Sentinel {};

    struct Iterator
    {
        Derived*  rng;

        friend auto operator==(Iterator it, Sentinel) { return it.rng->done(); }
        friend auto operator==(Sentinel, Iterator it) { return it.rng->done(); }

        friend auto operator!=(Iterator lhs, Sentinel rhs) { return !(lhs == rhs); }
        friend auto operator!=(Sentinel lhs, Iterator rhs) { return !(lhs == rhs); }

        auto& operator*()  {              return rng->current(); }
        auto& operator++() { rng->next(); return *this;          }
    };

    auto begin() { return Iterator{this}; }
    auto end()   { return Sentinel{};     }
};

int main()
{
    // "Hello World", no exclamation mark
    for (auto const& c : RangeAdaptor<PrimitiveStringRange<'!'>>{"Hello World!"})
        std::cout << c;
}

Live Example using g++ -std=c++1z (assembly using gcc.godbolt.org)

Conclusion: sentinels are not just a cute mechanism to press delimiters into the type system, they are general enough to support primitive "D-style" ranges (which themselves may have no notion of iterators) as a zero-overhead abstraction for the new C++1z range-for.

Why did the range based 'for' loop specification change in C++17?

Using

auto __begin = begin_expr, __end = end_expr;

requires both begin_expr and end_expr to return the same type. This means you cannot have a sentinel iterator type that is different from the beginning type. Using

auto __begin = begin_expr ;
auto __end = end_expr ;

fixes that issue while proving full backwards compatibility with C++14.

How is the range-based loop different to a for-each loop?

Syntax:

for ( range_declaration : range_expression) loop_statement  

produces code equivalent to:

{
    auto && __range = range_expression ; 
    auto __begin = begin_expr(__range); 
    auto __end = end_expr(__range); 
    for (;__begin != __end; ++__begin) { 
        range_declaration = *__begin; 
        loop_statement 
    } 
} 

While the std::for_each applies unary function to the specified range. So, there are two basic differences:

• The range-based for-loop syntax is cleaner and more universal, but you can't execute the code in loop for a specified range different than from begin() to end().
• Range-based for-loop can be applied to containers which don't have iterators defined, by defining begin() and end() functions.

You cannot compare it to the "generalized for-each idiom", because there is no standard idiom. To compare, you have to point out the concrete implementation and the difference is usually hidden in the details.

Why is a type of range based for loop over brace-init list illegal c++

It does not work because it isn't supposed to work. That's the design of the feature. That feature being list initialization, which as the name suggests is about initializing something.

When C++11 introduced initializer_list, it was done for precisely one purpose: to allow the system to generate an array of values from a braced-init-list and pass them to a properly-tagged constructor (possibly indirectly) so that the object could initialize itself with that sequence of values. The "proper tag" in this case being that the constructor took the newly-minted std::initializer_list type as its first/only parameter. That's the purpose of initializer_list as a type.

Initialization, broadly speaking, should not modify the values it is given. The fact that the array backing the list is a temporary also doubles-down on the idea that the input values should logically be non-modifiable. If you have:

std::vector<int> v = {1, 2, 3, 4, 5};

We want to give the compiler the freedom to make that array of 5 elements a static array in the compiled binary, rather than a stack array that bloats the stack size of this function. More to the point, we logically want to think of the braced-init-list like a literal.

And we don't allow literals to be modified.

Your attempt to make {a, b, c, d} into a range of modifiable references is essentially trying to take a construct that already exists for one purpose and turn it into a construct for a different purpose. You're not initializing anything; you're just using a seemingly convenient syntax that happens to make iterable lists of values. But that syntax is called a "braced-init-list", and it generates an initializer list; the terminology is not an accident.

If you take a tool intended for X and try to hijack it do Y, you're likely to encounter rough edges somewhere down the road. So the reason why it doesn't work is that this it's not meant to; these are not what braced-init-lists and initializer_lists are for.

You might next say "if that's the case, why does for(auto i: {1, 2, 3, 4, 5}) work at all, if braced-init-lists are only for initialization?"

Once upon a time, it didn't; in C++11, that would be il-formed. It was only in C++14 when auto l = {1, 2, 3, 4}; became legal syntax; an auto variable was allowed to automatically deduce a braced-init-list as an initializer_list.

Range-based for uses auto deduction for the range type, so it inherited this ability. This naturally led people to believe that braced-init-lists are about building ranges of values, not initializing things.

In short, someone's convenience feature led you to believe that a construct meant to initialize an objects is just a quick way to create an array. It never was.

---

Having established that braced-init-lists aren't supposed to do the thing you want them to do, what would it take to make them do what you want?

Well, there are basically two ways to do it: small scale and large scale. The large-scale version would be to change how auto i = {a, b, c, d}; works, so that it could (based on something) create a modifiable range of references to expressions. Range-based for would just use its existing auto-deduction machinery to pick up on it.

This is of course a non-starter. That definition already has a well-defined meaning: it creates a non-modifiable list of copies of those expressions, not references to their results. Changing it would be a breaking change.

A small-scale change would be to hack range-based for to do some fancy deduction based on whether the range expression is a braced-init-list or not. Now, because such ranges and their iterators are buried by the compiler-generated code for range-based for, you won't have as many backwards compatibility problems. So you could do make a rule where, if your range-statement defines a non-const reference variable, and the range-expression is a braced-init-list, then you do some different deduction mechanisms.

The biggest problem here is that it's a complete and total hack. If it's useful to do for(auto &i : {a, b, c d}), then it's probably useful to be able to get the same kind of range outside of a range-based for loop. As it currently stands, the rules about auto-deduction of braced-init-lists are consistent everywhere. Range-based for gains its capabilities simply because it uses auto deduction.

The last thing C++ needs is more inconsistency.

This is doubly important in light of C++20 adding an init-statement to range-for. These two things ought to be equivalent:

for(auto &i : {a, b, c, d})
for(auto &&rng = {a, b, c, d}; auto &i: rng)

But if you change the rules only based on the range-expression and range-statement, they wouldn't be. rng would be deduced according to existing auto rules, thus making the auto &i non-functional. And having a super-special-case rule that changes how the init-statement of a range-for behaves, different from the same statement in other locations, would be even more inconsistent.

Besides, it's not too difficult to write a generic reference_range function that takes non-const variadic reference parameters (of the same type) and returns some kind of random-access range over them. That will work everywhere equally and without compatibility problems.

So let's just avoid trying to make a syntactic construct intended for initializing objects into a generic "list of stuff" tool.

Range based for loop: Iterate over vector extended with one element

It can be done using the upcoming ranges feature.

Here's an example using Eric Niebler's range-v3 library:

#include <iostream>
#include <vector>

#include <range/v3/view/concat.hpp>
#include <range/v3/view/single.hpp>

int main() {
    std::vector<int> a = {1, 5, 3};
    int additional = 6;
    for (auto i : ranges::concat_view(ranges::single_view{additional}, a)) {
        std::cout << i;
    }
}

See it live!

by using views, all iterator operations are lazy, and no extra memory is used (e.g.: no extra vectors/arrays are created)

Or, without the for loop:

ranges::copy(ranges::concat_view(ranges::single_view{additional}, a), ranges::make_ostream_joiner(std::cout, ","));

See it live!

(Honestly, I like the for version better, though)

Standard-compliant solution

There's a small issue with the solution above: concat_view did not make it into C++20. If you want a strictly compliant solution, you may want to create your own version, or use join_view instead:

#include <iostream>
#include <vector>

#include <ranges>

int main() {
    std::vector<int> a = {1, 5, 3};
    int additional = 6;

    std::vector v{{additional}, a};

    for(int i : std::ranges::join_view{v}) {
        std::cout << i;
    }
}

Is there a range class in C++11 for use with range based for loops?

The C++ standard library does not have one, but Boost.Range has boost::counting_range, which certainly qualifies. You could also use boost::irange, which is a bit more focused in scope.

C++20's range library will allow you to do this via view::iota(start, end).

How can I create new style for creating loop?

My recommendation is that you use iterators, I wrote an example below. Something like this would do it in C++ < C++17:

#include <iostream>

template <class T>
struct RangeIter {
    RangeIter(T from, T to, T curr ) :
    _from(from), _to(to), _curr(curr) {
    }

    T operator*() const { 
      return _curr;
    }

    T operator++() {
      ++_curr;
      return _curr;
    }

    bool operator==(const RangeIter & other) {
      assert(_from == other._from && _to == other._to);
      return _curr == other._curr;
    }

    bool operator!=(const RangeIter & other) {
      return !(_curr == other._curr);
    }
    T _from, _to, _curr;
  };

template <class T>
struct Range {
  Range(T from, T to) : _from(from), _to(to) {}

  RangeIter<T> begin() { return RangeIter<T>(_from, _to, _from); }
  RangeIter<T> end() { 
    return RangeIter<T>(_from, _to, _to); 
  }

  T _from, _to; 
};

template <class T>
Range<T> makeRange(T to, T from) {
  return Range<T>(to, from);
}

int main() {

    for (auto i : makeRange(0, 10)) {
       std::cout << i << std::endl;
    }
}

For C++17 you can use different types for the begin and end iterator and improve on this. You can use sentinels. You could take a look here: How the new range-based for loop in C++17 helps Ranges TS?

A C++-17 only solution here:

#include <iostream>

template <class T>
struct RangeSentinel {
   RangeSentinel(T stopVal) : _stopVal(stopVal) {}

   T _stopVal;
};

template <class T>
struct RangeIter {
    RangeIter(T from, T to, T curr) :
    _from(from), _to(to), _curr(curr) {
    }

    T operator*() const { 
      return _curr;
    }

    T operator++() {
      ++_curr;
      return _curr;
    }

    bool operator==(const RangeSentinel<T> & other) {
      assert(_from == other._from && _to == other._to);
      return _curr == other._stopVal;
    }

    bool operator!=(const RangeSentinel<T> & other) {
      return !(_curr == other._stopVal);
    }
    T _from, _to, _curr;
  };

template <class T>
struct Range {
  Range(T from, T to) : _from(from), _to(to) {}

  RangeIter<T> begin() { return RangeIter<T>(_from, _to, _from); }
  RangeSentinel<T> end() { 
    return RangeSentinel<T>(_to); 
  }

  T _from, _to; 
};

template <class T>
Range<T> makeRange(T to, T from) {
  return Range<T>(to, from);
}

int main() {

    for (auto i : makeRange(0, 10)) {
       std::cout << i << std::endl;
    }
}

As you can see, in the C++17 solution I do not need to store again _from and _to variables, since the sentinel is a different type.

---

Related Topics