How to Convert an Iterable to a Stream

Convert Iterable to Stream using Java 8 JDK

There's a much better answer than using spliteratorUnknownSize directly, which is both easier and gets a better result. Iterable has a spliterator() method, so you should just use that to get your spliterator. In the worst case, it's the same code (the default implementation uses spliteratorUnknownSize), but in the more common case, where your Iterable is already a collection, you'll get a better spliterator, and therefore better stream performance (maybe even good parallelism). It's also less code:

StreamSupport.stream(iterable.spliterator(), false)
.filter(...)
.moreStreamOps(...);

As you can see, getting a stream from an Iterable (see also this question) is not very painful.

Make a Stream into an Iterable?

As explained in Why does Stream<T> not implement Iterable<T>?, an Iterable bears the expectation to be able to provide an Iterator more than once, which a Stream can’t fulfill. So while you can create an Iterable out of a Stream for an ad-hoc use, you have to be careful about whether attempts to iterate it multiple times could exist.

Since you said, “I need to pass those parts of string as an Iterable to a specific library”, there is no general solution as the code using the Iterable is outside your control.

But if you are the one who creates the stream, it is possible to create a valid Iterable which will simply repeat the stream construction every time an Iterator is requested:

Iterable<String> lines = () -> "this\nthat\nthe_other".lines().iterator();

This fulfills the expectation of supporting an arbitrary number of iterations, while not consuming more resources than a single stream when being traversed only once.

for(var s: lines) System.out.println(s);
lines.forEach(System.out::println);
System.out.println(String.join("\n", lines));

How to convert an Iterator to a Stream?

One way is to create a Spliterator from the Iterator and use that as a basis for your stream:

Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();
Stream<String> targetStream = StreamSupport.stream(
Spliterators.spliteratorUnknownSize(sourceIterator, Spliterator.ORDERED),
false);

An alternative which is maybe more readable is to use an Iterable - and creating an Iterable from an Iterator is very easy with lambdas because Iterable is a functional interface:

Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();

Iterable<String> iterable = () -> sourceIterator;
Stream<String> targetStream = StreamSupport.stream(iterable.spliterator(), false);

Easy way to convert Iterable to Collection

With Guava you can use Lists.newArrayList(Iterable) or Sets.newHashSet(Iterable), among other similar methods. This will of course copy all the elements in to memory. If that isn't acceptable, I think your code that works with these ought to take Iterable rather than Collection. Guava also happens to provide convenient methods for doing things you can do on a Collection using an Iterable (such as Iterables.isEmpty(Iterable) or Iterables.contains(Iterable, Object)), but the performance implications are more obvious.

Converting an AutoCloseable, Iterable class into a Stream

As stated in the javadoc, BaseStream.onClose() "Returns an equivalent stream with an additional close handler":

public class WorkingExample {
public static void main(final String[] args) {
MyCursor cursor = new MyCursor();
try (Stream<String> stream = StreamSupport.stream(cursor.spliterator(), false)
.onClose(cursor::close)) {
stream.forEach(System.out::println);
}
}
}

will call MyCursor.close() as desired.

Why does IterableT not provide stream() and parallelStream() methods?

This was not an omission; there was detailed discussion on the EG list in June of 2013.

The definitive discussion of the Expert Group is rooted at this thread.

While it seemed "obvious" (even to the Expert Group, initially) that stream() seemed to make sense on Iterable, the fact that Iterable was so general became a problem, because the obvious signature:

Stream<T> stream()

was not always what you were going to want. Some things that were Iterable<Integer> would rather have their stream method return an IntStream, for example. But putting the stream() method this high up in the hierarchy would make that impossible. So instead, we made it really easy to make a Stream from an Iterable, by providing a spliterator() method. The implementation of stream() in Collection is just:

default Stream<E> stream() {
return StreamSupport.stream(spliterator(), false);
}

Any client can get the stream they want from an Iterable with:

Stream s = StreamSupport.stream(iter.spliterator(), false);

In the end we concluded that adding stream() to Iterable would be a mistake.

What is the best/shortest way to convert an Iterable to a Stream, in Dart?

Here's a simple example:

var data = [1,2,3,4,5]; // some sample data
var stream = new Stream.fromIterable(data);

Using your code:

Future convert(thing) {
return someAsyncOperation(thing);
}

Stream doStuff(Iterable things) {
return new Stream.fromIterable(things
.map((t) async => await convert(t))
.where((value) => value != null));
}

How to convert an iterable to a stream?

Here's my streaming iterator an experimental branch of urllib3 supporting streaming chunked request via iterables:

class IterStreamer(object):
"""
File-like streaming iterator.
"""
def __init__(self, generator):
self.generator = generator
self.iterator = iter(generator)
self.leftover = ''

def __len__(self):
return self.generator.__len__()

def __iter__(self):
return self.iterator

def next(self):
return self.iterator.next()

def read(self, size):
data = self.leftover
count = len(self.leftover)

if count < size:
try:
while count < size:
chunk = self.next()
data += chunk
count += len(chunk)
except StopIteration:
pass

self.leftover = data[size:]

return data[:size]

Source with context:
https://github.com/shazow/urllib3/blob/filepost-stream/urllib3/filepost.py#L23

Related unit tests:
https://github.com/shazow/urllib3/blob/filepost-stream/test/test_filepost.py#L9

Alas this code hasn't made it into the stable branch yet as sizeless chunked requests are poorly supported, but it should be a good foundation for what you're trying to do. See the source link for examples showing how it can be used.

Why does StreamT not implement IterableT?

People have already asked the same on the mailing list ☺. The main reason is Iterable also has a re-iterable semantic, while Stream is not.

I think the main reason is that Iterable implies reusability, whereas Stream is something that can only be used once — more like an Iterator.

If Stream extended Iterable then existing code might be surprised when it receives an Iterable that throws an Exception the
second time they do for (element : iterable).



Related Topics



Leave a reply



Submit