Convert Iterable to Stream using Java 8 JDK
There's a much better answer than using spliteratorUnknownSize
directly, which is both easier and gets a better result. Iterable
has a spliterator()
method, so you should just use that to get your spliterator. In the worst case, it's the same code (the default implementation uses spliteratorUnknownSize
), but in the more common case, where your Iterable
is already a collection, you'll get a better spliterator, and therefore better stream performance (maybe even good parallelism). It's also less code:
StreamSupport.stream(iterable.spliterator(), false)
.filter(...)
.moreStreamOps(...);
As you can see, getting a stream from an Iterable
(see also this question) is not very painful.
Make a Stream into an Iterable?
As explained in Why does Stream<T> not implement Iterable<T>?, an Iterable
bears the expectation to be able to provide an Iterator
more than once, which a Stream
can’t fulfill. So while you can create an Iterable
out of a Stream
for an ad-hoc use, you have to be careful about whether attempts to iterate it multiple times could exist.
Since you said, “I need to pass those parts of string as an Iterable
to a specific library”, there is no general solution as the code using the Iterable
is outside your control.
But if you are the one who creates the stream, it is possible to create a valid Iterable
which will simply repeat the stream construction every time an Iterator
is requested:
Iterable<String> lines = () -> "this\nthat\nthe_other".lines().iterator();
This fulfills the expectation of supporting an arbitrary number of iterations, while not consuming more resources than a single stream when being traversed only once.
for(var s: lines) System.out.println(s);
lines.forEach(System.out::println);
System.out.println(String.join("\n", lines));
How to convert an Iterator to a Stream?
One way is to create a Spliterator
from the Iterator
and use that as a basis for your stream:
Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();
Stream<String> targetStream = StreamSupport.stream(
Spliterators.spliteratorUnknownSize(sourceIterator, Spliterator.ORDERED),
false);
An alternative which is maybe more readable is to use an Iterable
- and creating an Iterable
from an Iterator
is very easy with lambdas because Iterable
is a functional interface:
Iterator<String> sourceIterator = Arrays.asList("A", "B", "C").iterator();
Iterable<String> iterable = () -> sourceIterator;
Stream<String> targetStream = StreamSupport.stream(iterable.spliterator(), false);
Easy way to convert Iterable to Collection
With Guava you can use Lists.newArrayList(Iterable) or Sets.newHashSet(Iterable), among other similar methods. This will of course copy all the elements in to memory. If that isn't acceptable, I think your code that works with these ought to take Iterable
rather than Collection
. Guava also happens to provide convenient methods for doing things you can do on a Collection
using an Iterable
(such as Iterables.isEmpty(Iterable)
or Iterables.contains(Iterable, Object)
), but the performance implications are more obvious.
Converting an AutoCloseable, Iterable class into a Stream
As stated in the javadoc, BaseStream.onClose()
"Returns an equivalent stream with an additional close handler":
public class WorkingExample {
public static void main(final String[] args) {
MyCursor cursor = new MyCursor();
try (Stream<String> stream = StreamSupport.stream(cursor.spliterator(), false)
.onClose(cursor::close)) {
stream.forEach(System.out::println);
}
}
}
will call MyCursor.close()
as desired.
Why does IterableT not provide stream() and parallelStream() methods?
This was not an omission; there was detailed discussion on the EG list in June of 2013.
The definitive discussion of the Expert Group is rooted at this thread.
While it seemed "obvious" (even to the Expert Group, initially) that stream()
seemed to make sense on Iterable
, the fact that Iterable
was so general became a problem, because the obvious signature:
Stream<T> stream()
was not always what you were going to want. Some things that were Iterable<Integer>
would rather have their stream method return an IntStream
, for example. But putting the stream()
method this high up in the hierarchy would make that impossible. So instead, we made it really easy to make a Stream
from an Iterable
, by providing a spliterator()
method. The implementation of stream()
in Collection
is just:
default Stream<E> stream() {
return StreamSupport.stream(spliterator(), false);
}
Any client can get the stream they want from an Iterable
with:
Stream s = StreamSupport.stream(iter.spliterator(), false);
In the end we concluded that adding stream()
to Iterable
would be a mistake.
What is the best/shortest way to convert an Iterable to a Stream, in Dart?
Here's a simple example:
var data = [1,2,3,4,5]; // some sample data
var stream = new Stream.fromIterable(data);
Using your code:
Future convert(thing) {
return someAsyncOperation(thing);
}
Stream doStuff(Iterable things) {
return new Stream.fromIterable(things
.map((t) async => await convert(t))
.where((value) => value != null));
}
How to convert an iterable to a stream?
Here's my streaming iterator an experimental branch of urllib3 supporting streaming chunked request via iterables:
class IterStreamer(object):
"""
File-like streaming iterator.
"""
def __init__(self, generator):
self.generator = generator
self.iterator = iter(generator)
self.leftover = ''
def __len__(self):
return self.generator.__len__()
def __iter__(self):
return self.iterator
def next(self):
return self.iterator.next()
def read(self, size):
data = self.leftover
count = len(self.leftover)
if count < size:
try:
while count < size:
chunk = self.next()
data += chunk
count += len(chunk)
except StopIteration:
pass
self.leftover = data[size:]
return data[:size]
Source with context:
https://github.com/shazow/urllib3/blob/filepost-stream/urllib3/filepost.py#L23
Related unit tests:
https://github.com/shazow/urllib3/blob/filepost-stream/test/test_filepost.py#L9
Alas this code hasn't made it into the stable branch yet as sizeless chunked requests are poorly supported, but it should be a good foundation for what you're trying to do. See the source link for examples showing how it can be used.
Why does StreamT not implement IterableT?
People have already asked the same on the mailing list ☺. The main reason is Iterable also has a re-iterable semantic, while Stream is not.
I think the main reason is that
Iterable
implies reusability, whereasStream
is something that can only be used once — more like anIterator
.If
Stream
extendedIterable
then existing code might be surprised when it receives anIterable
that throws anException
the
second time they dofor (element : iterable)
.
Related Topics
How to Straighten a Rotated Rectangle Area of an Image Using Opencv in Python
How to Use Hex() Without 0X in Python
Fitting a Normal Distribution to 1D Data
How to Get Exception Message in Python Properly
What Does a Leading '\X' Mean in a Python String '\Xaa'
Passing a Data Frame Column and External List to Udf Under Withcolumn
Default Filter in Django Admin
Handling Urllib2's Timeout? - Python
Reading the Target of a .Lnk File in Python
Transform "List of Tuples" into a Flat List or a Matrix
Preserving Global State in a Flask Application
Replace Invalid Values with None in Pandas Dataframe
Get First Row Value of a Given Column
Django Template System, Calling a Function Inside a Model
Differencebetween Join and Merge in Pandas