Es6 Promises - Something Like Async.Each

JS: what's the promises equivalent of async.each?

Is there an official place for promises user docs (not the promises spec, actual example workflows and how you'd do them in promises) like there for async?

We have been putting together a page of general promise resources on the Q wiki. In terms of workflows and composition, I believe "How to Compose Node.js Promises with Q" would be the most helpful; it isn't really Node.js-specific, despite the title.

JavaScript ES6 promise for loop

As you already hinted in your question, your code creates all promises synchronously. Instead they should only be created at the time the preceding one resolves.

Secondly, each promise that is created with new Promise needs to be resolved with a call to resolve (or reject). This should be done when the timer expires. That will trigger any then callback you would have on that promise. And such a then callback (or await) is a necessity in order to implement the chain.

With those ingredients, there are several ways to perform this asynchronous chaining:

1. With a for loop that starts with an immediately resolving promise

2. With Array#reduce that starts with an immediately resolving promise

3. With a function that passes itself as resolution callback

4. With ECMAScript2017's async / await syntax

5. With ECMAScript2020's for await...of syntax

But let me first introduce a very useful, generic function.

Promisfying setTimeout

Using setTimeout is fine, but we actually need a promise that resolves when the timer expires. So let's create such a function: this is called promisifying a function, in this case we will promisify setTimeout. It will improve the readability of the code, and can be used for all of the above options:

const delay = ms => new Promise(resolve => setTimeout(resolve, ms));

See a snippet and comments for each of the options below.

1. With for

You can use a for loop, but you must make sure it doesn't create all promises synchronously. Instead you create an initial immediately resolving promise, and then chain new promises as the previous ones resolve:

const delay = ms => new Promise(resolve => setTimeout(resolve, ms));

for (let i = 0, p = Promise.resolve(); i < 10; i++) {
    p = p.then(() => delay(Math.random() * 1000))
         .then(() => console.log(i));
}

Promise.all vs [await x, await y] - Is it really the same?

No, you should not accept that:

return [await someFunction1(), await someFunction2()];

Is the same as:

return await Promise.all([someFunction1(), someFunction2()]);

^{I should also note that await in the above return await is not needed. Check out this blog post to learn more.}

They are different!

---

The first approach (sequential)

Let's determine the difference by inspecting how each of the two alternatives works.

[await someFunction1(), await someFunction2()];

Here, in an async context, we create an array literal. Note that someFunction1 is called (a function which probably returns a new promise each time it gets called).

So, when you call someFunction1, a new promise is returned, which then "locks" the async context because the preceding await.

In a nutshell, the await someFunction1() "blocks" the array initialization until the returned promise gets settled (by getting resolved or rejected).

The same process is repeated to someFunction2.

Note that, in this first approach, the two promises are awaited in sequence. There is, therefore, no similarity with the approach that uses Promise.all. Let's see why.

The second approach (non-sequential)

Promise.all([someFunction1(), someFunction2()])

When you apply Promise.all, it expects an iterable of promises. It waits for all the promises you give to resolve before returns a new array of resolved values, but don't wait each promise resolve until waiting another one. In essence, it awaits all the promises at the same time, so it is a kind of "non-sequential". As JavaScript is single-threaded, you cannot tell this "parallel", but is very similar in the behavior point of view.

So, when you pass this array:

[someFunction1(), someFunction2()]

You are actually passing an array of promises (which are returned from the functions). Something like:

[Promise<...>, Promise<...>]

Note that the promises are being created outside Promise.all.

So you are, in fact, passing an array of promises to Promise.all. When both of them gets resolved, the Promise.all returns the array of resolved values. I won't explain in all details how Promise.all works, for that, I suggest you checking out the documentation.

You can replicate this "non-sequential" approach by creating the promises before using the await. Like so:

const promise1 = someFunction1();
const promise2 = someFunction2();
return [await promise1, await promise2];

While promise1 is being waited, promise2 is already running (as it was created before the first await), so the behavior is similar to Promise.all's.

Using async/await with a forEach loop

Sure the code does work, but I'm pretty sure it doesn't do what you expect it to do. It just fires off multiple asynchronous calls, but the printFiles function does immediately return after that.

Reading in sequence

If you want to read the files in sequence, you cannot use forEach indeed. Just use a modern for … of loop instead, in which await will work as expected:

async function printFiles () {
  const files = await getFilePaths();

  for (const file of files) {
    const contents = await fs.readFile(file, 'utf8');
    console.log(contents);
  }
}

Reading in parallel

If you want to read the files in parallel, you cannot use forEach indeed. Each of the async callback function calls does return a promise, but you're throwing them away instead of awaiting them. Just use map instead, and you can await the array of promises that you'll get with Promise.all:

async function printFiles () {
  const files = await getFilePaths();

  await Promise.all(files.map(async (file) => {
    const contents = await fs.readFile(file, 'utf8')
    console.log(contents)
  }));
}

What is the difference between async/await forEach and Promise.all + map

The O.P.'s question was asking for clarification on another StackOverflow question found here. For further reading, and many other great answers on this general topic, please take a look at the link.

For Googlers who only saw the question title

Don't use async/await with forEach. Either use a for-of loop, or use Promise.all() with array.map().

If you have a general understanding on promises and async/await, the TL;DR on the differences between promise.all() + array.map() and .forEach(), is that it's impossible to await a forEach() itself. Yes, you can run tasks in parallel in .forEach() just like you can with .map(), but you can't wait for all of those parallel tasks to finish, then do something once they've all finished. The whole point of using .map() instead of .forEach() is so you can get a list of promises, collect them with Promise.all(), then await the whole thing. To see what I mean, just put a console.log('Finished') after a forEach(async () => ...), and you'll see the "finished" get logged out before everything has finished running in the .forEach() loop. My advice would be to just not use .forEach() with async logic (and really there's not a reason use ever use .forEach() anymore these days, as I explain further down).

For those who need something a bit more in depth, the rest of this answer will dive into more detail, first giving a small review on promises, then showing an in-depth explanation on how these approaches behave differently and why .forEach() is always the inferior solution when it comes to async/await.

A primer on promises and async/await

For the purposes of this discussion, you just have to remember that a promise is a special object that's promising that some task is going to be completed at some point in the future. You can attach listeners to a promise via .then(), to get notified when the task is completed, and to receive the resolved value.

An async function is simply a function that will always return a promise, no matter what. Even if you do async function doThing() { return 2 }, it's not going to return 2, it's going to return a promise that immediately resolves to the value 2. Note that an asynchronous function will always return a promise immediately, even if it takes a long time for the function to run. This is why it's called a "promise", it's promising that the function will eventually finish running, and if you want to be notified for when the function has finished, you can add an event listener to it, via .then() or await.

await is special syntax that lets you pause execution of an async function until a promise resolves. await will only effect the function it's directly inside. Behind the scenes, await is simply adding a special event listener to the promise's .then(), so it can know when the promise resolves and what value it resolves with.

async fn1() {
  async fn2() {
    await myPromise // This pauses execution of fn2(), not fn1()!
  }
  ...
}

async function fn1() {
  function fn2() {
    await myPromise // An error, because fn2() is not async.
  }
  ...
}

If you can get a good grasp of these principles, then you should be able to understand the next sections.

for-of

A for-of loop lets you execute your asynchronous tasks in serial, one after another. For example:

const delays = [1000, 1400, 1200];

// A function that will return a
// promise that resolves after the specified
// amount of time.
const wait = ms => new Promise(resolve => setTimeout(resolve, ms))

async function main() {
  console.log('start')
  for (const delay of delays) {
    await wait(delay)
    console.log('Finished waiting for the delay ' + delay)
  }
  console.log('finish')
}

main()

Any difference between await Promise.all() and multiple await?

Note:

This answer just covers the timing differences between await in series and Promise.all. Be sure to read @mikep's comprehensive answer that also covers the more important differences in error handling.

---

For the purposes of this answer I will be using some example methods:

• res(ms) is a function that takes an integer of milliseconds and returns a promise that resolves after that many milliseconds.
• rej(ms) is a function that takes an integer of milliseconds and returns a promise that rejects after that many milliseconds.

Calling res starts the timer. Using Promise.all to wait for a handful of delays will resolve after all the delays have finished, but remember they execute at the same time:

_{Example #1}

const data = await Promise.all([res(3000), res(2000), res(1000)])
//                              ^^^^^^^^^  ^^^^^^^^^  ^^^^^^^^^
//                               delay 1    delay 2    delay 3
//
// ms ------1---------2---------3
// =============================O delay 1
// ===================O           delay 2
// =========O                     delay 3
//
// =============================O Promise.all

async function example() {
  const start = Date.now()
  let i = 0
  function res(n) {
    const id = ++i
    return new Promise((resolve, reject) => {
      setTimeout(() => {
        resolve()
        console.log(`res #${id} called after ${n} milliseconds`, Date.now() - start)
      }, n)
    })
  }

  const data = await Promise.all([res(3000), res(2000), res(1000)])
  console.log(`Promise.all finished`, Date.now() - start)
}

example()

What is the best way to limit concurrency when using ES6's Promise.all()?

Note that Promise.all() doesn't trigger the promises to start their work, creating the promise itself does.

With that in mind, one solution would be to check whenever a promise is resolved whether a new promise should be started or whether you're already at the limit.

However, there is really no need to reinvent the wheel here. One library that you could use for this purpose is es6-promise-pool. From their examples:

var PromisePool = require('es6-promise-pool')
 
var promiseProducer = function () {
  // Your code goes here. 
  // If there is work left to be done, return the next work item as a promise. 
  // Otherwise, return null to indicate that all promises have been created. 
  // Scroll down for an example. 
}
 
// The number of promises to process simultaneously. 
var concurrency = 3
 
// Create a pool. 
var pool = new PromisePool(promiseProducer, concurrency)
 
// Start the pool. 
var poolPromise = pool.start()
 
// Wait for the pool to settle. 
poolPromise.then(function () {
  console.log('All promises fulfilled')
}, function (error) {
  console.log('Some promise rejected: ' + error.message)
})

How do I make this async foreach loop work with promises?

In synchronous code, continuation is performed when the line ends ;

With promises, continuation is performed via .then. You were using a promise constructor and resolved it immediately, you did not wait for any task at all. I'd map my work into tasks and then either chain them with then or await them serially.

//I'm assuming
zippyarray; // array of Zippy objects

var tasks = zippyarray.map(function(zippy,i){
    return function(){ // return a task on that zippy;
       // basic logic here
       return $.get({
            // ajax request
       }).then(function(data){
            // process data like in your code
            // possibly store later for later use too
            return process(data); // return the processed data;
       });
    }
});

Now we can execute them all sequentially:

 var p = tasks[0](); // start the first one
 for(var i = 1; i < tasks.length; i++) p = p.then(tasks[i]);
 p.then(function(result){
       // all available here
 });

Or better, serially:

$.when.apply(tasks.forEach(function(t){ return t(); })).then(function(results){
     // all done
})

---

Related Topics