Node Js Promise.All and Foreach

Node JS Promise.all and forEach

It's pretty straightforward with some simple rules:

• Whenever you create a promise in a then, return it - any promise you don't return will not be waited for outside.
• Whenever you create multiple promises, .all them - that way it waits for all the promises and no error from any of them are silenced.
• Whenever you nest thens, you can typically return in the middle - then chains are usually at most 1 level deep.
• Whenever you perform IO, it should be with a promise - either it should be in a promise or it should use a promise to signal its completion.

And some tips:

• Mapping is better done with .map than with for/push - if you're mapping values with a function, map lets you concisely express the notion of applying actions one by one and aggregating the results.
• Concurrency is better than sequential execution if it's free - it's better to execute things concurrently and wait for them Promise.all than to execute things one after the other - each waiting before the next.

Ok, so let's get started:

var items = [1, 2, 3, 4, 5];
var fn = function asyncMultiplyBy2(v){ // sample async action
    return new Promise(resolve => setTimeout(() => resolve(v * 2), 100));
};
// map over forEach since it returns

var actions = items.map(fn); // run the function over all items

// we now have a promises array and we want to wait for it

var results = Promise.all(actions); // pass array of promises

results.then(data => // or just .then(console.log)
    console.log(data) // [2, 4, 6, 8, 10]
);

// we can nest this of course, as I said, `then` chains:

var res2 = Promise.all([1, 2, 3, 4, 5].map(fn)).then(
    data => Promise.all(data.map(fn))
).then(function(data){
    // the next `then` is executed after the promise has returned from the previous
    // `then` fulfilled, in this case it's an aggregate promise because of 
    // the `.all` 
    return Promise.all(data.map(fn));
}).then(function(data){
    // just for good measure
    return Promise.all(data.map(fn));
});

// now to get the results:

res2.then(function(data){
    console.log(data); // [16, 32, 48, 64, 80]
});

How to execute promises simultaneously with forEach?

1. First, move the code flow that deals with each individual user value into its own separate async function.

   • Because of how async functions actually work (as each async function marks the boundary of each async state-machine) it isn't really feasible to do concurrent-async work inside a for statement.

2. Invoke that function from within the for loop.

3. Always prefer === over ==.

4. Use TypeScript. Using async JavaScript without TypeScript is a world of pain because you need to keep track of which functions return Promise<T> vs. those that don't, as well as correctly unwrapping Promise<T> objects.

5. Always, wherever possible, have your own try/catch statement that wraps the entire body of an async function, otherwise you'll have to deal with inconsistencies with how different JS engines and JS code handles thrown exceptions and objects.

import usersjson from './jsons/users.json';

async function updateUsers() {
    
    const promises = [];

    const users = usersjson;
    for( const user of users ) {
        const promise = updateUser( user ); // <-- Note this is *NOT* awaited here.
        promises.push( promise );
    }

    await Promise.all( promises ); // <-- This is where we await everything.
}

async function updateUser( user ) {
    
    try {
        const data = await getUser( url, user.Id, 'Id' );
        if( data.users.length === 0 ) {
            let createResult = await newUser( url, user );
            console.log( "Created new user %o", createResult  );
        }
        else {
            const updateResult = await updateUser( url, user, data.users[0].id );
            console.log( "Update user %o: Result: %o", user.Id, updateResult );
        }
    }
    catch( err ) {
        console.error( "Failed to update (or create) user with UserId %o: %o", user.userId, err );
    }
}

Error using foreach and Promisse.all in node

The problem is that foreach does not return an array.
I would suggest to use map instead.

Check the code below:

const users = [1,2,3,4,5];

Promise.all(
  users.map(async (element) => {
    const userPopulated = {};
    userPopulated.hours = Promise.resolve(element);
    userPopulated.generalinfo = element;
    return element;
  })
 ).then((a) => console.log(a))

Why is Promise.all needed for Array.map Array.forEach asynchronous operation?

When you await an array of promises, you wait for an array which is not a promise (even though it contain promises inside). await will immediately resolve values that are not a Promise. It's like if you did await 'hello', that would be resolving instantly.

Promise.all is a util that exposes a new Promise that resolves only when all the promises in the array passed as an argument are resolved. Since it's creates it's own promise, you can await on Promise.all.

[EDIT]
Careful, do not use await in loops like this

for (var q of questions) {
  caseScore += await q.grade();
}

The reason is that it translates to

questions[0].grade().then(score => 
  return questions[1].grade().then(score => 
    return questions[2].grade().then(score => 
      return questions[3].grade().then(score => 
        // ...
      );
    );
  );
);

it creates a lots of context in memory and makes everything serial which is not taking advantage of javascript async nature

if you want to have a sum or some values shared between promises you can use a context object

async function grade1() {  this.sum += await getSumFromServer();  this.nbOfTasks += 1;}
async function grade2() {  this.sum += await getSumFromServer();  this.nbOfTasks += 1;}
async function grade3() {  this.sum += await getSumFromServer();  this.nbOfTasks += 1;}
async function main() {  const context = {    sum: 0,    nbOfTasks: 0,  }    const promisesWithContext = [grade1, grade2, grade3]    .map(fn => fn.call(context));
  await Promise.all(promisesWithContext);    console.log(context);}
main();
// STUBfunction getSumFromServer() {  return new Promise((resolve) => {    setTimeout(() => {      resolve(Math.random() * 100)    }, 1000)  });}

How to use promise all in lodash forEach and resolve it and do res.send(data)

If you use Object.entries instead of _.forEach and .map to an array of promises, you can use Promise.all on that:

 Promise.all(
   Object.entries(req.body).map(value => gridObj.save(value))
 ).then(/*..*/).catch(/*...*/)

Using Promise.all on my db.collection forEach

async function getUsersGroups(users, req) {
  await Promise.all(users.map(user => {
    return db.collection("groups")
      .find({ "users._id": String(user._id) })
      .toArray()
      .then(group => {
        user.groups = group;
      })
  }));

  return users;
}

Hope this helps

Promises being evaluated before Promise.all in nested forEach, resulting in empty Promise.all

Yeah so the problem I see is that your for each loop is calling asynchronous code and expecting it to execute synchronously.

I'd probably do something like...

var promises = list.map(element => {
    return otherModule.getSublist(element).then(sublist => {

        // Map all of the sublists into a promise
        return Promise.all(sublist.map(subelement => {
            return otherOtherModule.promiseResolvingFunction(subelement));
        }));
    });
});
return Promise.all(promises);

Of course then you'd end up with an array of arrays. If you wanted to keep the result a flat array of sublist items, another option would be to first fetch all of your lists, then fetch all of your sublists from those results...

return Promise.all(list.map( element => otherModule.getSublist(element)))
  .then((sublists) => {
    let subListPromises = [];

    // Loop through each sublist, turn each item in it into a promise
    sublists.forEach( sublist => {
        sublistPromises = [
          ...sublistPromises, 
          sublist.map( subelement => otherOtherModule.promiseResolvingFunction(subelement))
        ]
    })

    // Return a promise dependent on *all* of the sublist elements
    return Promise.all(sublistPromises)
  })

---

Related Topics