How to synchronize a sequence of promises?
Here are some simple examples for how you sequence through an array executing each async operation serially (one after the other).
Let's suppose you have an array of items:
var arr = [...];
And, you want to carry out a specific async operation on each item in the array, one at a time serially such that the next operation does not start until the previous one has finished.
And, let's suppose you have a promise returning function for processing one of the items in the array fn(item):
Manual Iteration
function processItem(item) {
// do async operation and process the result
// return a promise
}
Then, you can do something like this:
function processArray(array, fn) {
var index = 0;
function next() {
if (index < array.length) {
fn(array[index++]).then(next);
}
}
return next();
}
processArray(arr, processItem);
Manual Iteration Returning Promise
If you wanted a promise returned from processArray() so you'd know when it was done, you could add this to it:
function processArray(array, fn) {
var index = 0;
function next() {
if (index < array.length) {
return fn(array[index++]).then(function(value) {
// apply some logic to value
// you have three options here:
// 1) Call next() to continue processing the result of the array
// 2) throw err to stop processing and result in a rejected promise being returned
// 3) return value to stop processing and result in a resolved promise being returned
return next();
});
}
} else {
// return whatever you want to return when all processing is done
// this returne value will be the ersolved value of the returned promise.
return "all done";
}
return next();
}
processArray(arr, processItem).then(function(result) {
// all done here
console.log(result);
}, function(err) {
// rejection happened
console.log(err);
});
Note: this will stop the chain on the first rejection and pass that reason back to the processArray returned promise.
Iteration with .reduce()
If you wanted to do more of the work with promises, you could chain all the promises:
function processArray(array, fn) {
return array.reduce(function(p, item) {
return p.then(function() {
return fn(item);
});
}, Promise.resolve());
}
processArray(arr, processItem).then(function(result) {
// all done here
}, function(reason) {
// rejection happened
});
Note: this will stop the chain on the first rejection and pass that reason back to the promise returned from processArray().
For a success scenario, the promise returned from processArray() will be resolved with the last resolved value of your fn callback. If you wanted to accumulate a list of results and resolve with that, you could collect the results in a closure array from fn and continue to return that array each time so the final resolve would be an array of results.
Iteration with .reduce() that Resolves With Array
And, since it now seems apparent that you want the final promise result to be an array of data (in order), here's a revision of the previous solution that produces that:
function processArray(array, fn) {
var results = [];
return array.reduce(function(p, item) {
return p.then(function() {
return fn(item).then(function(data) {
results.push(data);
return results;
});
});
}, Promise.resolve());
}
processArray(arr, processItem).then(function(result) {
// all done here
// array of data here in result
}, function(reason) {
// rejection happened
});
Working demo: http://jsfiddle.net/jfriend00/h3zaw8u8/
And a working demo that shows a rejection: http://jsfiddle.net/jfriend00/p0ffbpoc/
Iteration with .reduce() that Resolves With Array with delay
And, if you want to insert a small delay between operations:
function delay(t, v) {
return new Promise(function(resolve) {
setTimeout(resolve.bind(null, v), t);
});
}
function processArrayWithDelay(array, t, fn) {
var results = [];
return array.reduce(function(p, item) {
return p.then(function() {
return fn(item).then(function(data) {
results.push(data);
return delay(t, results);
});
});
}, Promise.resolve());
}
processArray(arr, 200, processItem).then(function(result) {
// all done here
// array of data here in result
}, function(reason) {
// rejection happened
});
Iteration with Bluebird Promise Library
The Bluebird promise library has a lot of concurrency controlling features built right in. For example, to sequence iteration through an array, you can use Promise.mapSeries().
Promise.mapSeries(arr, function(item) {
// process each individual item here, return a promise
return processItem(item);
}).then(function(results) {
// process final results here
}).catch(function(err) {
// process array here
});
Or to insert a delay between iterations:
Promise.mapSeries(arr, function(item) {
// process each individual item here, return a promise
return processItem(item).delay(100);
}).then(function(results) {
// process final results here
}).catch(function(err) {
// process array here
});
Using ES7 async/await
If you're coding in an environment that supports async/await, you can also just use a regular for loop and then await a promise in the loop and it will cause the for loop to pause until a promise is resolved before proceeding. This will effectively sequence your async operations so the next one doesn't start until the previous one is done.
async function processArray(array, fn) {
let results = [];
for (let i = 0; i < array.length; i++) {
let r = await fn(array[i]);
results.push(r);
}
return results; // will be resolved value of promise
}
// sample usage
processArray(arr, processItem).then(function(result) {
// all done here
// array of data here in result
}, function(reason) {
// rejection happened
});
FYI, I think my processArray() function here is very similar to Promise.map() in the Bluebird promise library which takes an array and a promise producing function and returns a promise that resolves with an array of resolved results.
@vitaly-t - Here some some more detailed comments on your approach. You are welcome to whatever code seems best to you. When I first started using promises, I tended to use promises only for the simplest things they did and write a lot of the logic myself when a more advanced use of promises could do much more of it for me. You use only what you are fully comfortable with and beyond that, you'd rather see your own code that you intimately know. That's probably human nature.
I will suggest that as I understood more and more of what promises can do for me, I now like to write code that uses more of the advanced features of promises and it seems perfectly natural to me and I feel like I'm building on well tested infrastructure that has lots of useful features. I'd only ask that you keep your mind open as you learn more and more to potentially go that direction. It's my opinion that it's a useful and productive direction to migrate as your understanding improves.
Here are some specific points of feedback on your approach:
You create promises in seven places
As a contrast in styles, my code has only two places where I explicitly create a new promise - once in the factory function and once to initialize the .reduce() loop. Everywhere else, I'm just building on the promises already created by chaining to them or returning values within them or just returning them directly. Your code has seven unique places where you're creating a promise. Now, good coding isn't a contest to see how few places you can create a promise, but that might point out the difference in leverage the promises that are already created versus testing conditions and creating new promises.
Throw-safety is a very useful feature
Promises are throw-safe. That means that an exception thrown within a promise handler will automatically reject that promise. If you just want the exception to become a rejection, then this is a very useful feature to take advantage of. In fact, you will find that just throwing yourself is a useful way to reject from within a handler without creating yet another promise.
Lots of Promise.resolve() or Promise.reject() is probably an opportunity for simplification
If you see code with lots of Promise.resolve() or Promise.reject() statements, then there are probably opportunities to leverage the existing promises better rather than creating all these new promises.
Cast to a Promise
If you don't know if something returned a promise, then you can cast it to a promise. The promise library will then do it's own checks whether it is a promise or not and even whether it's the kind of promise that matches the promise library you're using and, if not, wrap it into one. This can save rewriting a lot of this logic yourself.
Contract to Return a Promise
In many cases these days, it's completely viable to have a contract for a function that may do something asynchronous to return a promise. If the function just wants to do something synchronous, then it can just return a resolved promise. You seem to feel like this is onerous, but it's definitely the way the wind is blowing and I already write lots of code that requires that and it feels very natural once you get familiar with promises. It abstracts away whether the operation is sync or async and the caller doesn't have to know or do anything special either way. This is a nice use of promises.
The factory function can be written to create one promise only
The factory function can be written to create one promise only and then resolve or reject it. This style also makes it throw safe so any exception occuring in the factory function automatically becomes a reject. It also makes the contract to always return a promise automatic.
While I realize this factory function is a placeholder function (it doesn't even do anything async), hopefully you can see the style to consider it:
function factory(idx) {
// create the promise this way gives you automatic throw-safety
return new Promise(function(resolve, reject) {
switch (idx) {
case 0:
resolve("one");
break;
case 1:
resolve("two");
break;
case 2:
resolve("three");
break;
default:
resolve(null);
break;
}
});
}
If any of these operations were async, then they could just return their own promises which would automatically chain to the one central promise like this:
function factory(idx) {
// create the promise this way gives you automatic throw-safety
return new Promise(function(resolve, reject) {
switch (idx) {
case 0:
resolve($.ajax(...));
case 1:
resole($.ajax(...));
case 2:
resolve("two");
break;
default:
resolve(null);
break;
}
});
}
Using a reject handler to just return promise.reject(reason) is not needed
When you have this body of code:
return obj.then(function (data) {
result.push(data);
return loop(++idx, result);
}, function (reason) {
return promise.reject(reason);
});
The reject handler is not adding any value. You can instead just do this:
return obj.then(function (data) {
result.push(data);
return loop(++idx, result);
});
You are already returning the result of obj.then(). If either obj rejects or if anything chained to obj or returned from then .then() handler rejects, then obj will reject. So you don't need to create a new promise with the reject. The simpler code without the reject handler does the same thing with less code.
Here's a version in the general architecture of your code that tries to incorporate most of these ideas:
function factory(idx) {
// create the promise this way gives you automatic throw-safety
return new Promise(function(resolve, reject) {
switch (idx) {
case 0:
resolve("zero");
break;
case 1:
resolve("one");
break;
case 2:
resolve("two");
break;
default:
// stop further processing
resolve(null);
break;
}
});
}
// Sequentially resolves dynamic promises returned by a factory;
function sequence(factory) {
function loop(idx, result) {
return Promise.resolve(factory(idx)).then(function(val) {
// if resolved value is not null, then store result and keep going
if (val !== null) {
result.push(val);
// return promise from next call to loop() which will automatically chain
return loop(++idx, result);
} else {
// if we got null, then we're done so return results
return result;
}
});
}
return loop(0, []);
}
sequence(factory).then(function(results) {
log("results: ", results);
}, function(reason) {
log("rejected: ", reason);
});
Working demo: http://jsfiddle.net/jfriend00/h3zaw8u8/
Some comments about this implementation:
Promise.resolve(factory(idx))essentially casts the result offactory(idx)to a promise. If it was just a value, then it becomes a resolved promise with that return value as the resolve value. If it was already a promise, then it just chains to that promise. So, it replaces all your type checking code on the return value of thefactory()function.The factory function signals that it is done by returning either
nullor a promise whose resolved value ends up beingnull. The above cast maps those two conditions to the same resulting code.The factory function catches exceptions automatically and turns them into rejects which are then handled automatically by the
sequence()function. This is one significant advantage of letting promises do a lot of your error handling if you just want to abort processing and feed the error back on the first exception or rejection.The factory function in this implementation can return either a promise or a static value (for a synchronous operation) and it will work just fine (per your design request).
I've tested it with a thrown exception in the promise callback in the factory function and it does indeed just reject and propagate that exception back to reject the sequence promise with the exception as the reason.
This uses a similar method as you (on purpose, trying to stay with your general architecture) for chaining multiple calls to
loop().
Resolve promises one after another (i.e. in sequence)?
Update 2017: I would use an async function if the environment supports it:
async function readFiles(files) {
for(const file of files) {
await readFile(file);
}
};
If you'd like, you can defer reading the files until you need them using an async generator (if your environment supports it):
async function* readFiles(files) {
for(const file of files) {
yield await readFile(file);
}
};
Update: In second thought - I might use a for loop instead:
var readFiles = function(files) {
var p = Promise.resolve(); // Q() in q
files.forEach(file =>
p = p.then(() => readFile(file));
);
return p;
};
Or more compactly, with reduce:
var readFiles = function(files) {
return files.reduce((p, file) => {
return p.then(() => readFile(file));
}, Promise.resolve()); // initial
};
In other promise libraries (like when and Bluebird) you have utility methods for this.
For example, Bluebird would be:
var Promise = require("bluebird");
var fs = Promise.promisifyAll(require("fs"));
var readAll = Promise.resolve(files).map(fs.readFileAsync,{concurrency: 1 });
// if the order matters, you can use Promise.each instead and omit concurrency param
readAll.then(function(allFileContents){
// do stuff to read files.
});
Although there is really no reason not to use async await today.
JavaScript: Perform a chain of promises synchronously
A classic way to iterate over an array sequentually, calling some async operation on each array element is by using .reduce() and chaining to an initial promise as shown below:
The problem is I never know how many elements will be in the array, so
I would need a dynamic pattern.
You can use chaining of promises to sequence them one after. For this, it is useful to use .reduce() to iterate the array since it offers the right type of iteration that keeps track of an accumulated value (a promise in this case) as one iterates the array. You could make almost any array iteration scheme work using extra variables, this just lines up well with .reduce():
var ids = [1,2,3,4,5,6,7];
ids.reduce(function(p, item) {
return p.then(function() {
return doPromise(item);
});
}, Promise.resolve()).then(function(results) {
// all done here with array of results
});
This passes a resolved promise to ids.reduce() as the head of the promise chain. Then, it does a .then() on that promise and returns a new promise for each item in the array (via the .reduce() callback). The final result of the .reduce() call will be a promise which, when resolved means the entire chain is done.
The key to understanding this is to remember that p.then() returns a new promise so we just keep calling .then() on each new promise and we return the promise from each operation in each .then() handler. This has the effect of chaining all the promises together into one sequential chain.
Is it possible to mix the sync and async worlds, so that only one
request is active at a moment (sequence being not important)?
I'm not sure what you mean by "mix the sync and async worlds". The way to make sure that only one request is ever in-flight at a time is to sequence them to be one after the other so the next one starts only when the prior one finishes. This also happens to guarantee the execution order even though you say that isn't important in this case, but it is a by-product of making sure only one is in-flight at a time.
And, here's a working snippet:
function log(msg) { var d = document.createElement("div"); d.textContent = msg; document.body.appendChild(d);}
function doPromise(x) { return new Promise(function(resolve) { setTimeout(function() { log(x); resolve(x); }, Math.floor(Math.random() * 1000)); });}
var ids = [1,2,3,4,5,6,7];ids.reduce(function(p, item) { return p.then(function() { return doPromise(item); });}, Promise.resolve()).then(function() { // all done here log("all done");});How to execute promises in series?
Since you appear to be using the Bluebird promise library, you have a number of built-in options for sequencing your promise returning functions. You can use Promise.reduce(), Promise.map() with a concurrency value of 1, Promise.mapSeries or Promise.each(). If the iterator function returns a promise, all of these will wait for the next iteration until that promise resolves. Which to use depends more upon the mechanics of how your data is structured and what result you want (neither of which you actually show or describe).
Let's suppose you have an array of promise returning functions and you want to call them one at a time, waiting for the one to resolve before calling the next one. If you want all the results, then I'd suggest Promise.mapSeries():
let arrayOfPromiseReturningFunctions = [...];
// call all the promise returning functions in the array, one at a time
// wait for one to resolve before calling the next
Promise.mapSeries(arrayOfPromiseReturningFunctions, function(fn) {
return fn();
}).then(function(results) {
// results is an array of resolved results from all the promises
}).catch(function(err) {
// process error here
});
Promise.reduce() could also be used, but it would accumulate a single result, passing it from one to the next and end with one final result (like Array.prototype.reduce() does).
Promise.map() is a more general version of Promise.mapSeries() that lets you control the concurrency number (the number of async operations in flight at the same time).
Promise.each() will also sequence your functions, but does not accumulate a result. It assumes you either don't have a result or you are accumulating the result out-of-band or via side effects. I tend to not like to use Promise.each() because I don't like side effect programming.
How to make promises wait each other without using async/await pair?
You want to wait for your promises one by one, in order, so you should use recursion:
const doSomeCalls = (arrayWithNeededCalls) => {
const processCall = (index) => makeSomeCall(arrayWithNeededCalls[index]).then(res => {
console.log(`Call ${index} Completed going to next one`);
if (index < arrayWithNeededCalls.length) {
return processCall(index+1);
}
});
return processCall(0);
}
Executing promises in a sequence
Firstly, promises may or may not start executing immediately after creation.
(Note: by immediately, it is relative to the execution loop.)
In your above code, p1 to p5 starts counting down as soon as you leave the current execution loop.
To ensure the promises does not get executed before you want them to, you would have to wrap them in a promise generating function.
var p1 = function(){
return new Promise(function(resolve, reject){
console.log('p1');
setTimeout(resolve, 5000, "one");
});
};
var p2 = function(){
return new Promise(function(resolve, reject){
console.log('p2');
setTimeout(resolve, 3000, "two");
});
};
var p3 = function(){
return new Promise(function(resolve, reject){
console.log('p3');
setTimeout(resolve, 2000, "three");
});
};
var p4 = function(){
return new Promise(function(resolve, reject){
console.log('p4');
setTimeout(resolve, 1000, "four");
});
};
var p5 = function(){
return new Promise(function(resolve, reject){
console.log('p5');
setTimeout(resolve, 4000, "five");
});
};
For the above case, if you already know the number of promises you have, you can simply chain the promises together:
p1().then(p2).then(p3).then(p4).then(p5).then(function(){
// Hurray! All done!
console.log("All done :)");
});
However, if you have a variable amount of promises to chain in sequence, you would have to make use of loops (see @PitaJ's answer) or Array.reduce.
var arrayOfPromiseGeneratingFunctions = [p1, p2, p3, p4, p5]; // Can be of any size.
// Take the first promise-generating function as chain initializer
var initialFn = arrayOfPromiseGeneratingFunctions.shift();
// Here we're effectively doing promise chaining like the simple solution above
var finalPromise = arrayOfPromiseGeneratingFunctions.reduce(function(previousPromise, fn){
return previousPromise.then(fn)
}, initialFn());
finalPromise.then(function(){
// Last promise called
});
This solution is suitable for an arbitrary number of sequentially executed promises, so as long as you wrap the promises with a function.
Some other catches about this implementation:
1. In the example we wrap p1 to p5 into a function, so that they don't get executed in any way before you want it to.
2. You can pass results promise results from one to another, by adding the parameter in the promise-generating function. Since the first parameter would effectively be the resolve result of the last promise.
Further reading: https://github.com/kriskowal/q#sequences
for loop on promise don't follow the good order of output
Promise.all() is purposely used when you want things to run in parallel and it will tell you when all are done and they may each finish in any order.
There are lots of different ways to sequence things using promises. If you just have two function calls like your code shows, you can just do them manually:
ahaha(1).then(result => ahaha(2)).then(data => {
console.log("everything finished");
});
Or, a common pattern using .reduce():
[1,2].reduce(p, val => {
return p.then(() => ahaha(val));
}, Promise.resolve()).then(data => {
// everything done here
});
Or, my favorite using the Bluebird promise library:
Promise.mapSeries([1,2], ahaha).then(result => {
// everything done here
});
There are many other schemes, which you can see in these other answers:
How to synchronize a sequence of promises?
JavaScript: Perform a chain of promises synchronously
ES6 Promises - something like async.each?
How can I execute shell commands in sequence?
Can Promise load multi urls in order?
Synchronous loop in Promise all
Without using await (which is not in node.js v6.11.2, but would make this simpler), a classic pattern for serializing a bunch of async operations that return a promise is to use a reduce() loop like this:
arr.reduce(function(p, item) {
return p.then(function() {
return saveInDatabase(item).then((myResult) => ... );
});
}, Promise.resolve()).then(function() {
// all done here
}).catch(function(err) {
// error here
});
If you want to save all the results, you can use your .then(myResult => ...) handler to .push() the result into an array which you can access when done.
This will serialize all the calls to saveInDatabase(item) to it waits for the first one to be done before calling the second one, waits for the second one to be done before calling the third one, etc...
The default implementation here will stop if saveInDatabase(item) rejects. If you want to keep going (you don't say in your question), even when it gives an error, then you can add a .catch() to it to turn the rejected promise into a fulfilled promise.
In node.js v7+, you can use await in a regular for loop:
async function myFunc() {
let results = [];
for (let item of arr) {
let r = await saveInDatabase(item).then((myResult) => ... );
results.push(r);
}
return results;
}
myFunc().then(function(results) {
// all done here
}).catch(function(err) {
// error here
});
If you could run all the requests in parallel, then you could do that like this:
Promise.all(arr.map(item => {
return saveInDatabase(item).then((myResult) => ... );
})).then(function(results) {
// all done here
}).catch(function(err) {
// error here
});
In any of these, if you don't want it to stop upon a rejection, then add a .catch() to your saveInDatabase() promise chain to turn the rejection into a resolved promise with some known value or error value you can detect.
Related Topics
How Does JavaScript's Sort() Work
Difference Between Microtask and MACrotask Within an Event Loop Context
Jquery Load More Data on Scroll
Access Control Request Headers, Is Added to Header in Ajax Request with Jquery
Passing Environment-Dependent Variables in Webpack
Generic Deep Diff Between Two Objects
JavaScript Scroll Event for Iphone/Ipad
What's the Easiest Way to Call a Function Every 5 Seconds in Jquery
JavaScript Regex - Look Behind Alternative
How to Get a JavaScript Object's Class
How to Check a Radio Button with Jquery
How to Use Async/Await at the Top Level
Render Partial View Using Jquery in ASP.NET MVC
Find Out Whether Chrome Console Is Open
Javascript, Node.Js: Is Array.Foreach Asynchronous
Jquery.Parsejson Throws "Invalid JSON" Error Due to Escaped Single Quote in JSON