Understanding Promises in Node.Js

Understanding promises in Node.js

Promises in node.js promised to do some work and then had separate callbacks that would be executed for success and failure as well as handling timeouts. Another way to think of promises in node.js was that they were emitters that could emit only two events: success and error.

The cool thing about promises is you can combine them into dependency chains (do Promise C only when Promise A and Promise B complete).

By removing them from the core node.js, it created possibility of building up modules with different implementations of promises that can sit on top of the core. Some of these are node-promise and futures.

Missing something with understanding promises for linear code processes?

You can await your promise and store the result in a variable.

const account = await getAcct();

You will need to run your await inside of an async function and the await keyword will instruct the function to pause execution until the promise has resolved.

clarification about promises in javascript

Basic concepts

The whole point of the event loop is to have many microtasks that do not affect each other (hence by default there is no effect).

To chain microtasks first; there were callbacks, then Promises (then/catch) then the async/await API. The last two can be considered just syntactic sugar atop the callback concept. There are no 'functionalities' added but rather a different syntax to achieve the same stuff in simpler and more elegant ways (Pyhilosophicaly).

The event loop executes all the queued microtasks at each loop and repeats. Unless you have blocking code (and await is not to be considered blocking) your event loop never stalls and hence other tasks are not affected.

You are trying to understand await from the perspective of real blocking code as intended in other languages.

IMHO you first need to deeply understand how callbacks work, then study Promises (as a tool to make callbacks less messy) and then async/await (as a syntax to make Promises pretties). But keep in mind, the underlying system is the same: functions that call functions that get handled functions to be eventually called in future).

Specific questions

When a promise is created is starts running

Yes, but no. A promise does not run, a promise is only a contract you receive by a part of code that will be used to notify you of the outcome. So the promise does not run, is the mean that has been created for you after you requested a task to be executed.

So typically if a promise has been handled to you there is something 'running'. But Promise may be used differently and there may be something 'waiting'.

A promise is not linked to the execution of the task hence it can not start nor stop it.

Do I need to await on the promise if I'm not interested in the outcome

No, you are not required to. But keep in mind that not handling promise exceptions is being deprecated and may result in system failure. You should always handle (or let bubble) exceptions.

There would be a failure if there is an unhandled promise rejection. In synchronous code this is equivalent to an uncaught thrown error. Until now(-ish) uncaught promise rejections were tolerated but there isn't a really good reason for that. Node is moving to treat them the same as any other error that bubbles to the top.
VLAZ

You are considering promises only with async/await but the underlying Promise api is .then() and .catch(). Using this API you can use promises in a 'fire-and-forget' fashion:

async function Do() {
  await before();
  asyncDbCall().catch(err => console.error(err))
  await after();
}

In this example you are not waiting for asyncDbCall() but still .catch(err => console.error(err)) will result in the error being logged (some time in the future, probably even after Do() has completed).

Or you can branch off the execution to other async executions, take this complex example:

async function Do() {
    await before();
    // This will execute AFTER before() & only if before() succeeded
    asyncDbCall()
        .then(async value => {
            // This will execute after `asyncDbCall()` and only if it succeeded
            await something(value);
            // We reach here after `something()` and only if succeeded
        })
        .catch(err => {
            // This will execute if `asyncDbCall()` fails of IF ANYTHING
            // within `async value => {}` fails
            console.error(err);
        })
    // This will execute AFTER before() and only if before() succeeded and
    // asyncDbCall() call (but not Promise) succeeded
    await after();
}

Await it actually blocks the execution of the async function

Await stops the async function (hence also anything that is awaiting for the function) but does not affect anyway the event loop.

Promises execute when defined instead of Promises.all?

As mentioned in comments, promise objects actually get in "pending" state right after you executed an asynchronous operation and are "fulfilled" as soon as operation finishes.

So in your code you actually create and run async operations for all users, not only 20 of them.

There are two solutions for you. The main one is to create functions which return a promises and run only 20 at one time.

const jobs = users.map(user => () => user.update(...)); // array of functions
while (jobs.length > 0) {
   await Promise.all(jobs.splice(0,20).map(job => job())); // take 20 and process
}

An alternative solution is to use a library like bluebird which has a lot of useful methods for working with promises. One you'd probably like is map() which support concurrency limit.

Your second question was regarding errors in Promise.all what results in a fail of the whole series.
In order to prevent that you may add .catch() to each job and, for example, return a null there or any other value which then will help you to determine that some operations failed. Of course, this approach will also prevent Promise.all from interrupting.

const jobs = users.map(user => () => user.update().catch(e => null));

Node.js - Promise

If you return a Promise, you have to define when it is resolved and when it's rejected.

class developer {
  constructor(name, language) {
    this.name = name
    this.role = language
  }

  getName() {
    return this.name
  }

  setName(name) {
    this.name = name
  }

  getLanguage() {
    return this.language
  }

  setLanguage(language) {
    this.language = language
  }

  code() {
    this.getName();
    this.getLanguage();

    return new Promise((resolve, reject) => {

      if (this.language === 'nodejs') {
        resolve('console.log("Hello, ' + this.name + '!")');
      } else if (this.language === 'java') {
        resolve('System.out.println("Hello, ' + this.name + '!")');
      } else if (this.language === 'python') {
        resolve('print("Hello, ' + this.name + '!")');
      } else {
        reject('Unsupported language: ' + this.language);
      }
    });
  }
}

let mydeveloper = new developer('John Doe', 'English');

// Will print out -> Unsupported language: undefined
mydeveloper
  .code()
  .then((res) => {
    console.log(res);
  })
  .catch((err) => {
    console.log(err);
  });

// Set language to nodejs
mydeveloper.setLanguage('nodejs');

// Will print out -> console.log("Hello, John Doe!")
mydeveloper
  .code()
  .then((res) => {
    console.log(res);
  })
  .catch((err) => {
    console.log(err);
  });

// Set language to java
mydeveloper.setLanguage('java');

// Will print out -> System.out.println("Hello, John Doe!")
mydeveloper
  .code()
  .then((res) => {
    console.log(res);
  })
  .catch((err) => {
    console.log(err);
  });

// Set language to python
mydeveloper.setLanguage('python');

// Will print out -> print("Hello, John Doe!")
mydeveloper
  .code()
  .then((res) => {
    console.log(res);
  })
  .catch((err) => {
    console.log(err);
  });

Node.js Promise chain resolving too early

Because you haven't resolved the promise created sleep(i).then to the promise from the second sleep, so it doesn't wait for that second operation to finish before settling. You need a return in front of the second call:

function sleep(cycle) {
    console.log(`Cycle ${ cycle } is going to sleep`);

    return new Promise(resolve => {
        setTimeout(() => {
            console.log(`Cycle ${ cycle } slept for 2 seconds`);
            resolve();
        }, 2000);
    });
}

function process() {
    let cycles       = 1;
    let subprocesses = [];

    for (let i = 0; i < cycles; i++) {
        subprocesses.push(
            sleep(i).then(() => {
                console.log('Returned from sleep function');

                return sleep(i); // <============================
            })
        );
    }

    Promise.allSettled(subprocesses).then(results => {
        console.log('Done with the process');
    });
}

process();

JavaScript Dynamic Promises

Some issues:

A missing return statement in getVehicleName
A syntax issue in getVehicleName[vehicles_URL[i]] (should be parentheses)
As the promises for getting the vehicle names are independent, you would not chain them, but use Promise.all
arrVehicleData will always only have one element. There is no reason for an array there where it is used.

You are also taking the wrong approach in using request.get. The bottom function turns that API from a Promise-API to a callback API, only to do the reverse (from callback to promise) in the function just above it. You should just skip the callback layer and stick to promises:

"use strict";

const request = require("request-promise");
const BASE_URL = "http://swapi.dev/api";

getJson(`${BASE_URL}/people/1`).then(data => {
  return Promise.all(data.vehicles.map(getVehicleName));
}).then(vehicleNames => {
  console.log(vehicleNames);
  // Continue here...
});

function getVehicleName(url) {
  return getJson(url).then(vehicle => vehicle.name);
}

function getJson(url, callback) {
  return request.get(url).then(JSON.parse);
}

Finally, you should not use request-promise anymore since the request module, on which request-promise depends, has been deprecated

Node.js: When to use Promises vs Callbacks

First off, you pretty much never want to write code that is a mix of callbacks and promises for async operations. If you're moving to promises or introducing some promises, then you probably want to refactor the callbacks in that same section of code into promises. For the appropriate types of operations, there are so many advantages of promises over plain callbacks that it is well worth the effort to convert when already working in an area of code.

Promises are great for:

Monitoring synchronous operations
That need to notify only once (usually completion or error)
Coordinating or managing multiple asynchronous operations such as sequencing or branching async operations or managing multiple operations in flight at the same time
Propagating errors from nested or deeply nested async operations
Getting code ready for the use of async/await (or using it now with a transpiler)
Operations that fit the Promise model where there are only three states: pending, fulfilled and rejected and where the state transitions from pending => fulfilled or from pending => rejected can then not change (a single one-way transition).
Dynamically linking or chaining asynchronous operations (such as do these two async operations, examine the result, then decide which other async operations to do based on the intermediate result)
Managing a mix of asynchronous and synchronous operations
Automatically catching and propagating upwards any exceptions that occur in async completion callbacks (in plain callbacks these exceptions are sometimes silently hidden).

Plain callbacks are good for things that promises cannot do:

Synchronous notifications (such as the callback for Array.prototype.map())
Notifications that may occur more than once (and thus need to call the callback more than once). Promises are one-shot devices and cannot be used for repeat notifications.
Situations that cannot be mapped into the pending, fulfilled, rejected one-way state model.

And, I'd also add EventEmitter to the mix.

EventEmitters are great for:

Publish/subscribe type notifications
An interface with an event model, particular when events can occur more than once (like streams)
Loose couplings when 3rd party code wants to participate or monitor something without any more of an API than an eventEmitter. No API to design. Just make an eventEmitter public and define some events and the data that goes with them.

Notes about converting plain callback code to Promises

If your callbacks fit the node calling convention with the callback passed as the last argument and called like this callback(err, result), then you somewhat automatically wrap the parent function in a promise with util.promisify() in node.js or if using the Bluebird promise library, with Promise.promisify().

With Bluebird, you can even promisify an entire module (that uses async callbacks in the node.js calling convention) at once such as:

const Promise = require('bluebird');
const fs = Promise.promisifyAll(require('fs'));

fs.writeFileAsync("file.txt", data).then(() => {
    // done here
}).catch(err => {
    // error here
});

In node.js version 8+

There is now util.promisify() which will convert an async function that uses the node.js async calling convention to a function that returns a promise.

Example from the doc:

const util = require('util');
const fs = require('fs');

const stat = util.promisify(fs.stat);

// usage of promisified function
stat('.').then((stats) => {
  // Do something with `stats`
}).catch((error) => {
  // Handle the error.
});

Why doesn't .then() need the async keyword when used (similar to await)? How does Javascript know it's an asynchronous operation?

The purpose of async/await is to allow writing async code in a serial manner, which is mentally simpler to reason about (for some human-beings). This is useful, if you need to wait for async operation to finish before you continue with the rest of the code. For example, if you need to pass result of async operation as parameter.

Example 1

function asyncOperation1(n) { return Promise.resolve(n+1); }
function asyncOperation2(n) { return Promise.resolve(n/2); }
function asyncOperation3(n) { return Promise.resolve(n*3); }
function errorHandler(err) { console.error(err); }

function main() {
  // flow-control
  asyncOperation1(1)
    .then(asyncOperation2)
    .then(asyncOperation3)
    .then(continueAfterAsync)
    .catch(errorHandler)

  // function wrapper
  function continueAfterAsync(result) {
    console.log(result);
  }
}

main();

With async/await the code of the main function above may look like

async main() {
  try {
    console.log(
      await asyncOperation3(
        await asyncOperation2(
          await asyncOperation1(1)
        )
      )
    );
  } catch(err) {
    errorHandler(err);
  }
}

Pay attention that we don't need to rewrite async operation functions to be async function asyncOperation... to use await, but we need to declare main function as async main.

Which one is better(?) is the mater of developers's taste and previous programming languages experience. The benefit that I can see is that you don't need to wrap everything into functions and introduce additional flow-control code, leaving this complexity to JavaScript compiler.

However, there are cases, when you want to schedule some parallel tasks and you don't care which one will finish first. These kind of things would be relatively hard to do with async/await only.

Example 2

function main() {
  Promise
    .all(
      ['srv1', 'srv2', 'srv3'].map(
        srv => fetch(`${srv}.test.com/status`)
      )
    ])
    .then(
      responses => responses.some(res => res.status !== 200) ?
        console.error('some servers have problems') :
        console.log('everything is fine')
    )
    .catch(err => console.error('some servers are not reachable', err))
}

So, we see that there is a room for both .then() and await to coexist.

In some cases function may be either synchronous or asynchronous, depending on business logic (I know it's ugly, but in some cases it's unavoidable). And here we come to your main question

why don't we need to mark an asynchronous operation with .then() and we have to do it with await

In other words, why do we need async keyword at all?

Example 3

// without `async`
function checkStatus(srv) {
  if (!srv.startsWith('srv')) {
    throw new Error('An argument passed to checkStatus should start with "srv"')
  }
  return fetch(`https://${srv}.test.com/status`);
}

function main() {
  // this code will print message
  checkStatus('srv1')
    .then(res => console.log(`Status is ${res.status === 200 ? 'ok': 'error'}`))
    .catch(err => console.error(err));

  // this code will fail with
  // Uncaught TypeError: (intermediate value).then is not a function
  checkStatus('svr1')
    .then(res => console.log(`Status is ${res.status === 200 ? 'ok': 'error'}`))
    .catch(err => console.error(err));
}

However, if we define async function checkStatus, compiler will wrap the runtime error into rejected promise return value, and both parts of the main function will work.

Now let's imagine that JavaScript allows to write functions that use await without specifying async in front of them.

Example 4 (not a valid Javascript)

function checkStatus(srv) {
  if (cache[srv]) {
    data = cache[srv];
  } else {
    data = (await fetch(`https://${srv}.test.com/status`)).json();
  }
  data.x.y = 'y';
  return data;
}

What would you expect checkStatus to return? Promise, raw value or throw exception (in case data.x is undefined)?

If you say Promise, then it would be hard for developer that uses this function to understand why inside of checkStatus one can write data.x and outside of it (await data).x is required.

If raw value, the whole execution flow becomes cumbersome, and you can no longer rely on the fact that JavaScript is a single-threaded language, where no-one can change the value of the variable between two lines of code that are written in serial manner.

As you noticed, async/await is a syntactic sugar. If this syntax allows me to avoid possible runtime errors at earlier stage and keep the language backward compatible, I'm eager to pay the price of putting extra async in front of async functions.

Also, I would recommend to read the answers to JS async/await - why does await need async?

Understanding Promises in Node.Js