Making Multiple Http Requests Asynchronously

Asynchronous, Multiple HTTP requests in a While Loop

As the error says, you can't await a coroutine more than once. Instead of passing a coroutine into run_forever and then awaiting it in a loop, passing the coroutine's argument(s) instead and await a new coroutine each iteration of the loop.

class Client:
    async def run_forever(self, value, interval):
        while True:
            res = await rqequest(value)
            await self._response(response, interval)

You also need to change how you await run_forever. await is blocking, so when you await something with an infinite loop, you'll never reach the next line. Instead, you want to gather multiple coroutines as once.

async def main():
    c = Client()
    await asyncio.gather(
        c.run_forever("X", interval=1),
        c.run_forever("Y", interval=2),
        c.run_forever("Z", interval=3),
    )

Wait for multiple http requests to finish before running a function in angular

First off the async/await keywords are for use with promises, not Observables. So you can convert an observable to a promise using toPromise() or you can use rxjs ForkJoin. You also need to change the methods to return either an observable or return a promise, you can't do much with Subscription which is what subscribe returns when you call it and that is what you are passing now from getRiskTable.

One other issue is that you are not taking advantage of the type system, do not use any everywhere because these errors could have been caught at transpile time and now you have to guess why it's failing at run time which is more difficult.

import { forkJoin } from 'rxjs';
import { shareReplay } from 'rxjs/operators';

getRiskTable() : Observable<Risk[]> {
  const tableObservable = this.riskService.getRiskTable().pipe(shareReplay());
  tableObservable.subscribe((res) => this.riskTable = res, (err) => {this.error = err;});
  return tableObservable;
}

getAllData() {
  let riskTable = this.getRiskTable();
  let risks = this.getAllRisks(); // should also return some observable

  forkJoin(riskTable, risks).subscribe(_ => {
    // all observables have been completed
  });
}

Note that I also added shareReplay which ensures that multiple subscriptions to the same observable do not result in multiple calls to your API end point. One result is shared between multiple subscriptions to the observable.

Subscribe to multiple async http calls

You can use forkJoin to combine all the requests and get the results as callback when all the requests are completed.

import { forkJoin } from 'rxjs';
import { switchMap } from 'rxjs/operators';

ngOnInit() {
 forkJoin([this.service.getProvince(),this.service.getLingueStraniere(),this.service.getTitoliPreferenziali(), this.service.getRiserve()])
     .pipe(switchMap(result => { 
          console.log('Province', result[0]);
          console.log('LingueStraniere', result[1]);
          console.log('TitoliPreferenziali', result[2]);
          console.log('Riserve', result[3]);
          return this.service.getDomanda();
      }))
      .subscribe((domanda: any) => {
         console.log('getDomanda', domanda);
       });                                                          
}

Fast and asynchronous way of making multiple http requests in JAVA

Answering my own question. Tried Apaches asynchronous http client but after a while I started using Ning's async client and I am happy with it.

Multiple async requests simultaneously

aiohttp with Native Coroutines (async/await)

Here is a typical pattern that accomplishes what you're trying to do. (Python 3.7+.)

One major change is that you will need to move from requests, which is built for synchronous IO, to a package such as aiohttp that is built specifically to work with async/await (native coroutines):

import asyncio
import aiohttp  # pip install aiohttp aiodns


async def get(
    session: aiohttp.ClientSession,
    color: str,
    **kwargs
) -> dict:
    url = f"https://api.com/{color}/"
    print(f"Requesting {url}")
    resp = await session.request('GET', url=url, **kwargs)
    # Note that this may raise an exception for non-2xx responses
    # You can either handle that here, or pass the exception through
    data = await resp.json()
    print(f"Received data for {url}")
    return data


async def main(colors, **kwargs):
    # Asynchronous context manager.  Prefer this rather
    # than using a different session for each GET request
    async with aiohttp.ClientSession() as session:
        tasks = []
        for c in colors:
            tasks.append(get(session=session, color=c, **kwargs))
        # asyncio.gather() will wait on the entire task set to be
        # completed.  If you want to process results greedily as they come in,
        # loop over asyncio.as_completed()
        htmls = await asyncio.gather(*tasks, return_exceptions=True)
        return htmls


if __name__ == '__main__':
    colors = ['red', 'blue', 'green']  # ...
    # Either take colors from stdin or make some default here
    asyncio.run(main(colors))  # Python 3.7+

There are two distinct elements to this, one being the asynchronous aspect of the coroutines and one being the concurrency introduced on top of that when you specify a container of tasks (futures):

• You create one coroutine get that uses await with two awaitables: the first being .request and the second being .json. This is the async aspect. The purpose of awaiting these IO-bound responses is to tell the event loop that other get() calls can take turns running through that same routine.
• The concurrent aspect is encapsulated in await asyncio.gather(*tasks). This maps the awaitable get() call to each of your colors. The result is an aggregate list of returned values. Note that this wrapper will wait until all of your responses come in and call .json(). If, alternatively, you want to process them greedily as they are ready, you can loop over asyncio.as_completed: each Future object returned represents the earliest result from the set of the remaining awaitables.

Lastly, take note that asyncio.run() is a high-level "porcelain" function introduced in Python 3.7. In earlier versions, you can mimic it (roughly) like:

# The "full" versions makes a new event loop and calls
# loop.shutdown_asyncgens(), see link above
loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(main(colors))
finally:
    loop.close()

---

Limiting Requests

There are a number of ways to limit the rate of concurrency. For instance, see asyncio.semaphore in async-await function or large numbers of tasks with limited concurrency.

Waiting for multiple asynchronous http requests to complete

Promise.all waits for all fulfillments (or the first rejection). You can go through below link for different ways to achieve this using Promise.allSettled().

Already been asked: Wait until all promises complete even if some rejected

or https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled

How to send multiple of asynchronous requests from client to server

Asynchronous means sending message and not waiting for response.

//Get the response message from server - so you wait whole 5 seconds for response :)
String responseMessage = reader.readLine();

The simplest solution in this case is to remove waiting for response each time. So, get rid of above lines from loop inside Client class.

In this particular client-sever case you do not need additional threads, if you would perform asynchronous things inside one application, then so. Take a look at Java Futures with some tutorial on how to use them.
But if you want to get a result from server, you have to wait anyway. And you want to get results of all calcuations. Hence, you have to store all incoming requests somewhere. Simple, naive and impractical, but showing asynchronicity concept code may look like this

public class Client {
  public static void main(String[] args) throws IOException {
    try {
        long start = System.currentTimeMillis();
        BufferedReader reader = null;
        for(int i = 0 ; i < 20; i++){
            Socket s = new Socket("localhost", 7777);
            PrintWriter writer = new PrintWriter(s.getOutputStream());
            InputStreamReader streamReader = new InputStreamReader(s.getInputStream());
            reader = new BufferedReader(streamReader);
            // just make sure you send data and do not wait for response
            System.out.println("Sending " + i  + " : " + (System.currentTimeMillis() - start)); 
            writer.println(i);
            writer.flush();
        }
        //this line works like future.get(), it hangs client until it receives result
        String responseMessage = reader.readLine();
        // process returned data as you want
        System.out.println(responseMessage);
    }
    catch (IOException ex) {
        ex.printStackTrace(); // (**)
    }
  }
}

public class Server {
  public static void main(String[] args) throws IOException {
    try {
        //Make a ServerSocket to listen for message
        ServerSocket ss = new ServerSocket(7777);
        Socket s;
        //we need to store incoming requests to process, and return them
        List<String> integerList = new LinkedList<>();

        while (true) {
            s = ss.accept();

            InputStreamReader streamReader = new InputStreamReader(s.getInputStream());
            BufferedReader reader = new BufferedReader(streamReader);
            String message = reader.readLine();
            System.out.println(message);
            // do something here
            
            Thread.sleep(5000);

            PrintWriter writer = new PrintWriter(s.getOutputStream());
            integerList.add(message);
            writer.println(integerList);
            writer.close();
        }
    } catch (IOException | InterruptedException ex) {
        System.out.println(ex);
    }
  }
}

---

Related Topics