How to Perform Multiple Guzzle Requests at the Same Time

How to perform multiple Guzzle requests at the same time?

From the docs:
http://guzzle3.readthedocs.org/http-client/client.html#sending-requests-in-parallel

For an easy to use solution that returns a hash of request objects mapping to a response or error, see http://guzzle3.readthedocs.org/batching/batching.html#batching

Short example:

<?php

$client->send(array(
    $client->get('http://www.example.com/foo'),
    $client->get('http://www.example.com/baz'),
    $client->get('http://www.example.com/bar')
));

How to perform concurrent requests with GuzzleHttp

Guzzle provides fulfilled and rejected callabcks in the pool. here I performed a test by your values, read more at Guzzle docs:

    $client = new Client([
        'http_errors'     => false,
        'connect_timeout' => 0.50, //////////////// 0.50
        'timeout'         => 1.00, //////////////// 1.00
        'headers' => [
          'User-Agent' => 'Test/1.0'
        ]
      ]);

$requests = function ($total) {
    $uris = [
        'https://httpbin.org/get',
        'https://httpbin.org/delay/1',
        'https://httpbin.org/delay/2',
        'https://httpbin.org/status/500',
        ];
    for ($i = 0; $i < count($uris); $i++) {
        yield new Request('GET', $uris[$i]);
    }
};

$pool = new Pool($client, $requests(8), [
    'concurrency' => 10,
    'fulfilled' => function ($response, $index) {
        // this is delivered each successful response
        print_r($index."fulfilled\n");
    },
    'rejected' => function ($reason, $index) {
        // this is delivered each failed request
        print_r($index."rejected\n");
    },
]);
// Initiate the transfers and create a promise
$promise = $pool->promise();
// Force the pool of requests to complete.
$promise->wait();

response

0fulfilled
3fulfilled
1rejected
2rejected

if you want to use your code above you can also pass response status in your $promises, here is an example:

use Psr\Http\Message\ResponseInterface;
use GuzzleHttp\Exception\RequestException;
....
$client = new Client([
    'http_errors'     => false,
    'connect_timeout' => 1.50, //////////////// 0.50
    'timeout'         => 2.00, //////////////// 1.00
    'headers' => [
      'User-Agent' => 'Test/1.0'
    ]
  ]);

            $promises = [
        'success' => $client->getAsync('https://httpbin.org/get')->then(
            function (ResponseInterface $res) {
                echo $res->getStatusCode() . "\n";
            },
            function (RequestException $e) {
                echo $e->getMessage() . "\n";
                echo $e->getRequest()->getMethod();
            }
        )
        ,
        'success' => $client->getAsync('https://httpbin.org/delay/1')->then(
            function (ResponseInterface $res) {
                echo $res->getStatusCode() . "\n";
            },
            function (RequestException $e) {
                echo $e->getMessage() . "\n";
                echo $e->getRequest()->getMethod();
            }
        ),
        'failconnecttimeout' => $client->getAsync('https://httpbin.org/delay/2')->then(
            function (ResponseInterface $res) {
                echo $res->getStatusCode() . "\n";
            },
            function (RequestException $e) {
                echo $e->getMessage() . "\n";
                echo $e->getRequest()->getMethod();
            }
        ),
        'fail500' => $client->getAsync('https://httpbin.org/status/500')->then(
            function (ResponseInterface $res) {
                echo $res->getStatusCode() . "\n";
            },
            function (RequestException $e) {
                echo $e->getMessage() . "\n";
                echo $e->getRequest()->getMethod();
            }
        ),
      ];

  $results = Promise\settle($promises)->wait();

Multiple request in guzzle

Don't rely on that condition. Either rely on the next entry not being null or count the total entries you have and compare it to the total entry.

---

Spotify exposes a total number of entries in the pagination wrapper around the response. You can make a first request with the first 50 entries, then make concurrent requests for all remaining chunks, because you know the total number at that point.

You have to use asyncRequest() for the further requests, which returns a promise, and schedule all your remaining requests. Then you can wait for the promises sequentially using the wait() instance method. The order of your wait() calls doesn't matter, because wait() will tick the internal event loop and make progress for any of your requests. All further wait() calls take either way shorter to run or even resolve immediately.

Unfortunately, you will have to construct the URLs manually, instead of being able to rely on the next entry for your URLs.

I'd recommend to add some limit of concurrency, Spotify probably has some guidelines for that. Guzzle offers a Pool implementation for that.

problem with multiple HTTP requests in Laravel with Guzzle

Guzzle 6 uses a different approach to Guzzle 3, so you should use something like:

use function GuzzleHttp\Promise\all;

$header = ['headers' => ['X-Auth-Token' => 'MyKey']];
$client = new Client(['debug' => true]);
$responses = all([
    $client->getAsync('https://api.football-data.org/v2/teams/666', $header),
    $client->getAsync('https://api.football-data.org/v2/teams/1920', $header),
    $client->getAsync('https://api.football-data.org/v2/teams/6806', $header)
])->wait();
$data = [];
foreach ($responses as $i => $res) {
    $data[$i] = json_decode($res->getBody()->getContents(), true);
}
return $data;

Take a look at different questions on the same topic (#1, #2) to see more usage examples.

How can I use guzzle to send concurrent requests with timeout limits?

I found the answer before I finished writing the question, but I thought I would still post it in case it helps someone else.

The solution was to replace this call:

$results = Promise\unwrap($promises);

With this:

$results = Promise\settle($promises)->wait(true);

The settle function will not throw an exception if a promise is rejected, allowing you to get the full responses array (with both fulfilled and rejected responses).

---

Related Topics