Run PHP Task Asynchronously

Run PHP Task Asynchronously

I've used the queuing approach, and it works well as you can defer that processing until your server load is idle, letting you manage your load quite effectively if you can partition off "tasks which aren't urgent" easily.

Rolling your own isn't too tricky, here's a few other options to check out:

  • GearMan - this answer was written in 2009, and since then GearMan looks a popular option, see comments below.
  • ActiveMQ if you want a full blown open source message queue.
  • ZeroMQ - this is a pretty cool socket library which makes it easy to write distributed code without having to worry too much about the socket programming itself. You could use it for message queuing on a single host - you would simply have your webapp push something to a queue that a continuously running console app would consume at the next suitable opportunity
  • beanstalkd - only found this one while writing this answer, but looks interesting
  • dropr is a PHP based message queue project, but hasn't been actively maintained since Sep 2010
  • php-enqueue is a recently (2017) maintained wrapper around a variety of queue systems
  • Finally, a blog post about using memcached for message queuing

Another, perhaps simpler, approach is to use ignore_user_abort - once you've sent the page to the user, you can do your final processing without fear of premature termination, though this does have the effect of appearing to prolong the page load from the user perspective.

How to run the PHP code asynchronous

If you wanted to run it from the browser (perhaps you're not familiar with the command line) you could still do it. I researched many solutions for this a few months ago and the most reliable and simplest to implement was the following from How to post an asynchronous HTTP request in PHP

<?php


$params['my_param'] = $a_value;
post_async('http:://localhost/batch/myjob.php', $params);

/*
* Executes a PHP page asynchronously so the current page does not have to wait for it to finish running.
*
*/
function post_async($url, array $params)
{
foreach ($params as $key => &$val) {
if (is_array($val)) $val = implode(',', $val);
$post_params[] = $key.'='.urlencode($val);
}
$post_string = implode('&', $post_params);

$parts=parse_url($url);

$fp = fsockopen($parts['host'],
isset($parts['port'])?$parts['port']:80,
$errno, $errstr, 30);

$out = "POST ".$parts['path']." HTTP/1.1\r\n";
$out.= "Host: ".$parts['host']."\r\n";
$out.= "Content-Type: application/x-www-form-urlencoded\r\n";
$out.= "Content-Length: ".strlen($post_string)."\r\n";
$out.= "Connection: Close\r\n\r\n";
if (isset($post_string)) $out.= $post_string;

fwrite($fp, $out);
fclose($fp);
}

Let's say the file above is in your web root directory (/var/www) for example and is called runjobs.php. By visiting http://localhost/runjobs.php your myjob.php file would start to run. You'd probably want to add some output to the browser to let you know it was submitted successfully and it wouldn't hurt to add some security if your web server is open to the rest of the world. One nice thing about this solution if you add some security is that you can start the job anywhere you can find a browser.

Run part of PHP code asynchronously from inside PHP program

You can use $f3->abort() to send the output/response to the browser and process your other blocking function afterwards. That's not a real asynchron solution but would work. You could also use something like php-icicle to add threads support, but that maybe requires some other php modules being installed.

Run PHP exec() asynchronously, but check for completion?

You should start an asynchronous command line php script that encodes both videos and then sends an email :

upload.php :

exec('/usr/bin/php -f encode_files.php > /dev/null 2>/dev/null &"');
echo "Files will be encoded, come back later !";

encode_files.php

exec('avconv ...'); // Synchronously ! Without > /dev/null etc ...
exec('avconv ...'); // webm ...

mail('user@user.com', 'Encoding complete ! ', 'Great ! ');

I left the call as "bash -c exec ..." but i think there are shorter ways to call php scripts asynchronously :
Asynchronous shell exec in PHP
You can even pass params (like the user/video id, ...)

$cmd = 'nohup /usr/bin/php -f /path/to/php/file.php action=generate var1_id=23 var2_id=35 gen_id=535 > /path/to/log/file.log & printf "%u" $!';
$pid = shell_exec($cmd);

Running a php script asynchronously

The likely issue is that script.php is opening a session and whichever page you are trying to visit is also trying to open a session but has to wait until script.php is all set with it.

Somewhere in script.php you will need to add:

ignore_user_abort(1); // Let the script run even if user leaves the page
set_time_limit(0); // Let script run forever

session_write_close(); // Close the session. This will allow for other pages to open it
// You will still be able to read your session data but can no longer write to it unless it is re-opened

sleep(60);

PHP Asynchronous Task

While you can send the output buffer early and keep processing (see Continue processing after closing connection) it's a hacky solution and not suited to PHPs request life-cycle.

A better method is to create a processing queue. When the client hits your service, you return the success status and insert a processing request in the queue (for example, in a database table / key-value store, etc). A separate process (e.g. cron job running a CLI script) picks up the tasks and executes them first in first out. This allows long running scripts to execute out of band - perhaps even on a different server.

As an aside, this also allows you to free up the web server for more requests even if the client needs to check on the result of the long running process. The client can poll the server to check if the job is complete, and get the full response when it's ready.



Related Topics



Leave a reply



Submit