How to Make Perl Wait for Child Processes Started in the Background with System()

How can I make Perl wait for child processes started in the background with system()?

Using fork/exec/wait isn't so bad:

my @a = (1, 2, 3);
for my $p (@a) {
   my $pid = fork();
   if ($pid == -1) {
       die;
   } elsif ($pid == 0) {
      exec '/bin/sleep', $p or die;
   }
}
while (wait() != -1) {}
print "Done\n";

How to wait for running process to complete in perl when running process is not child process?

Note   A simple-minded one-liner with kill 0, $pid is at end, commented.

---

We need to detect completion of an external program, which had not been started by this script. The question asks about using waitpid. To copy my early comment:

You cannot. You can only wait on a child process. See perldoc wait (or waitpid, it's the same), first sentence.

The wait and waitpid wait for signals delivered to the script regarding the fate of its child(ren). There is no reason for the script to receive such signals about processes that it did not start.

---

We know the process's id and its name. Its PID can be used to poll for whether it is running. Using pid on its own is not completely reliable since in between our checks the process can finish and a random new one be assigned the same pid. We can use the program's name to strengthen this.

On a Linux system information about a process can be obtained by utilizing (the many) ps options. Either of these returns the program's full invocation

ps --no-headers -o cmd PID
ps --no-headers -p PID -o cmd

The returned string may start with the interpreter's path (for a Perl script, for example), followed by the program's full name. The version ps -p PID -o comm= returns only the program's name, but I find that it may break that word on a hyphen (if there), resulting in an incomplete name. This may need tweaking on some systems, please consult your man ps. If there is no process with given PID we get nothing back.

Then we can check for PID and if found check whether the name for that PID matches the program. The program's name is known and we could just hardcode that. However, it is still obtained by the script as it starts, using the above ps command, to avoid ambiguities. (Then it is also in the same format for later comparison.) This itself is checked against the known name since there is no guarantee that the PID at the time of script execution is indeed for the expected program.

use warnings;
use strict;

# For testing. Retrieve your PID as appropriate for real use    
my $ext_pid = $ARGV[0] || $$;

my $cmd_get_name = "ps --no-headers -o cmd $ext_pid";

# For testing.  Replace 'sleep' by your program name for real use
my $known_prog_name = 'sleep';

# Get the name of the program with PID
my $prog_name = qx($cmd_get_name);

# Test against the known name, exit if there is a mismatch
if ($prog_name !~ $known_prog_name) {
    warn "Mismatch between:\n$prog_name\n$known_prog_name -- $!";
    exit;
}   

my $name;
while ( $name = qx($cmd_get_name) and $name =~ /$prog_name/ )
{
    print "Sleeping 1 sec ... \n";
    sleep 1;
}   
# regex above may need slight adjustment, depending on format of ps return

The command output received via qx() above (backtick operator) contains a newline. If that proves to be a problem in what the script does it can be chomp-ed, which would require a slight adjustment. The remaining loophole is that that very program may have finished and was restarted between the checks, and with the same PID.

This would be tested by running in a shell

sleep 30 &
script.pl `ps aux | egrep '[s]leep'`

The egrep is grep -E. The output from `ps ...` contains multiple words. These are passed as command line arguments to our script, which uses the first one as the PID. If there is a problem with it run the ps filtering first and then manually enter the PID as script's input argument. The sleep of 30 seconds above is to give enough time to do all this on the command line.

The code can be simplified by matching $name with a hard-coded $prog_name, if the program name is unique enough and it will not change.
The hard-coded name is used above, but for a check and it generates a warning if mismatched. (If we rely on it only hardcoded we cannot issue warnings if it mismatches, since that is then a part of code's operation.)

---

If the process is owned by the same user as the script one can use kill 0, $pid, as

while ( kill 0, $ext_pid ) { sleep 1 }

Then you'd either have to make another call to check the name or be content with the (small) possibility of an error in what actual process the $pid represents.

The module Proc::ProcessTable can be used for much of this

Python - call perl as subprocess - wait for finish its background process and print to shell

I have fix this by changing the Perl script based on this answer How can I make Perl wait for child processes started in the background with system()?

I have changed

system("something &") 

to

my $pid = fork();
if ($pid == -1) {
   print("Failed to fork");
   die;
} elsif ($pid == 0) {
   exec "something ";
}

and add

while (wait() != -1) {}

to the script end

How to wait for grandchild process (`bash` retval becomes -1 in Perl due to SIG CHLD)

I thought the quickest solution would be to add sleep of a second or two at the bottom of the bash script to wait for the zombie icc to complete. But that didn't work.

If I didn't already have a SIG ALRM (in the real program) I agree the best choice would be to wrap the whole thing in a eval. Even thought that would be pretty ugly for a 500 line program.

Without the local($?), every `system` call gets $? = -1. The $? I need in this case is after waitpid, then unfortunately set to -1 after the sig handler exits. So I find this works. New lines shown with ###

my $timer_pid;
my $chld_status;    ###
$SIG{CHLD} = sub {
    local($!, $?);
    while((my $pid = waitpid(-1, WNOHANG)) > 0)
    {
        $chld_status = $?;    ###
        if($pid == $timer_pid)
        {
            die "Timeout\n";
        }
    }
};

...
my @compile = `./compile_test.sh 2>&1`;
my $status = ($? == -1) ? $chld_status : $?;    ###
...

Wait for child processes with timeout

Have a look at alarm() - which triggers a kill signal ALRM after the specified timeout. Which'll either just kill the process, or you can use:

    $SIG{'ALRM'} = \&some_sub_to_handle_alarms;

alarm() doesn't propagate to forked processes, so you can set it on your 'parent' so it just times out waiting for children.

You probably don't need to fork your pidTimer though - you can just have your main process sit in a loop.

The following demonstrates:

use strict;
use warnings;
use feature qw(say);

my $pid1 = fork();
if ( $pid1 == 0 ) {    # Simulated Task 1
    sleep 10;
    exit 0;
}

my $pid2 = fork();
if ( $pid2 == 0 ) {    # Simulated Task 2
    sleep 5;
    exit 0;
}

say "Waiting for child processes..";
my $counter = 20;
local $SIG{ALRM} = sub {
    say --$counter;
    alarm 1;
};
alarm 1;

while ((my $pid = wait) != -1) {
    say "Child with PID=$pid finished..";
}

alarm 0;
say "Done.";

Outputs:

Waiting for child processes..
19
18
17
16
Child with PID=55240 finished..
15
14
13
12
11
Child with PID=55239 finished..
Done.

Perl: Waiting for background process to finish

The simplest thing that you're doing wrong is using & at the end of the exec commandline -- that means you're forking twice, and the process that you're waiting on will exit immediately.

I don't actually see what purpose fork/exec are serving you at all here, though, if you're not redirecting I/O and not doing anything but wait for the exec'd process to finish; that's what system is for.

system("./deployer -d $domain.$enviro -e $enviro >> /tmp/$domain.$enviro") 
                and logit("Problem running deployer: $?");

will easily serve to replace the first twelve lines of your code.

And just as a note in passing, fork doesn't return -1 on failure; it returns undef, so that whole check is entirely bogus.

---

Related Topics