Linux: Send Mail After a Process Id Finishes or Is Killed

What killed my process and why?

If the user or sysadmin did not kill the program the kernel may have. The kernel would only kill a process under exceptional circumstances such as extreme resource starvation (think mem+swap exhaustion).

What's the best way to send a signal to all members of a process group?

You don't say if the tree you want to kill is a single process group. (This is often the case if the tree is the result of forking from a server start or a shell command line.) You can discover process groups using GNU ps as follows:

 ps x -o  "%p %r %y %x %c "

If it is a process group you want to kill, just use the kill(1) command but instead of giving it a process number, give it the negation of the group number. For example to kill every process in group 5112, use kill -TERM -- -5112.

How to suppress Terminated message after killing in bash?

The short answer is that you can't. Bash always prints the status of foreground jobs. The monitoring flag only applies for background jobs, and only for interactive shells, not scripts.

see notify_of_job_status() in jobs.c.

As you say, you can redirect so standard error is pointing to /dev/null but then you miss any other error messages. You can make it temporary by doing the redirection in a subshell which runs the script. This leaves the original environment alone.

(script 2> /dev/null)

which will lose all error messages, but just from that script, not from anything else run in that shell.

You can save and restore standard error, by redirecting a new filedescriptor to point there:

exec 3>&2          # 3 is now a copy of 2
exec 2> /dev/null  # 2 now points to /dev/null
script             # run script with redirected stderr
exec 2>&3          # restore stderr to saved
exec 3>&-          # close saved version

But I wouldn't recommend this -- the only upside from the first one is that it saves a sub-shell invocation, while being more complicated and, possibly even altering the behavior of the script, if the script alters file descriptors.

---

EDIT:

For more appropriate answer check answer given by Mark Edgar

How to wait in bash for several subprocesses to finish, and return exit code !=0 when any subprocess ends with code !=0?

wait also (optionally) takes the PID of the process to wait for, and with $! you get the PID of the last command launched in the background.
Modify the loop to store the PID of each spawned sub-process into an array, and then loop again waiting on each PID.

# run processes and store pids in array
for i in $n_procs; do
    ./procs[${i}] &
    pids[${i}]=$!
done

# wait for all pids
for pid in ${pids[*]}; do
    wait $pid
done

Why is kill different from a terminal and from system()?

Even to a local host, ssh will take tens of milliseconds to connect and execute the command.

When killed by the script, ssh is dies within a few milliseconds of starting, so it never gets a chance to execute the command on the remote host.

When killed by a human, ssh has multiple seconds to connect and execute.

In other words, the sleep doesn't die in the first case -- it never starts.

You can confirm this by making your script wait a second before killing ssh.

Why child process still alive after parent process was killed in Linux?

No, when you kill a process alone, it will not kill the children.

You have to send the signal to the process group if you want all processes for a given group to receive the signal

For example, if your parent process id has the code 1234, you will have to specify the parentpid adding the symbol minus followed by your parent process id:

kill -9 -1234

Otherwise, orphans will be linked to init, as shown by your third screenshot (PPID of the child has become 1).

---

Related Topics