Bash 'Swallowing' Sub-Shell Children Process When Executing a Single Command

Bash 'swallowing' sub-shell children process when executing a single command

There's actually a comment in the bash source that describes much of the rationale for this feature:

/* If this is a simple command, tell execute_disk_command that it
   might be able to get away without forking and simply exec.
   This means things like ( sleep 10 ) will only cause one fork.
   If we're timing the command or inverting its return value, however,
   we cannot do this optimization. */
if ((user_subshell || user_coproc) && (tcom->type == cm_simple || tcom->type == cm_subshell) &&
    ((tcom->flags & CMD_TIME_PIPELINE) == 0) &&
    ((tcom->flags & CMD_INVERT_RETURN) == 0))
  {
    tcom->flags |= CMD_NO_FORK;
    if (tcom->type == cm_simple)
      tcom->value.Simple->flags |= CMD_NO_FORK;
  }

In the bash -c '...' case, the CMD_NO_FORK flag is set when determined by the should_suppress_fork function in builtins/evalstring.c.

It is always to your benefit to let the shell do this. It only happens when:

• Input is from a hardcoded string, and the shell is at the last command in that string.
• There are no further commands, traps, hooks, etc. to be run after the command is complete.
• The exit status does not need to be inverted or otherwise modified.
• No redirections need to be backed out.

This saves memory, causes the startup time of the process to be slightly faster (since it doesn't need to be forked), and ensures that signals delivered to your PID go direct to the process you're running, making it possible for the parent of sh -c 'sleep 10' to determine exactly which signal killed sleep, should it in fact be killed by a signal.

However, if for some reason you want to inhibit it, you need but set a trap -- any trap will do:

# run the noop command (:) at exit
bash -c 'trap : EXIT; sleep 10'

Bash: set array within braces in a while loop? (sub-shell problem)

The problem is not the braces, but the pipe. Because you're using a pipe, the assignment to Error[run] is happening in a subshell, so that assignment disappears when the subshell exits.

Change:

{ cpio -ti --quiet <"$archive" 2>'/dev/null' || local -a Error[run]='cpio'; } | grep -Ei '$extlist'

to:

cpio -ti --quiet <"$archive" 2>'/dev/null' | grep -Ei "$extlist"
[[ ${PIPESTATUS[0]} -ne 0 ]] && Error[run]='cpio'

(btw, need double quotes in the grep part)

Bash script catch signal but wait afterwards for processes to terminate

Something rewritted

In order to avoid some useless forks.

clock(){  local prefix=C interval=2
    trap : RTMIN{,+{{,1}{1,2,3,4,5},6,7,8,9,10}}
    while :;do
        printf "%s: %(%d.%m %H:%M:%S)T\n" $prefix -1
        sleep $interval
    done
}

volume(){  local prefix=V vol=() field playback val foo
    while IFS=':[]' read field playback val foo;do
        [ "$playback" ] && [ -z "${playback//*Playback*}" ] && [ "$val" ] &&
            vol+=(${val%\%})
    done < <(amixer get Master)
    suffix='%%'
    if [ "$vol" = "off" ] ;then
        icon="&" #alternative: deaf:  mute: 
        suffix=''
    elif (( vol > 50 )) ;then  icon="("
    elif (( vol > 30 )) ;then  icon="("
    else                       icon="'"
    fi
    printf -v values "%3s$suffix " ${vol[@]}
    printf "%s%s %s\n" $prefix "$icon" "$values"
}

clock & volume &

trap volume RTMIN+2
trap : RTMIN{,+{{,1}{1,3,4,5},6,7,8,9,10,12}}
echo -e "To get status, run:\n  kill -RTMIN+2 $$"

while :;do wait ;done

Regarding my last comment about stereo bug, there is a volume function working for stereo, mono or even quadra:

volume(){
    local prefix=V vol=() field playback val foo
    local -i overallvol=0
    while IFS=':[]' read field playback val foo ;do
        [ "$playback" ] && [ -z "${playback//*Playback*}" ] && [ "$val" ] && {
            vol+=($val)
            val=${val%\%}
            overallvol+=${val//off/0}
        }
      done < <(
        amixer get Master
    )
    overallvol=$overallvol/${#vol[@]}
    if (( overallvol == 0 )) ;then
        icon="&"
      elif (( overallvol > 50 )) ;then
        icon="("
      elif (( overallvol > 30 )) ;then
        icon="("
      else
        icon="'"
    fi
    printf "%s%s %s\n" $prefix "$icon" "${vol[*]}"
}

or even:

volume(){
    local prefix=V vol=() field playback val foo icons=(⏻ ¼ ¼ ¼ ½ ½ ¾ ¾ ¾ ¾ ¾)
    local -i overallvol=0
    while IFS=':[]' read field playback val foo ;do
        [ "$playback" ] && [ -z "${playback//*Playback*}" ] && [ "$val" ] && {
            vol+=($val)
            val=${val%\%}
            overallvol+=${val//off/0}
        }
      done < <(
        amixer get Master
    )
    overallvol=$overallvol/${#vol[@]}
    printf "%s%s %s\n" $prefix "${icons[(9+overall)/10]}" "${vol[*]}"

Some explanations

Regarding useless forks in volume() function

I've posted there some ideas to improve the job, reducing resource eating and doing same job of choosing an icon as function of current volume set.

About while :;do wait;done loop

As requested sample stand for an infinite loop in backgrounded sub function, the main script use same infinite loop.

But as question title stand for wait afterwards for processes to terminate, I have to agree with oguz-ismail's comment.

In fact, last line would better be written:

until wait;do :;done

For more information on how wait command work and good practice, please have a look on good oguz-ismail's answer

why does sh.exe swallow `\` and how to escape `\`?

The problem is that the back-slash needs to be escaped twice:

• First by the shell where you invoke the sh command
• Then a second time by the sh command itself.

So when you use the command-line

sh -c 'ls C:\\\Users\\\timothee'

the running shell will remove one back-slash and pass 'ls C:\\Users\\timothee to the sh command.

The sh command in turn needs that double back-slash in its own processing, and will pass C:\Users\timothee to the ls command.

Bash redirect stdout and stderr to seperate files with timestamps

The trick is to make sure that tee, and the process substitution running your log function, exits before the script as a whole does -- so that when the shell that started the script prints its prompt, there isn't any backgrounded process that might write more output after it's done.

As a working example (albeit one focused more on being explicit than terseness):

#!/usr/bin/env bash
stdout_log=stdout.log; stderr_log=stderr.log

log () {
  file=$1; shift
  while read -r line; do
    printf '%(%s)T %s\n' -1 "$line"
  done >> "$file"
}

# first, make backups of your original stdout and stderr
exec {stdout_orig_fd}>&1 {stderr_orig_fd}>&2
# for stdout: start your process substitution, record its PID, start tee, record *its* PID
exec {stdout_log_fd}> >(log "$stdout_log"); stdout_log_pid=$!
exec {stdout_tee_fd}> >(tee "/dev/fd/$stdout_log_fd"); stdout_tee_pid=$!
exec {stdout_log_fd}>&- # close stdout_log_fd so the log process can exit when tee does
# for stderr: likewise
exec {stderr_log_fd}> >(log "$stderr_log"); stderr_log_pid=$!
exec {stderr_tee_fd}> >(tee "/dev/fd/$stderr_log_fd" >&2); stderr_tee_pid=$!
exec {stderr_log_fd}>&- # close stderr_log_fd so the log process can exit when tee does
# now actually swap out stdout and stderr for the processes we started
exec 1>&$stdout_tee_fd 2>&$stderr_tee_fd {stdout_tee_fd}>&- {stderr_tee_fd}>&-

# ...do the things you want to log here...
echo "this goes to stdout"; echo "this goes to stderr" >&2

# now, replace the FDs going to tee with the backups...
exec >&"$stdout_orig_fd" 2>&"$stderr_orig_fd"

# ...and wait for the associated processes to exit.
while :; do
  ready_to_exit=1
  for pid_var in stderr_tee_pid stderr_log_pid stdout_tee_pid stdout_log_pid; do
    # kill -0 just checks whether a PID exists; it doesn't actually send a signal
    kill -0 "${!pid_var}" &>/dev/null && ready_to_exit=0
  done
  (( ready_to_exit )) && break
  sleep 0.1 # avoid a busy-loop eating unnecessary CPU by sleeping before next poll
done

---

So What's With The File Descriptor Manipulation?

A few key concepts to make sure we have clear:

• All subshells have their own copies of the file descriptor table as created when they were fork()ed off from their parent. From that point forward, each file descriptor table is effectively independent.
• A process reading from (the read end of) a FIFO (or pipe) won't see an EOF until all programs writing to (the write end of) that FIFO have closed their copies of the descriptor.

...so, if you create a FIFO pair, fork() off a child process, and let the child process write to the write end of the FIFO, whatever's reading from the read end will never see an EOF until not just the child, but also the parent, closes their copies.

Thus, the gymnastics you see here:

• When we run exec {stdout_log_fd}>&-, we're closing the parent shell's copy of the FIFO writing to the log function for stdout, so the only remaining copy is the one used by the tee child process -- so that when tee exits, the subshell running log exits too.
• When we run exec 1>&$stdout_tee_fd {stdout_tee_fd}>&-, we're doing two things: First, we make FD 1 a copy of the file descriptor whose number is stored in the variable stdout_tee_fd; second, we delete the stdout_tee_fd entry from the file descriptor table, so only the copy on FD 1 remains. This ensures that later, when we run exec >&"$stdout_orig_fd", we're deleting the last remaining write handle to the stdout tee function, causing tee to get an EOF on stdin (so it exits, thus closing the handle it holds on the log function's subshell and letting that subshell exit as well).

---

Some Final Notes On Process Management

Unfortunately, how bash handles subshells created for process substitutions has changed substantially between still-actively-deployed releases; so while in theory it's possible to use wait "$pid" to let a process substitution exit and collect its exit status, this isn't always reliable -- hence the use of kill -0.

However, if wait "$pid" worked, it would be strongly preferable, because the wait() syscall is what removes a previously-exited process's entry from the process table: It is guaranteed that a PID will not be reused (and a zombie process-table entry is left as a placeholder) if no wait() or waitpid() invocation has taken place.

Modern operating systems try fairly hard to avoid short-term PID reuse, so wraparound is not an active concern in most scenarios. However, if you're worried about this, consider using the flock-based mechanism discussed in https://stackoverflow.com/a/31552333/14122 for waiting for your process substitutions to exit, instead of kill -0.

---

Related Topics