Why Does This Shell Pipeline Exit

Why does this shell pipeline exit?

After base64 outputs 10 bytes, head gets enough inputs and exits. When the former attempts to output more bytes, it will receive SIGPIPE signal and hence exit too。For the same reason, cat will exit in turn.

Shell pipe: Exit immediately when one command fails

The bash documentation says in its section about pipelines:

Each command in a pipeline is executed in its own subshell [...]

"In its own subshell" means that a new bash process is spawned, which then gets to execute the actual command. Each subshell starts successfully, even when it immediately determines that the command it is asked to execute doesn't exist.

This explains why the entire pipe can be set up successfully even when one of the commands is nonsense. Bash does not check if each command can be run, it delegates that to the subshells. That also explains why, for example, the command nonexisting-command | touch hello will throw a "command not found" error, but the file hello will be created nonetheless.

In the same section, it also says:

The shell waits for all commands in the pipeline to terminate before returning a value.

In sleep 5 | nonexisting-command, as A.H. pointed out, the sleep 5 terminates after 5 seconds, not immediately, hence the shell will also wait 5 seconds.

I don't know why the implementation was done this way. In cases like yours, the behavior is surely not as one would expect.

Anyway, one slightly ugly workaround is to use FIFOs:

mkfifo myfifo
./long-running-script.sh > myfifo &
whoops-a-typo < myfifo

Here, the long-running-script.sh is started and then the scripts fails immediately on the next line. Using mutiple FIFOs, this could be extended to pipes with more than two commands.

Pipe output and capture exit status in Bash

There is an internal Bash variable called $PIPESTATUS; it’s an array that holds the exit status of each command in your last foreground pipeline of commands.

<command> | tee out.txt ; test ${PIPESTATUS[0]} -eq 0

Or another alternative which also works with other shells (like zsh) would be to enable pipefail:

set -o pipefail
...

The first option does not work with zsh due to a little bit different syntax.

Exit when one process in pipe fails

The main issue at hand here is clearly the pipe. In bash, when executing a command of the form

command1 | command2

and command2 dies or terminates, the pipe which receives the output (/dev/stdout) from command1 becomes broken. The broken pipe, however, does not terminate command1. This will only happen when it tries to write to the broken pipe, upon which it will exit with sigpipe. A simple demonstration of this can be seen in this question.

If you want to avoid this problem, you should make use of process substitution in combination with input redirection. This way, you avoid pipes. The above pipeline is then written as:

command2 < <(command1)

In the case of the OP, this would become:

./script.sh < <(tee /dev/stderr) | tee /dev/stderr

which can also be written as:

./script.sh < <(tee /dev/stderr) > >(tee /dev/stderr)

Why does exit not exit my script when I pipe it to tee?

As covered elsewhere (including, somewhat indirectly, BashFAQ #24), pipelines create subshells, so your exit exits only the subshell.

You can avoid this by replacing the pipeline with a redirection to a process substitution:

{ echo "hello world"; exit 1; } > >(tee -a logfile.log)

Generate other exit behavior if output from pipeline is empty

I would use the PIPESTATUS to check the exit code of egrep:

#!/bin/bash
# construct regex from input
# set FILE according to input file
egrep "${regex}" "${FILE}" | doing stuff | sort
if [[ ${PIPESTATUS[0] != 0 ]]; then
echo "error message" 1>&2
exit 3
fi

Some context:

${PIPESTATUS[@]} is just an array wich contains the exit code of every program you chained up. $? will just give you the exit code of the last command in the pipe.

Jenkins pipeline bubble up the shell exit code to fail the stage

The sh step returns the same status code that your actual sh command (your script in this case) returns. From sh documentation :

Normally, a script which exits with a nonzero status code will cause the step to fail with an exception.

You have to make sure that your script returns a nonzero status code when it fails. If you're not sure what your script returns, you can check the return value using the returnStatus param of the sh step, which will not fail the build but will return the status code. E.g:

def statusCode = sh script:script, returnStatus:true

You can then use this status code to set the result of your current build.

You can use :

  • currentBuild.result = 'FAILURE' or currentBuild.result = 'UNSTABLE' to mark the step as red/yellow respectively. In this case the build will still process the next steps.
  • error "Your error message" if you want the build to fail and exit immediately.


Related Topics



Leave a reply



Submit