Why does this shell pipeline exit?
After base64
outputs 10 bytes, head
gets enough inputs and exits. When the former attempts to output more bytes, it will receive SIGPIPE signal and hence exit too。For the same reason, cat
will exit in turn.
Shell pipe: Exit immediately when one command fails
The bash documentation says in its section about pipelines:
Each command in a pipeline is executed in its own subshell [...]
"In its own subshell" means that a new bash process is spawned, which then gets to execute the actual command. Each subshell starts successfully, even when it immediately determines that the command it is asked to execute doesn't exist.
This explains why the entire pipe can be set up successfully even when one of the commands is nonsense. Bash does not check if each command can be run, it delegates that to the subshells. That also explains why, for example, the command nonexisting-command | touch hello
will throw a "command not found" error, but the file hello
will be created nonetheless.
In the same section, it also says:
The shell waits for all commands in the pipeline to terminate before returning a value.
In sleep 5 | nonexisting-command
, as A.H. pointed out, the sleep 5
terminates after 5 seconds, not immediately, hence the shell will also wait 5 seconds.
I don't know why the implementation was done this way. In cases like yours, the behavior is surely not as one would expect.
Anyway, one slightly ugly workaround is to use FIFOs:
mkfifo myfifo
./long-running-script.sh > myfifo &
whoops-a-typo < myfifo
Here, the long-running-script.sh
is started and then the scripts fails immediately on the next line. Using mutiple FIFOs, this could be extended to pipes with more than two commands.
Pipe output and capture exit status in Bash
There is an internal Bash variable called $PIPESTATUS
; it’s an array that holds the exit status of each command in your last foreground pipeline of commands.
<command> | tee out.txt ; test ${PIPESTATUS[0]} -eq 0
Or another alternative which also works with other shells (like zsh) would be to enable pipefail:
set -o pipefail
...
The first option does not work with zsh
due to a little bit different syntax.
Exit when one process in pipe fails
The main issue at hand here is clearly the pipe. In bash, when executing a command of the form
command1 | command2
and command2
dies or terminates, the pipe which receives the output (/dev/stdout
) from command1
becomes broken. The broken pipe, however, does not terminate command1
. This will only happen when it tries to write to the broken pipe, upon which it will exit with sigpipe
. A simple demonstration of this can be seen in this question.
If you want to avoid this problem, you should make use of process substitution in combination with input redirection. This way, you avoid pipes. The above pipeline is then written as:
command2 < <(command1)
In the case of the OP, this would become:
./script.sh < <(tee /dev/stderr) | tee /dev/stderr
which can also be written as:
./script.sh < <(tee /dev/stderr) > >(tee /dev/stderr)
Why does exit not exit my script when I pipe it to tee?
As covered elsewhere (including, somewhat indirectly, BashFAQ #24), pipelines create subshells, so your exit
exits only the subshell.
You can avoid this by replacing the pipeline with a redirection to a process substitution:
{ echo "hello world"; exit 1; } > >(tee -a logfile.log)
Generate other exit behavior if output from pipeline is empty
I would use the PIPESTATUS to check the exit code of egrep:
#!/bin/bash
# construct regex from input
# set FILE according to input file
egrep "${regex}" "${FILE}" | doing stuff | sort
if [[ ${PIPESTATUS[0] != 0 ]]; then
echo "error message" 1>&2
exit 3
fi
Some context:
${PIPESTATUS[@]} is just an array wich contains the exit code of every program you chained up. $? will just give you the exit code of the last command in the pipe.
Jenkins pipeline bubble up the shell exit code to fail the stage
The sh
step returns the same status code that your actual sh command (your script in this case) returns. From sh documentation :
Normally, a script which exits with a nonzero status code will cause the step to fail with an exception.
You have to make sure that your script returns a nonzero status code when it fails. If you're not sure what your script returns, you can check the return value using the returnStatus
param of the sh
step, which will not fail the build but will return the status code. E.g:
def statusCode = sh script:script, returnStatus:true
You can then use this status code to set the result of your current build.
You can use :
currentBuild.result = 'FAILURE'
orcurrentBuild.result = 'UNSTABLE'
to mark the step as red/yellow respectively. In this case the build will still process the next steps.error "Your error message"
if you want the build to fail and exit immediately.
Related Topics
Update Specific Field in Text File in Specific Line
Dockerfile Cmd 'Command Not Found'
How Come _Exit(0) (Exiting by Syscall) Prevents Me from Receiving Any Stdout Content
Where Is $Path Set? Specifically Where Is My MAC Port Path Being Set
How to Run Ionic in the Background
Error When Installing Opencv on Ubuntu 14.04
Write to Port 0Cf8H Fails with Segfault
Permission Denied (Publickey), on Linux Aws Server How to Fix It
How to Install Asp on Linux Ubuntu Hardy
Compute Base64 Encoded Hash from a Given Hash
Dotnetcore: Cross Platform Version of Getinvalidfilenamechars
Bash - Calculate the Average of Numbers Inputted
Overhead of Supporting Floating Point Arithmetic Inside the Linux Kernel
Automating Killall Then Killall Level 9
How to Make My .Bat File Run Linux Command to Remote Linux