Is It Ok to Use the Same Input File as Output of a Piped Command

Is it OK to use the same input file as output of a piped command?

No, it is not ok. All commands in a pipeline execute at the same time, and the shell prepares redirections before executing the commands. So, it is likely that the command will overwrite the file before cat reads it.

You need sponge(1) from moreutils.

Why piping to the same file doesn't work on some platforms?

In general this can be expected to break. The processes in a pipeline are all started up in parallel, so the > junk at the end of the line will usually truncate your input file before the process at the head of the pipelining has finished (or even started) reading from it.

Even if bash under Cygwin let's you get away with this you shouldn't rely on it. The general solution is to redirect to a temporary file and then rename it when the pipeline is complete.

How can I use a file in a command and redirect output to the same file without truncating it?

You cannot do that because bash processes the redirections first, then executes the command. So by the time grep looks at file_name, it is already empty. You can use a temporary file though.

#!/bin/sh
tmpfile=$(mktemp)
grep -v 'seg[0-9]\{1,\}\.[0-9]\{1\}' file_name > ${tmpfile}
cat ${tmpfile} > file_name
rm -f ${tmpfile}

like that, consider using mktemp to create the tmpfile but note that it's not POSIX.

How to use the output of 2 applications as the input of another (Windows/DOS batch files)?

The fc command expects command line arguments that specify files containing the input data to compare.

But fc does not read the console input (STDIN, hande 0; see Redirection), which is something completely different than that command line arguments, so you cannot use input redirection (<) or piping (|, with fc on the right side) to provide the input data.

So you will have to use temporary files, as you anyway already did.

Pipe output to two different commands

It should be ok if you use both tee and mkfifo.

mkfifo pipe
cat pipe | (command 1) &
echo 'test' | tee pipe | (command 2)

Use pipe of commands as argument for diff

If you're using bash:

diff file <(grep -E '^[0-9]+$' file)

The <(COMMAND) sequence expands to the name of a pseudo-file (such as /dev/fd/63) from which you can read the output of the command.

But for this particular case, ruakh's solution is simpler. It takes advantage of the fact that - as an argument to diff causes it to read its standard input. The <(COMMAND) syntax becomes more useful when both arguments to diff are command output, such as:

diff <(this_command) <(that_command)

How to assign a piped output as a variable while pipe is continuous

The problem is that you only run read (and then updates the status) once, so it reads a single line (and updates the status once). You need a loop, so it'll repeat the read+update process over & over. You can use a while loop to do this. If it should exit when there's no more input to process, make read the while condition:

aria2c $url --summary-interval=5 2>&1 |
    tee output.log |
    grep -oP "(\d+(\.\d+)?(?=%))" |
    while read text; do
        curl -s "https://api.legram.org/bot${tg_token}/editMessageText" --data "message_id=${msg_id}&text=DOWNLOADED-${text}&chat_id=${ch_id}&parse_mode=HTML&disable_web_page_preview=True"
    done

---

Related Topics