Bash Tries to Execute Commands in Heredoc

Bash tries to execute commands in heredoc

Quote EOF so that bash takes inputs literally:

cat <<'EOF' > what.txt
a=`echo $1 | awk -F. '{print $NF}'`
b=`echo $2 | tr '[:upper:]' '[:lower:]'`
EOF

Also start using $() for command substitution instead of old and problematic ``.

How to execute curl command stored in heredoc in bash script?

I learned from this post to make heredoc work with curl command.

As the comment made by @Gordon Davisson in current post, we should not mix command with data. Since the json data set to -d option is only data and other parts is command, so I decide to use heredoc to store only the json data and remain other parts to be command itself, rather than store them in string by heredoc.

The result bash script should be something like this:

#/bin/bash

response3=$(curl -v www.stackoverflow.com \
  -d @- <<- MULTI_STRING_SCOPE
        {
            "hello":"world"
        }
    MULTI_STRING_SCOPE
  )

Notice: heredoc indent only works with tab, not with blanks. Be careful, especially when you are working with editors like Visual Studio Code, which may have already set indent as blanks for you.

Invoke function from bash heredoc

You'll need to transfer the code verbatim to the other server to run it there. A heredoc with an unquoted starting delimiter works similarly to a double quoted string, which means any command substitutions like the one you have running before the resulting string is passed to the remote server. You can solve this by putting the function definition inside the heredoc and single quoting the starting delimiter:

ssh myserver <<'EOF'
_git_clone() {
    git clone myrepo
    git fetch --all --tags
}
echo $(_git_clone)
EOF

Alternatively you can define the function locally but then substitute its definition within the heredoc:

_git_clone() {
    git clone myrepo
    git fetch --all --tags
}

ssh myserver <<EOF
$(declare -f _git_clone)
EOF

Running commands in a script as another user using a here doc

What is happening is, the embedded $(...) command gets executed before the here-document is passed to su. That is, the actual script that is passed to su is something more like this:

/usr/bin/ruby -e "#!/System/Library/Frameworks/Ruby.framework/Versions/Current/usr/bin/ruby
# This script installs to /usr/local only. To install elsewhere you can just
# untar https://github.com/Homebrew/brew/tarball/master anywhere you like or
# change the value of HOMEBREW_PREFIX.
HOMEBREW_PREFIX = "/usr/local".freeze
HOMEBREW_REPOSITORY = "/usr/local/Homebrew".freeze
HOMEBREW_CACHE = "#{ENV["HOME"]}/Library/Caches/Homebrew".freeze
...

And so on. In other words, the output of $(...) got inserted into the here-document.

To avoid that, you need to escape the $:

su - User << EOF

/usr/bin/ruby -e "\$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" </dev/null

EOF

Alternatively, you can tell the shell to treat the entire here-document literally without any interpolation, by enclosing the starting EOF within double-quotes:

su - User << "EOF"

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" </dev/null

EOF

How can I execute all script parameters that use here doc?

The heredoc will replace stdin for your script. If you want it to be accessed as a parameter, use command substitution like

./sshpass ssh $user@$server /bin/bash $(cat << EOF
    echo "do this..."
    echo "do that..."
    echo "and the other..."
EOF
)

although that probably won't end up doing exactly what you want, because it will be passing each word as it's own positional argument, so you will be running

ssh $user@$server echo "do this..." echo "do that..." echo "and the other..."

which will have the first echo get all the rest as arguments. You'll need semi colons at the end of each command, and to have quotes around the whole thing so you don't do some of it remotely and some of it locally. So I should have recommended it as:

./sshpass ssh $user@$server /bin/bash "$(cat << EOF
    echo 'do this...';
    echo 'do that...';
    echo 'and the other...'
EOF
)"

but this still gives me an uneasy feeling as it's very likely easy to "do the wrong thing" with something like this

Using && after a heredoc in bash

Chaining commands in a single line

You can put the control operator && right after the EOF word in your here document and you can chain more than one command:

cat > file <<-EOF && echo -n "hello " && echo world

It will wait for your here-document and then will print hello world.

Example

$ cat > file <<-EOF && echo -n "hello " && echo world
> a
> b
> EOF
hello world

$ cat file
a
b

---

Chaining commands after the heredoc delimiter

Now, if you want to place the following commands after the heredoc, you can group it in curly braces and continue chaining commands as follows:

echo -n "hello " && { cat > file <<-EOF
a
b
EOF
} && echo world

Example

$ echo -n "hello " && { cat > file <<-EOF
> a
> b
> EOF
> } && echo world
hello world

$ cat file
a
b

---

Using the set built in

If you're going to use set [-+]e instead of chained commands with &&, you must notice that surrounding a chunk of code with set -e and set +e is not a direct alternative and you must take care of the following:

Surrounding dependent commands with set [-+]e

echo first_command
false # it doesn't stop the execution of the script

# surrounded commands
set -e
echo successful_command_a
false # here stops the execution of the script
echo successful_command_b
set +e

# this command is never reached
echo last_command

As you can see, if you need to go on executing commands after the surrounded commands, this solution doesn't work.

Grouping Commands to the rescue

Instead, you can group the surrounded commands in order to create a subshell as follows:

echo first_command
false # it doesn't stop the execution of the script

# surrounded commands executed in a subshell
(
set -e
echo successful_command_a
false # here stops the execution of the group
echo successful_command_b
set +e # actually, this is not needed here
)

# the script is alive here
false # it doesn't stop the execution of the script
echo last_command

So, if you need to execute something else after your chained commands and you want to use the set builtin, consider the examples above.

Also notice the following about subshells:

Command substitution, commands grouped with parentheses, and asynchronous commands are invoked in a subshell environment that is a duplicate of the shell environment, except that traps caught by the shell are reset to the values that the shell inherited from its parent at invocation. Builtin commands that are invoked as part of a pipeline are also executed in a subshell environment. Changes made to the subshell environment cannot affect the shell’s execution environment.

Execute loop instruction from here DOC

You need to use a single quoted here document:

sudo /usr/bin/su - glob  << 'EOF'
[…]
EOF

Otherwise shell substitutions such as $(…) are executed in the current context and not as the other user. This is similar to how variables and other shell substitutions in single quoted strings are not expanded.

How can I write a heredoc to a file in Bash script?

Read the Advanced Bash-Scripting Guide Chapter 19. Here Documents.

Here's an example which will write the contents to a file at /tmp/yourfilehere

cat << EOF > /tmp/yourfilehere
These contents will be written to the file.
        This line is indented.
EOF

Note that the final 'EOF' (The LimitString) should not have any whitespace in front of the word, because it means that the LimitString will not be recognized.

In a shell script, you may want to use indentation to make the code readable, however this can have the undesirable effect of indenting the text within your here document. In this case, use <<- (followed by a dash) to disable leading tabs (Note that to test this you will need to replace the leading whitespace with a tab character, since I cannot print actual tab characters here.)

#!/usr/bin/env bash

if true ; then
    cat <<- EOF > /tmp/yourfilehere
    The leading tab is ignored.
    EOF
fi

If you don't want to interpret variables in the text, then use single quotes:

cat << 'EOF' > /tmp/yourfilehere
The variable $FOO will not be interpreted.
EOF

To pipe the heredoc through a command pipeline:

cat <<'EOF' |  sed 's/a/b/'
foo
bar
baz
EOF

Output:

foo
bbr
bbz

... or to write the the heredoc to a file using sudo:

cat <<'EOF' |  sed 's/a/b/' | sudo tee /etc/config_file.conf
foo
bar
baz
EOF

---

Related Topics