Using Source to Include Part of a File in a Bash Script

Source files in a bash script

Execute Shell Script Using . ./ (dot space dot slash)

While executing the shell script using “dot space dot slash”, as shown below, it will execute the script in the current shell without forking a sub shell.

$ . ./setup.bash

In other words, this executes the commands specified in the setup.bash in the current shell, and prepares the environment for you.

How to include file in a bash shell script

Simply put inside your script :

source FILE

Or

. FILE # POSIX compliant

$ LANG=C help source
source: source filename [arguments]
Execute commands from a file in the current shell.

Read and execute commands from FILENAME in the current shell.  The
entries in $PATH are used to find the directory containing FILENAME.
If any ARGUMENTS are supplied, they become the positional parameters
when FILENAME is executed.

Exit Status:
Returns the status of the last command executed in FILENAME; fails if
FILENAME cannot be read.

How best to include other scripts?

I tend to make my scripts all be relative to one another.
That way I can use dirname:

#!/bin/sh

my_dir="$(dirname "$0")"

"$my_dir/other_script.sh"

Sourcing the source files using bash script

By using dirname you get the directory of where the script is located, therefore it's easy to source other files locally close to your script and don't worry about specifying the correct path each time the script bundle is relocated.

For instance if you have in your script source $macros/some_script.sh then it will not break when the bundle is located in the /usr/local/bin/ or /bin/ or ...

Regarding $BASH_SOURCE see: https://stackoverflow.com/a/35006505/2146346

How to get the part of a file after the first line that matches a regular expression

The following will print the line matching TERMINATE till the end of the file:

sed -n -e '/TERMINATE/,$p'

Explained: -n disables default behavior of sed of printing each line after executing its script on it, -e indicated a script to sed, /TERMINATE/,$ is an address (line) range selection meaning the first line matching the TERMINATE regular expression (like grep) to the end of the file ($), and p is the print command which prints the current line.

This will print from the line that follows the line matching TERMINATE till the end of the file:
(from AFTER the matching line to EOF, NOT including the matching line)

sed -e '1,/TERMINATE/d'

Explained: 1,/TERMINATE/ is an address (line) range selection meaning the first line for the input to the 1st line matching the TERMINATE regular expression, and d is the delete command which delete the current line and skip to the next line. As sed default behavior is to print the lines, it will print the lines after TERMINATE to the end of input.

If you want the lines before TERMINATE:

sed -e '/TERMINATE/,$d'

And if you want both lines before and after TERMINATE in two different files in a single pass:

sed -e '1,/TERMINATE/w before
/TERMINATE/,$w after' file

The before and after files will contain the line with terminate, so to process each you need to use:

head -n -1 before
tail -n +2 after

IF you do not want to hard code the filenames in the sed script, you can:

before=before.txt
after=after.txt
sed -e "1,/TERMINATE/w $before
/TERMINATE/,\$w $after" file

But then you have to escape the $ meaning the last line so the shell will not try to expand the $w variable (note that we now use double quotes around the script instead of single quotes).

I forgot to tell that the new line is important after the filenames in the script so that sed knows that the filenames end.

How would you replace the hardcoded TERMINATE by a variable?

You would make a variable for the matching text and then do it the same way as the previous example:

matchtext=TERMINATE
before=before.txt
after=after.txt
sed -e "1,/$matchtext/w $before
/$matchtext/,\$w $after" file

to use a variable for the matching text with the previous examples:

## Print the line containing the matching text, till the end of the file:
## (from the matching line to EOF, including the matching line)
matchtext=TERMINATE
sed -n -e "/$matchtext/,\$p"

## Print from the line that follows the line containing the
## matching text, till the end of the file:
## (from AFTER the matching line to EOF, NOT including the matching line)
matchtext=TERMINATE
sed -e "1,/$matchtext/d"

## Print all the lines before the line containing the matching text:
## (from line-1 to BEFORE the matching line, NOT including the matching line)
matchtext=TERMINATE
sed -e "/$matchtext/,\$d"

The important points about replacing text with variables in these cases are:

1. Variables ($variablename) enclosed in single quotes ['] won't "expand" but variables inside double quotes ["] will. So, you have to change all the single quotes to double quotes if they contain text you want to replace with a variable.
2. The sed ranges also contain a $ and are immediately followed by a letter like: $p, $d, $w. They will also look like variables to be expanded, so you have to escape those $ characters with a backslash [\] like: \$p, \$d, \$w.

How to prevent sourcing a part of a bash script?

The problem is that the entire if/elif/else statement is parsed as a unit, so it can't contain invalid syntax.

What you could do is exit the sourced script before executing the zsh-specific code:

shell=$(ps -p $$ -oargs=)

if [ $shell = "bash" ]; then
    for f in ~/.functions.d/*.sh; do source $f; done
    return
fi

if [ $shell = "zsh" ]; then
    for f (~/.functions.d/**/*.sh) source $f
fi

However, a more general solution is to extract the bash-specific and zsh-specific code into separate scripts.

shell=$(ps -p $$ -oargs=)

if [ $shell = "bash" ]; then
    source load_functions.bash
fi

if [ $shell = "zsh" ]; then
    source load_functions.zsh
fi

load_functions.bash would contain the first for loop, while load_functions.zsh would have the second one.

           

How can I reference a file for variables using Bash?

The short answer

Use the source command.

---

An example using source

For example:

config.sh

#!/usr/bin/env bash
production="liveschool_joe"
playschool="playschool_joe"
echo $playschool

script.sh

#!/usr/bin/env bash
source config.sh
echo $production

Note that the output from sh ./script.sh in this example is:

~$ sh ./script.sh 
playschool_joe
liveschool_joe

This is because the source command actually runs the program. Everything in config.sh is executed.

---

Another way

You could use the built-in export command and getting and setting "environment variables" can also accomplish this.

Running export and echo $ENV should be all you need to know about accessing variables. Accessing environment variables is done the same way as a local variable.

To set them, say:

export variable=value

at the command line. All scripts will be able to access this value.

Find and Replace Inside a Text File from a Bash Command

The easiest way is to use sed (or perl):

sed -i -e 's/abc/XYZ/g' /tmp/file.txt

Which will invoke sed to do an in-place edit due to the -i option. This can be called from bash.

If you really really want to use just bash, then the following can work:

while IFS='' read -r a; do
    echo "${a//abc/XYZ}"
done < /tmp/file.txt > /tmp/file.txt.t
mv /tmp/file.txt{.t,}

This loops over each line, doing a substitution, and writing to a temporary file (don't want to clobber the input). The move at the end just moves temporary to the original name. (For robustness and security, the temporary file name should not be static or predictable, but let's not go there.)

For Mac users:

sed -i '' 's/abc/XYZ/g' /tmp/file.txt

(See the comment below why)

How do I get the directory where a Bash script is located from within the script itself?

#!/usr/bin/env bash

SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )

is a useful one-liner which will give you the full directory name of the script no matter where it is being called from.

It will work as long as the last component of the path used to find the script is not a symlink (directory links are OK). If you also want to resolve any links to the script itself, you need a multi-line solution:

#!/usr/bin/env bash

SOURCE=${BASH_SOURCE[0]}
while [ -L "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
  DIR=$( cd -P "$( dirname "$SOURCE" )" >/dev/null 2>&1 && pwd )
  SOURCE=$(readlink "$SOURCE")
  [[ $SOURCE != /* ]] && SOURCE=$DIR/$SOURCE # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
done
DIR=$( cd -P "$( dirname "$SOURCE" )" >/dev/null 2>&1 && pwd )

This last one will work with any combination of aliases, source, bash -c, symlinks, etc.

Beware: if you cd to a different directory before running this snippet, the result may be incorrect!

Also, watch out for $CDPATH gotchas, and stderr output side effects if the user has smartly overridden cd to redirect output to stderr instead (including escape sequences, such as when calling update_terminal_cwd >&2 on Mac). Adding >/dev/null 2>&1 at the end of your cd command will take care of both possibilities.

To understand how it works, try running this more verbose form:

#!/usr/bin/env bash

SOURCE=${BASH_SOURCE[0]}
while [ -L "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
  TARGET=$(readlink "$SOURCE")
  if [[ $TARGET == /* ]]; then
    echo "SOURCE '$SOURCE' is an absolute symlink to '$TARGET'"
    SOURCE=$TARGET
  else
    DIR=$( dirname "$SOURCE" )
    echo "SOURCE '$SOURCE' is a relative symlink to '$TARGET' (relative to '$DIR')"
    SOURCE=$DIR/$TARGET # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
  fi
done
echo "SOURCE is '$SOURCE'"
RDIR=$( dirname "$SOURCE" )
DIR=$( cd -P "$( dirname "$SOURCE" )" >/dev/null 2>&1 && pwd )
if [ "$DIR" != "$RDIR" ]; then
  echo "DIR '$RDIR' resolves to '$DIR'"
fi
echo "DIR is '$DIR'"

And it will print something like:

SOURCE './scriptdir.sh' is a relative symlink to 'sym2/scriptdir.sh' (relative to '.')
SOURCE is './sym2/scriptdir.sh'
DIR './sym2' resolves to '/home/ubuntu/dotfiles/fo fo/real/real1/real2'
DIR is '/home/ubuntu/dotfiles/fo fo/real/real1/real2'

---

Related Topics