Forcing Bash to Expand Variables in a String Loaded from a File

Forcing bash to expand variables in a string loaded from a file

I stumbled on what I think is THE answer to this question: the envsubst command:

echo "hello \$FOO world" > source.txt
export FOO=42
envsubst < source.txt

This outputs: hello 42 world

If you would like to continue work on the data in a file destination.txt, push this back to a file like this:

envsubst < source.txt > destination.txt

In case it's not already available in your distro, it's in the
GNU package gettext.

@Rockallite

  • I wrote a little wrapper script to take care of the '$' problem.

(BTW, there is a "feature" of envsubst, explained at
https://unix.stackexchange.com/a/294400/7088
for expanding only some of the variables in the input, but I
agree that escaping the exceptions is much more convenient.)

Here's my script:

#! /bin/bash
## -*-Shell-Script-*-
CmdName=${0##*/}
Usage="usage: $CmdName runs envsubst, but allows '\$' to keep variables from
being expanded.
With option -sl '\$' keeps the back-slash.
Default is to replace '\$' with '$'
"

if [[ $1 = -h ]] ;then echo -e >&2 "$Usage" ; exit 1 ;fi
if [[ $1 = -sl ]] ;then sl='\' ; shift ;fi

sed 's/\\\$/\${EnVsUbDolR}/g' | EnVsUbDolR=$sl\$ envsubst "$@"

How to expand shell variables in a text file?

This solution is not elegant, but it works. Create a script call shell_expansion.sh:

echo 'cat <<END_OF_TEXT' >  temp.sh
cat "$1" >> temp.sh
echo 'END_OF_TEXT' >> temp.sh
bash temp.sh >> "$2"
rm temp.sh

You can then invoke this script as followed:

bash shell_expansion.sh Text_File.msh Text_File_expanded.msh

Bash expand variable in a variable

Using eval is the classic solution, but bash has a better (more easily controlled, less blunderbuss-like) solution:

  • ${!colour}

The Bash (4.1) reference manual says:

If the first character of parameter is an exclamation point (!), a level of variable indirection
is introduced. Bash uses the value of the variable formed from the rest of parameter as
the name of the variable; this variable is then expanded and that value is used in the rest
of the substitution, rather than the value of parameter itself. This is known as indirect
expansion
.

For example:

$ Green=$'\033[32;m'
$ echo "$Green" | odx
0x0000: 1B 5B 33 32 3B 6D 0A .[32;m.
0x0007:
$ colour=Green
$ echo $colour
Green
$ echo ${!colour} | odx
0x0000: 1B 5B 33 32 3B 6D 0A .[32;m.
0x0007:
$

(The odx command is very non-standard but simply dumps its data in a hex format with printable characters shown on the right. Since the plain echo didn't show anything and I needed to see what was being echoed, I used an old friend I wrote about 24 years ago.)

How can I force bash to expand a variable to pass it as an argument?

The order in which the shell parses various aspects of the command line is not obvious, and it matters for things like this.

First, the wildcards aren't expanded at declaration, they're expanded after the variable value is substituted (note: in these examples I'll pretend I have your filesystem):

$ var1=*
$ echo "$var1" # double-quotes prevent additional parsing of the variable's value
*
$ echo $var1 # without double-quotes, variable value undergoes wildcard expansion and word splitting
Audiobooks:

Downloads:
(etc)

BTW, the ~ is expanded at declaration, confusing things even further:

$ var2=~
$ echo "$var2" # again, double-quotes let me see what's actually in the variable
/home/yajo

The problem with ~/rpmbuild/{SRPMS,RPMS/*}/enki-*.rpm1 is that while the shell does wildcard expansion (*) on the value after substitution, it doesn't do brace expansion ({SRPMS,RPMS/*}), so it's actually looking for directory names with braces and commas in the name... and not finding any.

The best way to handle this is generally to store the file list as an array; if you do this right, everything gets expanded at declaration:

$ var2=(~/rpmbuild/{SRPMS,RPMS/*}/enki-*.rpm)
$ echo "${var2[@]}" # This is the proper way to expand an array into a word list
/home/yajo/rpmbuild/RPMS/noarch/enki-12.10.3-1.fc18.noarch.rpm etc...

Note that arrays are a bash extension, and will not work in plain POSIX shells. So be sure to start your script with #!/bin/bash, not #!/bin/sh.

String expansion - escaped quoted variable to value

My question is, can you get the variable to expand correctly, e.g., print "/home//bash_file" and "/expanded/variable/path"?

Yes you can use envsubst program, that substitutes the values of environment variables:

while read -r sourceFile; do
envsubst <<< "$sourceFile"
done < <(sed -n "s/^\(source\|\.\|\$include\) //p" "$PATH_SOME_SCRIPT")

Copy expanded bash variable to another variable

Bash will perform wildcard expansion for unquoted variables. Assuming that you defined a pattern in a variable (like var1='PATTERN') you can force expansion by placing it outside any quotes:

function foobar {
echo ARGS="$@"
}

var1='/home/user/file*'
foobar $var1

In the above example, foobar will be called with all matching files, each as separate argument. You can replace foobar with external command. Same effect.

Forcing expansion into other variables is a little bit more tricky and risky. Assuming the pattern source is trusted, one can use:

var1='/home/user/file*'
set -- $var1
var3=$*

Execute bash within some text file

Thank you to @joshmeranda for pointing me in the right direction, this solved my problem

echo -e "$(eval "echo -e \"`<template.yaml`\"")"

While eval can be dangerous, in my case its usage is controlled.



Related Topics



Leave a reply



Submit