Replace Bash Variables in Template File

How to replace ${} placeholders in a text file?

Update

Here is a solution from yottatsa on a similar question that only does replacement for variables like $VAR or ${VAR}, and is a brief one-liner

i=32 word=foo envsubst < template.txt

Of course if i and word are in your environment, then it is just

envsubst < template.txt

On my Mac it looks like it was installed as part of gettext and from MacGPG2

Old Answer

Here is an improvement to the solution from mogsie on a similar question, my solution does not require you to escale double quotes, mogsie's does, but his is a one liner!

eval "cat <<EOF
$(<template.txt)
EOF
" 2> /dev/null

The power on these two solutions is that you only get a few types of shell expansions that don't occur normally $((...)), `...`, and $(...), though backslash is an escape character here, but you don't have to worry that the parsing has a bug, and it does multiple lines just fine.

Replace bash variables in template file

Security tips

This don't take care of security issues! Using eval is evil!

The compatible answer is not better!

Of course, you have to be confident about the content of your template!!

If else, try using sed! (see my last answer)

Quick way bash only!:

Under bash you can simply:

eval "INSTALLPATH='/somepath/somewhere' SITEURL='example.com' PHPSERV='127.0.0.1:9000'; echo \"$(<template)\""

or

eval "INSTALLPATH='/somepath/somewhere'
    SITEURL='example.com'
    PHPSERV='127.0.0.1:9000';
    echo \"$(<template)\""

As you're using eval, you could store your resulting config file into one variable:

eval "INSTALLPATH='/somepath/somewhere'
    SITEURL='example.com'
    PHPSERV='127.0.0.1:9000';
    cfgBody=\"$(<template)\""

Then

echo "$cfgBody"

and/or

echo "$cfgBody" >/cfgpath/cfgfile

Doing this into a loop

tmplBody="$(<template)"
while read INSTALLPATH SITEURL PHPSERV CFGFILE;do
    [ "$CFGFILE" ] && eval "echo \"$tmplBody\"" >"$CFGFILE"
  done <<<"
    /somepath/somewhere            example.com  127.0.0.1:9000  /tmp/file1
    '/some\ other\ path/elsewhere' sample2.com  127.0.0.1:9001  /tmp/file2
"

Note: On second line, there are escaped spaces (prepanded with a backshash \ and quotes '. The backslash tell read to not split the variable, and the quotes have to be added into the resulting /tmp/file2.

Soft way (compatible answer)

Under posix shell, you may do this way:

#!/bin/sh

(
    cat <<eohead
    #!/bin/sh
    INSTALLPATH='/somepath/somewhere'
    SITEURL='example.com'
    PHPSERV='127.0.0.1:9000';
    cat <<eof
eohead
    cat template
    echo eof
) | /bin/sh

This don't require bash, was tested under dash and busybox.

bash Without eval !

sedcmd=''
for var in INSTALLPATH SITEURL PHPSERV;do
    printf -v sc 's/${%s}/%s/;' $var "${!var//\//\\/}"
    sedcmd+="$sc"
  done
sed -e "$sedcmd" <template

Could work into a loop:

while read INSTALLPATH SITEURL PHPSERV CFGFILE;do
    if  [ "$CFGFILE" ] ;then
        sedcmd=''
        for var in INSTALLPATH SITEURL PHPSERV;do
            printf -v sc 's/${%s}/%s/;' $var "${!var//\//\\/}"
            sedcmd+="$sc"
          done
        sed -e "$sedcmd" <template >"$CFGFILE"
      fi
  done <<<"
    /somepath/somewhere             example.com  127.0.0.1:9000  /tmp/file1
    '/some\ other\ path/elsewhere'  sample2.com  127.0.0.1:9001  /tmp/file2
"

Compatible answer, using sed

This could work without so called bashisms, into a loop:

#!/bin/sh

while read INSTALLPATH SITEURL PHPSERV CFGFILE;do

    sedcmd="s|\\\${INSTALLPATH}|${INSTALLPATH}|;"
    sedcmd="${sedcmd}s|\\\${SITEURL}|${SITEURL}|;"
    sedcmd="${sedcmd}s|\\\${PHPSERV}|${PHPSERV}|;"

    sed -e "$sedcmd" template >"$CFGFILE"

  done <<eof
    /somepath/somewhere             example.com  127.0.0.1:9000  /tmp/file1
    '/some\ other\ path/elsewhere'  sample2.com  127.0.0.1:9001  /tmp/file2
eof

Comparing outputs:

diff -u99 /tmp/file{1,2}
--- /tmp/file1        2015-05-31 11:02:03.407463963 +0200
+++ /tmp/file2        2015-05-31 11:02:03.407463963 +0200
@@ -1,22 +1,22 @@
 server {
     listen 80;
-    root /somepath/somewhere;
-    server_name example.com;
+    root '/some other path/elsewhere';
+    server_name sample2.com;

     client_max_body_size 20m;
     client_body_timeout 120s;

     location / {
         try_files /public/router.php =404;
         fastcgi_split_path_info ^(.+?\.php)(/.*)$;
-        fastcgi_pass 127.0.0.1:9000;
+        fastcgi_pass 127.0.0.1:9001;
         fastcgi_index router.php;
         fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
         include /etc/nginx/fastcgi_params;
     }

     location /assets {
         try_files /app/$uri =404;
     }

 }

Bash Templating: How to build configuration files from templates with Bash?

You can use this:

perl -p -i -e 's/\$\{([^}]+)\}/defined $ENV{$1} ? $ENV{$1} : $&/eg' < template.txt

to replace all ${...} strings with corresponding enviroment variables (do not forget to export them before running this script).

For pure bash this should work (assuming that variables do not contain ${...} strings):

#!/bin/bash
while read -r line ; do
    while [[ "$line" =~ (\$\{[a-zA-Z_][a-zA-Z_0-9]*\}) ]] ; do
        LHS=${BASH_REMATCH[1]}
        RHS="$(eval echo "\"$LHS\"")"
        line=${line//$LHS/$RHS}
    done
    echo "$line"
done

. Solution that does not hang if RHS references some variable that references itself:

#!/bin/bash
line="$(cat; echo -n a)"
end_offset=${#line}
while [[ "${line:0:$end_offset}" =~ (.*)(\$\{([a-zA-Z_][a-zA-Z_0-9]*)\})(.*) ]] ; do
    PRE="${BASH_REMATCH[1]}"
    POST="${BASH_REMATCH[4]}${line:$end_offset:${#line}}"
    VARNAME="${BASH_REMATCH[3]}"
    eval 'VARVAL="$'$VARNAME'"'
    line="$PRE$VARVAL$POST"
    end_offset=${#PRE}
done
echo -n "${line:0:-1}"

WARNING: I do not know a way to correctly handle input with NULs in bash or preserve the amount of trailing newlines. Last variant is presented as it is because shells “love” binary input:

1. read will interpret backslashes.
2. read -r will not interpret backslashes, but still will drop the last line if it does not end with a newline.
3. "$(…)" will strip as many trailing newlines as there are present, so I end … with ; echo -n a and use echo -n "${line:0:-1}": this drops the last character (which is a) and preserves as many trailing newlines as there was in the input (including no).

How do I replace template variables in text file with data in bash script

The easiest is use bash itself:

original=$(cat file.txt)
read -r -d '' replacement <<'EOF'
        "arn:aws:ecs:*:588068252125:cluster/${aws:PrincipalTag/Service}-*"   \
        "arn:aws:ecs:*:588068252125:task/${aws:PrincipalTag/Service}-*" \
        "arn:aws:ecs:*:588068252125:container-instance/${aws:PrincipalTag/Service}-*"  \
        "arn:aws:ecs:*:588068252125:task-definition/${aws:PrincipalTag/Service}-*:*"  \
        "arn:aws:ecs:*:588068252125:service/${aws:PrincipalTag/Service}-*"  \
EOF
placeholder='"<ARN>"'
modified=${original/$placeholder/$replacement}
echo "$modified"

Look for ${parameter/pattern/string} in man bash.

Read template file and replace variable using Bash

Place single quotes (not double) around $line:

eval echo '$line'

Edit: I misunderstood the question previously. The solution does not require an eval, but can be done using a simple substitution:

line=${line/\${website_name\}/$website_name}
echo $line

Using sed to replace text in a template file with variable containing quotes

Save this to file "mksql.bash" or so...

sed "s/REPLACEME/$(sed "s/.*/'&'/" | paste -s -d, -)/" < template_file.txt

and use it like:

bash mksql.bash < your_id_file

will produce

select distinct tt.transaction_id||','||tt.entity_id||','||cm.user_id
from transaction_tracker tt, course_membership cm
tt.entity_id = cm.course_id and
tt.transaction_id IN ('41aeb908-dfc7-4cf8-8285-31ca184dc1c5','da877ffa-49bc-4f07-b692-4873870fcb37','a555cdd0-e100-42cb-a355-140de7958b36');

the "template_file.txt" contain your sql template:

select distinct tt.transaction_id||','||tt.entity_id||','||cm.user_id
from transaction_tracker tt, course_membership cm
tt.entity_id = cm.course_id and
tt.transaction_id IN (REPLACEME);

I've replaced the much XX..X with the "REPLACEME" because i'm lazy counting Xes

btw, welcome in SO... :)

Sed to replace a script variable by certificate

Pulling out of comments to get better visibility ...

Consider running the files through base64 and embedding the result into the script, then on the other end run base64 -d to decrypt the data and store in the target files.

Using base64 encoded data should eliminate most (all?) of the sed headaches of dealing with special characters and/or trying to find a sed script delimiter that's not in the data.

---

OP/Manitoba's reply comment:

That did the trick. I used HARBOR_CRT=$(sudo cat /data/harbor.crt | base64 -w 0) to convert certificate to B64 and echo $CFG_HARBOR_CRT | base64 --decode to decode.

---

Related Topics