How to Convert Multiline File into a String in Bash with Newline Character

How to convert multiline file into a string in bash with newline character?

One way using awk:

$ awk '$1=$1' ORS='\\n' file
-----BEGIN CERTIFICATE-----\nMIIDBjCCMIIDB\nMIIDBjCCMIIDB\n....\nMIIDBjCCMIIDB==\n-----END CERTIFICATE-----\n

Convert multiline string in a variable in bash to a single-line string with newlines escaped

Bash has two built-in ways to quote values suitable for re-ingestion. These will handle not only newlines but also tabs, quotes, and backslashes:

❯ echo "${str@Q}"
$'\nHello\nWorld\n'

❯ printf '%q\n' "$str"
$'\nHello\nWorld\n'

Alternatively, if you simply want to replace newlines and nothing else you can use ${var//search/replace} syntax to do replacements:

❯ echo "${str//$'\n'/\\n}"
\nHello\nWorld\n

Convert multiline string to array

Set IFS (Internal Field Separator). Shell uses the IFS variable to determine what the field separators are. By default, IFS is set to the space character. Change it to the newline character, as demonstrated below:

#!/bin/bash
names="Netgear
Hon Hai Precision Ind. Co.
Apple"
    
SAVEIFS=$IFS   # Save current IFS (Internal Field Separator)
IFS=$'\n'      # Change IFS to newline char
names=($names) # split the `names` string into an array by the same name
IFS=$SAVEIFS   # Restore original IFS

for (( i=0; i<${#names[@]}; i++ ))
do
    echo "$i: ${names[$i]}"
done

Output

0: Netgear
1: Hon Hai Precision Ind. Co.
2: Apple

Multi-line string with extra space (preserved indentation)

Heredoc sounds more convenient for this purpose. It is used to send multiple commands to a command interpreter program like ex or cat

cat << EndOfMessage
This is line 1.
This is line 2.
Line 3.
EndOfMessage

The string after << indicates where to stop.

To send these lines to a file, use:

cat > $FILE <<- EOM
Line 1.
Line 2.
EOM

You could also store these lines to a variable:

read -r -d '' VAR << EOM
This is line 1.
This is line 2.
Line 3.
EOM

This stores the lines to the variable named VAR.

When printing, remember the quotes around the variable otherwise you won't see the newline characters.

echo "$VAR"

Even better, you can use indentation to make it stand out more in your code. This time just add a - after << to stop the tabs from appearing.

read -r -d '' VAR <<- EOM
    This is line 1.
    This is line 2.
    Line 3.
EOM

But then you must use tabs, not spaces, for indentation in your code.

Convert specified column in a multi-line string into single comma-separated line

You can use awk and sed:

awk -vORS=, '{ print $2 }' file.txt | sed 's/,$/\n/'

Or if you want to use a pipe:

echo "data" | awk -vORS=, '{ print $2 }' | sed 's/,$/\n/'

To break it down:

awk is great at handling data broken down into fields
-vORS=, sets the "output record separator" to ,, which is what you wanted
{ print $2 } tells awk to print the second field for every record (line)
file.txt is your filename
sed just gets rid of the trailing , and turns it into a newline (if you want no newline, you can do s/,$//)

Multiline output to single line concatenation with delimiter string

You can also do it with awk

git diff --name-only HEAD master |
awk 'NR > 1 {printf("||")} {printf("file:%s",$0)} END{print ""}'

What this awk does is:

for all input lines but the first, output: ||
for all input lines, output: file: + <input line content>
when there's no more input lines to read, output a newline character

remark: the END{print ""} is unrequited when you do out=$(git .. | awk ...)

How to Convert Multiline File into a String in Bash with Newline Character