In a Linux Shell How to Process Each Line of a Multiline String

In a Linux shell how can I process each line of a multiline string?

Use this (it is loop of reading each line from file file)

cat file | while read -r a; do echo $a; done

where the echo $a is whatever you want to do with current line.

UPDATE: from commentators (thanks!)

If you have no file with multiple lines, but have a variable with multiple lines, use

echo "$variable" | while read -r a; do echo $a; done

UPDATE2: "read -r" is recommended to disable backslashed (\) chars interpretation (check mtraceur comments; supported in most shells). It is documented in POSIX 1003.1-2008 http://pubs.opengroup.org/onlinepubs/9699919799/utilities/read.html

By default, unless the -r option is specified, <backslash> shall act as an escape character. .. The following option is supported: -r - Do not treat a <backslash> character in any special way. Consider each to be part of the input line.

Multi-line string with extra space (preserved indentation)

Heredoc sounds more convenient for this purpose. It is used to send multiple commands to a command interpreter program like ex or cat

cat << EndOfMessage
This is line 1.
This is line 2.
Line 3.
EndOfMessage

The string after << indicates where to stop.

To send these lines to a file, use:

cat > $FILE <<- EOM
Line 1.
Line 2.
EOM

You could also store these lines to a variable:

read -r -d '' VAR << EOM
This is line 1.
This is line 2.
Line 3.
EOM

This stores the lines to the variable named VAR.

When printing, remember the quotes around the variable otherwise you won't see the newline characters.

echo "$VAR"

Even better, you can use indentation to make it stand out more in your code. This time just add a - after << to stop the tabs from appearing.

read -r -d '' VAR <<- EOM
    This is line 1.
    This is line 2.
    Line 3.
EOM

But then you must use tabs, not spaces, for indentation in your code.

How to output a multiline string in Bash?

Here documents are often used for this purpose.

cat << EOF
usage: up [--level <n>| -n <levels>][--help][--version]

Report bugs to: 
up home page:
EOF

They are supported in all Bourne-derived shells including all versions of Bash.

Bash splitting a multi line string by a multi-character delimiter into an array

In case you don't want to change default RS value then could you please try following.

awk '{gsub("DELIMITER",ORS)} 1' Input_file

How to concatenate multiple lines of output to one line?

Use tr '\n' ' ' to translate all newline characters to spaces:

$ grep pattern file | tr '\n' ' '

Note: grep reads files, cat concatenates files. Don't cat file | grep!

Edit:

tr can only handle single character translations. You could use awk to change the output record separator like:

$ grep pattern file | awk '{print}' ORS='" '

This would transform:

one
two 
three

to:

one" two" three" 

How to parse multiple line output as separate variables

I'd use readarray to populate an array variable just in case there's spaces in your command's output that shouldn't be used as field separators that would end up messing up foo=( ... ). And you can use shell parameter expansion substring syntax to get the last character of a variable; no need for that awk bit in your var2:

#!/usr/bin/env bash

readarray -t lines < <(printf "%s\n" "Projekt/AX/include/ax.h" "Projekt/AX/src/ax.cpp")

for line in "${lines[@]}"; do
    printf "%s\n%s\n" "$line" "${line: -1}" # Note the space before the -1
done

will display

Projekt/AX/include/ax.h
h
Projekt/AX/src/ax.cpp
p

Read line by line from a variable in shell scripting

Consider the following multi-line variable

x=$(echo -e "a\nb\nc d e")

and a simple process for each line: just echo it with a prefix=LINE: and with single quotes around the line. Either of the following codes will satisfy that requirement:

while read line; do echo "LINE: '${line}'"; done <<< "$x"

or

while read line; do echo "LINE: '${line}'"; done < <(echo "$x")

Neither creates a subshell (so you can, e.g., set variables in the loop and access them outside of it), and both output

LINE: 'a'
LINE: 'b'
LINE: 'c d e'

But suppose instead you have

x=$(echo -e "a \n b\nc d e")
# note--------^--^

and that leading and trailing whitespace matter for your application (e.g., parsing Git porcelain). Both the above codes will give exactly the same output for the latter variable/data as for the former, which is not what you want. To preserve leading and trailing whitespace, replace while read line with while IFS= read -r line . I.e., either of the following codes

while IFS= read -r line; do echo "LINE: '${line}'"; done <<< "$x"

or

while IFS= read -r line; do echo "LINE: '${line}'"; done < <(echo "$x")

will produce

LINE: 'a '
LINE: ' b'
LINE: 'c d e'

See Greg Wooledge's excellent Bash FAQ for details.

---

Related Topics