Read Line by Line in Bash Script

Read a file line by line assigning the value to a variable

The following reads a file passed as an argument line by line:

while IFS= read -r line; do
    echo "Text read from file: $line"
done < my_filename.txt

This is the standard form for reading lines from a file in a loop. Explanation:

• IFS= (or IFS='') prevents leading/trailing whitespace from being trimmed.
• -r prevents backslash escapes from being interpreted.

Or you can put it in a bash file helper script, example contents:

#!/bin/bash
while IFS= read -r line; do
    echo "Text read from file: $line"
done < "$1"

If the above is saved to a script with filename readfile, it can be run as follows:

chmod +x readfile
./readfile filename.txt

If the file isn’t a standard POSIX text file (= not terminated by a newline character), the loop can be modified to handle trailing partial lines:

while IFS= read -r line || [[ -n "$line" ]]; do
    echo "Text read from file: $line"
done < "$1"

Here, || [[ -n $line ]] prevents the last line from being ignored if it doesn't end with a \n (since read returns a non-zero exit code when it encounters EOF).

If the commands inside the loop also read from standard input, the file descriptor used by read can be chanced to something else (avoid the standard file descriptors), e.g.:

while IFS= read -r -u3 line; do
    echo "Text read from file: $line"
done 3< "$1"

(Non-Bash shells might not know read -u3; use read <&3 instead.)

Read line by line in Bash script

What you have is piping the text "cat test" into the loop.

You just want:

cat test | \
while read CMD; do
    echo $CMD
done

How to read file line by line in Bash script?

Your code leads me to believe you want each line in one variable.

Try this script (I know this can be done easier and prettier, but this is a simple and readable example):

#!/bin/bash
file="cam.txt"

while read -r line
do
    printf 'Line: %s\n' "$line"

    current=$line
    last=$current
    secondlast=$last

    printf 'Loop: %s %s %s\n' "$current" "$last" "$secondlast"
done < $file

printf 'After: %s %s %s\n' "$current" "$last" "$secondlast"

Simpler version:

{ read -r first; read -r second; read -r third; } <cam.txt
printf 'After: %s %s %s\n' "$first" "$second" "$third"

Read line by line from standard input Bash

1. How do I read line by line from standard input in Bash? Until now I
   used "read string" but I do not think that it reads a line at a time.

The prototype for read is:

read [options] name[s ...]

read will read a line of input into name name1 name2 ... splitting the line based on the contents of the Internal Field Separator (IFS). The default for IFS is ' \t\n' (that is space tab newline). If you only provide a single variable to read, you will read the entire line into that variable (unless you have set a new delimiter with the -d option to read). If you provide more than one variable, (e.g. read -r name name1) word splitting will occur based on the current value of IFS. Meaning if you provide the string hello world to:

read -r name

name="hello world". On the other hand, if you provide the same string to:

read -r name name1

name="hello", name1="world". What if you have excess words in the line but only 2 variables? Say your string is now "hello big wide world", what happens with:

read -r name name1

name="hello", name1="big wide world". The words in string are assigned to your variables in order and if there are insufficient variables to hold each word in the string, the last variable will contain all remaining words in the string not previously assigned.

You change how word splitting occurs by altering IFS. Take a close look at the answer provided by anubhava for an example. You are free to specify any character you would like the words to be split on. (helpful in say parsing a csv file to set IFS=$',\n' and have the words split on ',' instead of space)

To ensure you read an entire line into a variable, you can provide only a single variable to read and set IFS='$\n' to ensure word splitting only occurs on newline. (Note: providing the change as part of the while loop limits the IFS alteration to the scope of that loop. For example:

while IFS='$\n' read -r line; do
    # do whatever with line
done

Will ensure that each line on stdin will be read into line while preserving normal word-splitting outside the loop. Inside the loop you can then add each line to an array as anubhava shows in his answer. (to preserve all whitespace IFS= is used)

Looping through the content of a file in Bash

One way to do it is:

while read p; do
  echo "$p"
done <peptides.txt

As pointed out in the comments, this has the side effects of trimming leading whitespace, interpreting backslash sequences, and skipping the last line if it's missing a terminating linefeed. If these are concerns, you can do:

while IFS="" read -r p || [ -n "$p" ]
do
  printf '%s\n' "$p"
done < peptides.txt

---

Exceptionally, if the loop body may read from standard input, you can open the file using a different file descriptor:

while read -u 10 p; do
  ...
done 10<peptides.txt

Here, 10 is just an arbitrary number (different from 0, 1, 2).

read file line by line and sum each line individually

You can do this fairly easily in bash itself making use of built-in parameter expansions to trim leading zeros from the beginning of each line in order to sum the digits for odd / even.

When reading from a file (either a named file or stdin by default), you can use the initialization with default to use the first argument (positional parameter) as the filename (if given) and if not, just read from stdin, e.g.

#!/bin/bash

infile="${1:-/dev/stdin}"     ## read from file provide as $1 or stdin

Which you will use infile with your while loop, e.g.

while read -r line; do        ## loop reading each line
  ...
done < "$infile"

To trim the leading zeros, first obtain the substring of leading zeros trimming all digits from the right until only zeros remain, e.g.

  leading="${line%%[1-9]*}"                         ## get leading 0's

Now using the same type parameter expansion with # instead of %% trim the leading zeros substring from the front of line saving the resulting number in value, e.g.

  value="${line#$leading}"                          ## trim from front

Now zero your sum and loop over the digits in value to obtain the sum of digits:

  for ((i=0;i<${#value};i++)); do                   ## loop summing digits
    sum=$((sum + ${value:$i:1}))
  done

All that remains is your even / odd test. Putting it altogether in a short example script that intentionally outputs the sum of digits in addition to your wanted "odd" / "even" output, you could do:

#!/bin/bash

infile="${1:-/dev/stdin}"     ## read from file provide as $1 or stdin

while read -r line; do                              ## read each line
  [ "$line" -eq "$line" 2>/dev/null ] || continue   ## validate integer
  
  leading="${line%%[1-9]*}"                         ## get leading 0's
  value="${line#$leading}"                          ## trim from front
  sum=0                                             ## zero sum
  
  for ((i=0;i<${#value};i++)); do                   ## loop summing digits
    sum=$((sum + ${value:$i:1}))
  done
  
  printf "%s (sum=%d) - " "$line" "$sum"            ## output line w/sum
                                                    ## (temporary output)
  if ((sum % 2 == 0)); then                         ## check odd / even
    echo "even"
  else
    echo "odd"
  fi
done < "$infile"

(note: you can actually loop over the digits in line and skip removing the leading zeros substring. The removal ensure that if the whole value is used it isn't interpreted as an octal value -- up to you)

Example Use/Output

Using a quick process substitution to provide input of 001 - 020 on stdin you could do:

$ ./sumdigitsoddeven.sh < <(printf "%03d\n" {1..20})
001 (sum=1) - odd
002 (sum=2) - even
003 (sum=3) - odd
004 (sum=4) - even
005 (sum=5) - odd
006 (sum=6) - even
007 (sum=7) - odd
008 (sum=8) - even
009 (sum=9) - odd
010 (sum=1) - odd
011 (sum=2) - even
012 (sum=3) - odd
013 (sum=4) - even
014 (sum=5) - odd
015 (sum=6) - even
016 (sum=7) - odd
017 (sum=8) - even
018 (sum=9) - odd
019 (sum=10) - even
020 (sum=2) - even

You can simply remove the output of "(sum=X)" when you have confirmed it operates as you expect and redirect the output to your new file. Let me know if I understood your question properly and if you have further questions.

---

Related Topics