Looping through lines in a file in bash, without using stdin
You must open as a different file descriptor
while read p <&3; do
echo "$p"
echo 'Hit enter for the next one'
read x
done 3< list.txt
Update: Just ignore the lengthy discussion in the comments below. It has nothing to do with the question or this answer.
Looping through the content of a file in Bash
One way to do it is:
while read p; do
echo "$p"
done <peptides.txt
As pointed out in the comments, this has the side effects of trimming leading whitespace, interpreting backslash sequences, and skipping the last line if it's missing a terminating linefeed. If these are concerns, you can do:
while IFS="" read -r p || [ -n "$p" ]
do
printf '%s\n' "$p"
done < peptides.txt
Exceptionally, if the loop body may read from standard input, you can open the file using a different file descriptor:
while read -u 10 p; do
...
done 10<peptides.txt
Here, 10 is just an arbitrary number (different from 0, 1, 2).
Looping through a file but ignoring the first two lines in Bash
Given the small number of lines (2) to be ignored, I suggest the following:
{
read; read
while IFS="" read -r p || [ -n "$p" ] ; do
...
done
} < diff.txt
read; read
reads the first two lines from stdin (diff.txt).
How do I iterate over each line in a file with Bash?
Use cat
for concatenating or displaying. No need for it here.
file="/path/to/file"
while read line; do
echo "${line}"
done < "${file}"
bash script won't loop over lines in file when certain command executed
The problem was the same as:
I'm reading a file line by line and running ssh or ffmpeg, only the first line gets processed!
so the solution was to redirect stdin:
rosbag play -r 10 $bag </dev/null
reading a file line by line using the shell
while IFS= read -r line < input_file
do
echo "$line"
done
In this scenario, you are re-initiating the read of input_file with every iteration of the loop. As read reads line by line, each re-initialisation will print the first line. As you never progress past this first line, it will print the first line only indefinitely
while IFS= read -r line
do
echo "$line"
done < input_file
In this scenario, the input_file is redirected in to the while loop and then the read line takes place with each line of the redirected output until it gets to the end of the output and then end of the file.
An alternate way of processing the file is to cat file_input through to the while loop:
cat input_file | while IFS= read -r line
do
echo "$line"
done
This works in the same way as the previous section but is less efficient due to the pipe. The flow of the output runs from left to right as opposed to right to left. This is often easier to understand. The output of cat input_file is fed (piped) into the while loop and each line is read till the end of the output/file.
Using bash for loop to iterate through file
The for
loop takes its iteration values from its in
clause or from the content of $@
; it does not read from stdin.
In your last example, the entire command's stdin is being redirected from a.txt
and it produces no output because for
doesn't read from stdin and its command block's commands don't either.
The read
command, however, does read from stdin and that's why the while read arg;
loop works.
Credit to Glenn for explaining the meaning of a for
command without an in
clause:
The value of $@
is empty and that is why your for
loop produces no output. Redirecting a command's stdin from a file does not set $@
. If you had placed that for
loop in a script and executed it with some arguments, then your for
loop would have printed the command line's arguments instead of the content of a.txt
.
Read a file line by line assigning the value to a variable
The following reads a file passed as an argument line by line:
while IFS= read -r line; do
echo "Text read from file: $line"
done < my_filename.txt
This is the standard form for reading lines from a file in a loop. Explanation:
IFS=
(orIFS=''
) prevents leading/trailing whitespace from being trimmed.-r
prevents backslash escapes from being interpreted.
Or you can put it in a bash file helper script, example contents:
#!/bin/bash
while IFS= read -r line; do
echo "Text read from file: $line"
done < "$1"
If the above is saved to a script with filename readfile
, it can be run as follows:
chmod +x readfile
./readfile filename.txt
If the file isn’t a standard POSIX text file (= not terminated by a newline character), the loop can be modified to handle trailing partial lines:
while IFS= read -r line || [[ -n "$line" ]]; do
echo "Text read from file: $line"
done < "$1"
Here, || [[ -n $line ]]
prevents the last line from being ignored if it doesn't end with a \n
(since read
returns a non-zero exit code when it encounters EOF).
If the commands inside the loop also read from standard input, the file descriptor used by read
can be chanced to something else (avoid the standard file descriptors), e.g.:
while IFS= read -r -u3 line; do
echo "Text read from file: $line"
done 3< "$1"
(Non-Bash shells might not know read -u3
; use read <&3
instead.)
Related Topics
How to Access the Base Filename of a File You Are Sourcing in Bash
How to Manage Log Verbosity Inside a Shell Script
Comma Separated Values to Single Inverted Quote and Comma Separated Values
Npm Install Causes Errors Like Npm Err! Tar.Unpack Untar Error on Debian
How to Extract One Column from Multiple Files, and Paste Those Columns into One File
Linux Serial Port Buffer Not Empty When Opening Device
How to Cross Compile R Packages for MACos from a Linux Environment
Calculate Mean of Each Column Ignoring Missing Data with Awk
Converting a Pcap Trace to Netflow Format
Termios Vmin Vtime and Blocking/Non-Blocking Read Operations
What Is the Use of Gfp_User Flag in Kmalloc
What Really Is the "Linger Time" That Can Be Set with So_Linger on Sockets
Best Way to Make Linux Web Services
Ksh Storing Result of a Command to a Variable
If Adding a Command That Repeats Every 10 Minutes to Crontab, When Does the First Job Run