How can I loop over the output of a shell command?
Never for
loop over the results of a shell command if you want to process it line by line unless you are changing the value of the internal field separator $IFS
to \n
. This is because the lines will get subject of word splitting which leads to the actual results you are seeing. Meaning if you for example have a file like this:
foo bar
hello world
The following for loop
for i in $(cat file); do
echo "$i"
done
gives you:
foo
bar
hello
world
Even if you use IFS='\n'
the lines might still get subject of Filename expansion
I recommend to use while
+ read
instead because read
reads line by line.
Furthermore I would use pgrep
if you are searching for pids belonging to a certain binary. However, since python might appear as different binaries, like python2.7
or python3.4
I suggest to pass -f
to pgrep
which makes it search the whole command line rather than just searching for binaries called python
. But this will also find processes which have been started like cat foo.py
. You have been warned! At the end you can refine the regex passed to pgrep
like you wish.
Example:
pgrep -f python | while read -r pid ; do
echo "$pid"
done
or if you also want the process name:
pgrep -af python | while read -r line ; do
echo "$line"
done
If you want the process name and the pid in separate variables:
pgrep -af python | while read -r pid cmd ; do
echo "pid: $pid, cmd: $cmd"
done
You see, read
offers a flexible and stable way to process the output of a command line-by-line.
Btw, if you prefer your ps .. | grep
command line over pgrep
use the following loop:
ps -ewo pid,etime,cmd | grep python | grep -v grep | grep -v sh \
| while read -r pid etime cmd ; do
echo "$pid $cmd $etime"
done
Note how I changed the order of etime
and cmd
. Thus to be able to read cmd
, which can contain whitespace, into a single variable. This works because read
will break down the line into variables, as many times as you specified variables. The remaining part of the line - possibly including whitespace - will get assigned to the last variable which has been specified in the command line.
Looping through the content of a file in Bash
One way to do it is:
while read p; do
echo "$p"
done <peptides.txt
As pointed out in the comments, this has the side effects of trimming leading whitespace, interpreting backslash sequences, and skipping the last line if it's missing a terminating linefeed. If these are concerns, you can do:
while IFS="" read -r p || [ -n "$p" ]
do
printf '%s\n' "$p"
done < peptides.txt
Exceptionally, if the loop body may read from standard input, you can open the file using a different file descriptor:
while read -u 10 p; do
...
done 10<peptides.txt
Here, 10 is just an arbitrary number (different from 0, 1, 2).
How to loop through a list in shell?
Use a bash
while-loop
, the loop can be done over a command or an input file.
while IFS= read -r string
do
some_stuff to do
done < <(command_that_produces_string)
With an example, I have a sample file with contents as
$ cat file
My
name
is not
relevant
here
I have modified the script to echo the line as it reads through the file
$ cat script.sh
#!/bin/bash
while IFS= read -r string
do
echo "$string"
done < file
produces an o/p when run as ./script.sh
My
name
is not
relevant
here
The same can also be done over a bash-command, where we adopt process-substitution (
<())
to run the command on the sub-shell.
#!/bin/bash
while IFS= read -r -d '' file; do
echo "$file"
done < <(find . -maxdepth 1 -mindepth 1 -name "*.txt" -type f -print0)
The above simple find
lists all files from the current directory (including ones with spaces/special-characters). Here, the output of find
command is fed to stdin
which is parsed by while-loop
.
ansible loop over shell command output
I needed to do this with_indexed_items: "{{ vcs.stdout.split('\n')}}"
How to process each output line in a loop?
One of the easy ways is not to store the output in a variable, but directly iterate over it with a while/read loop.
Something like:
grep xyz abc.txt | while read -r line ; do
echo "Processing $line"
# your code goes here
done
There are variations on this scheme depending on exactly what you're after.
If you need to change variables inside the loop (and have that change be visible outside of it), you can use process substitution as stated in fedorqui's answer:
while read -r line ; do
echo "Processing $line"
# your code goes here
done < <(grep xyz abc.txt)
Related Topics
What's the Difference Between Nohup and Ampersand
How to Get the Total Cpu Usage of an Application from /Proc/Pid/Stat
Have Bash Script Answer Interactive Prompts
Get Destination Address of a Received Udp Packet
What Is the Runtime Performance Cost of a Docker Container
Ld Cannot Find an Existing Library
Is There Any API For Determining the Physical Address from Virtual Address in Linux
How to Set the Working Directory of the Parent Process
How to Use Local Docker Images With Minikube
More Elegant "Ps Aux | Grep -V Grep"
Forcing Bash to Expand Variables in a String Loaded from a File
Pipe To/From the Clipboard in a Bash Script
Find and Replace With Sed in Directory and Sub Directories
How to Permanently Export a Variable in Linux
What Is Better "Int 0X80" or "Syscall" in 32-Bit Code on Linux
Calling Printf in X86_64 Using Gnu Assembler