Linux Bash Script for Loop

Shell script for loop syntax

Brace expansion, {x..y} is performed before other expansions, so you cannot use that for variable length sequences.

Instead, use the seq 2 $max method as user mob stated.

So, for your example it would be:

max=10
for i in `seq 2 $max`
do
    echo "$i"
done

Looping through the content of a file in Bash

One way to do it is:

while read p; do
  echo "$p"
done <peptides.txt

As pointed out in the comments, this has the side effects of trimming leading whitespace, interpreting backslash sequences, and skipping the last line if it's missing a terminating linefeed. If these are concerns, you can do:

while IFS="" read -r p || [ -n "$p" ]
do
  printf '%s\n' "$p"
done < peptides.txt

---

Exceptionally, if the loop body may read from standard input, you can open the file using a different file descriptor:

while read -u 10 p; do
  ...
done 10<peptides.txt

Here, 10 is just an arbitrary number (different from 0, 1, 2).

Bash script - for loop and if-else

UPDATE

This is the working complete code for this particular question, I'll keep the original slightly more generic code below for future reference since it is less complex (less nested if/else) for others searching for similar problems.

for i in {1..4};
  do
   if hdfs dfs -test -e $mypath/*;
     then
       if [ $i -eq 4 ]
        then
          echo "script exiting now with code 1"
       else
          echo "Folder is full"
          sleep 10
       fi
   else
    echo "Folder is empty"
    break
  fi
done

---

I think something like this will work, which is pretty much what you have. You just have to put what you're checking for in the if statement and this code should work for you.

for i in {1..4}
do
   if [ <check for subdirs> ]
   then 
       echo "Folder is empty!" 
       break
   else
       sleep 1800
   fi
done

For loop in command line runs bash script reading from text file line by line

A while loop is probably what you need. Put the space separated strings in the file text.file :

cat text.file
bingo yankee
bravo delta

Then write the script in question like below.

#!/bin/bash
while read -r arg1 arg2
do
 /path/to/your/script.sh "$arg1" "$arg2"
done<text.file

How can I loop over the output of a shell command?

Never for loop over the results of a shell command if you want to process it line by line unless you are changing the value of the internal field separator $IFS to \n. This is because the lines will get subject of word splitting which leads to the actual results you are seeing. Meaning if you for example have a file like this:

foo bar
hello world

The following for loop

for i in $(cat file); do
    echo "$i"
done

gives you:

foo
bar
hello
world

Even if you use IFS='\n' the lines might still get subject of Filename expansion

---

I recommend to use while + read instead because read reads line by line.

Furthermore I would use pgrep if you are searching for pids belonging to a certain binary. However, since python might appear as different binaries, like python2.7 or python3.4 I suggest to pass -f to pgrep which makes it search the whole command line rather than just searching for binaries called python. But this will also find processes which have been started like cat foo.py. You have been warned! At the end you can refine the regex passed to pgrep like you wish.

Example:

pgrep -f python | while read -r pid ; do
    echo "$pid"
done

or if you also want the process name:

pgrep -af python | while read -r line ; do
    echo "$line"
done

If you want the process name and the pid in separate variables:

pgrep -af python | while read -r pid cmd ; do
    echo "pid: $pid, cmd: $cmd"
done

You see, read offers a flexible and stable way to process the output of a command line-by-line.

---

Btw, if you prefer your ps .. | grep command line over pgrep use the following loop:

ps -ewo pid,etime,cmd | grep python | grep -v grep | grep -v sh \
  | while read -r pid etime cmd ; do
    echo "$pid $cmd $etime"
done

Note how I changed the order of etime and cmd. Thus to be able to read cmd, which can contain whitespace, into a single variable. This works because read will break down the line into variables, as many times as you specified variables. The remaining part of the line - possibly including whitespace - will get assigned to the last variable which has been specified in the command line.

Bash script for loop with if-else statement

You are just missing a few spaces ...

#!/bin/bash 
a="ACC_A"
b="ACC_B" 
c="ACC_C"
d="ACC_D"

USERS="ACC_A ACC_B ACC_C ACC_X ACC_Y ACC_Z" 

for ACCOUNT in $USERS; do
        if [ "$ACCOUNT" = "$a" ] || [ "$ACCOUNT" = "$b" ] || [ "$ACCOUNT" = "$c" ] || [ "$ACCOUNT" = "$d" ]; then
              echo "Skipping $ACCOUNT"
    else
              echo "this $ACCOUNT"
    fi
done

and a quick test:

$ bash accounts.sh 
Skipping ACC_A
Skipping ACC_B
Skipping ACC_C
this ACC_X
this ACC_Y
this ACC_Z

for loop only get executed once | bash shell

If $folder contains new lines, try the loop like that:

while IFS= read -r entry; do
    echo "... $entry ..."
done <<< "$folder"

Edit: See also Bash: Iterating over lines in a variable

Bash for loop not writing to file

As already noted in the comments, opening foo.csv for output will truncate it in most shells. (Even if that was not the case, opening the file and running cut and paste repeatedly looks quite inefficient.)

If you don’t mind keeping all the data in memory at one point in time, a simple AWK or Bash script can do this type of processing without any further processes such as cut or paste.

awk -F'\t' '    { lines[FNR] = lines[FNR] "\t" $5 }
            END { for (l in lines) print substr(lines[l], 2) }' \
    *.txt > foo.csv

(The output should not be called .csv, but I’m sticking with the naming from the question nonetheless.)

Actually, one doesn’t really need awk for this, Bash will do:

#!/bin/bash
lines=()
for file in *.txt; do
  declare -i i=0
  while IFS=$'\t' read -ra line; do
    lines[i++]+=$'\t'"${line[4]}"
  done < "$file"
done
printf '%s\n' "${lines[@]/#?}" > foo.csv

(As a side note, "${lines[@]:1}" would remove the first line, not the first (\t) character of each line. (This particular expansion syntax works differently for strings (scalars) and arrays in Bash.) Hence "${lines[@]/#?}" (another way to express the removal of the first character), which does get applied to each array element.)

Running multiple loop at once in bash script

You need to use a different variable for the loop than the initial array. Otherwise you are overwriting the array during the first iteration.

Here I'm using different variables for the array and the loop variable:

#!/bin/bash

regions=('us-east-1' 'eu-central-1')
envs=('prod' 'stage')
    
for region in "${regions[@]}"
do
    for env in "${envs[@]}"
    do
        echo "$region"
        echo "$env"
    done
done

Output:

us-east-1
prod
us-east-1
stage
eu-central-1
prod
eu-central-1
stage

---

PS: Another option would be to use a string of multiple words and have bash iterating over that:

for a in foo bar
do
    for b in baz qux
    do
        echo "${a}"
        echo "${b}"
    done
done

Looping over a fixed string or variable string in bash script

The fundamental misunderstanding here is as follows: The shell doesn't loop over strings; it only loops over lists of what it calls "words". When you perform an unquoted expansion (like $var), unless it's in a context that implicitly suppresses string splitting (also called "word splitting"), the contents of $var are split on characters in IFS (by default: tabs, spaces and newlines) to form a list of words, and then each of those words is expanded as a glob (so *.txt can be replaced with a.txt b.txt c.txt -- or file one.txt, for that matter, if a matching name containing spaces exists in the current directory).

In for x in "one two three four", one two three four -- because it was quoted -- is a single word. By contrast, for x in one two three four or for x in "one" "two" "three" "four" would iterate over each of one, two, three, and four as a distinct word.

When you run var="one two three four"; for x in $var, then $var is -- because the expansion is unquoted -- split into multiple words, so one is a word, two a second one, three a third, four a fourth.

On the other hand, if you ran for x in "$var", the quotes would suppress string splitting and globbing, so the inside of the loop would only run once, with the entire string held together.

---

The best practice is not to use for x in $var at all. When you want to have a list of things in bash, use an array to store that list.

#!/usr/bin/env bash
#              ^^^^ - NOT sh, which doesn't support arrays

var=( "first word" "second word" )
for x in "${var[@]}"; do
  echo "$x"
done

---

Related Topics