Linux Command Line Using for Loop and Formatting Results

Bash: For Loop & save each output as a new column in a csv

Something like

echo "Sample,Total_Reads" > Combined_Outputs.csv
for file in *.bam; do
    printf "%s,%s\n" "$file" "$(samtools view -c "$file")"
done >> Combined_Outputs.csv

Print one line for each file, and move the output redirection outside of the loop for efficiency.

Bash run string formatted command in a for loop

First, there's no need for using string formatting to generate code here, or in general -- BashFAQ #50 describes the use cases and better-practice alternatives. This could be as simple as the following:

for j in in "${foo[@]}"; do touch "${j}_file.txt"; done

---

Second, if you must, do it like so:

printf -v cmd 'touch %q_file.txt; ' "${foo[@]}"
eval "$cmd"

This will set cmd='touch bar_file.txt; touch baz_file.txt ', and then execute it.

Use of %q when content is to be later parsed by a shell (as with eval) ensures that your array elements are formatted in a way that will survive shell-parsing -- so if you had foo=( "name with spaces" 'name with $(rm -rf $HOME) dangerous content' ), you would be correctly touching 'name with spaces_file.txt' and 'name with $(rm -rf $HOME) dangerous content_file.txt', not executing the dangerous content and touching multiple files based on the spaces.

for loop output formatting: add newline with a description and space

This will merge them as you require it:

for f in /home/user/*.txt;do
    echo "This is ${f##*/}" >> /home/user/catfiles.txt
    /bin/cat "${f}" >> /home/user/catfiles.txt
    echo >> /home/user/catfiles.txt
done

The file name is printed without its path. If you want the path printed, too, then simply replace ${f##*/} with ${f}.

Update

${variable##pattern} is called parameter substitution. Bash will search for pattern in variable and remove the longest match. In this case the variable is f and pattern is */ which matches any String that ends with a slash. The double hash ## indicates to remove the longest String in f that can be matched by */. Since f is a path it will match and remove everything up to and including the last /, thus leaving only the filename.
The Bash documentation has further and more detailed info on this subject. Alternatively, you can search in man bash for Parameter Expansion.

Looping through the content of a file in Bash

One way to do it is:

while read p; do
  echo "$p"
done <peptides.txt

As pointed out in the comments, this has the side effects of trimming leading whitespace, interpreting backslash sequences, and skipping the last line if it's missing a terminating linefeed. If these are concerns, you can do:

while IFS="" read -r p || [ -n "$p" ]
do
  printf '%s\n' "$p"
done < peptides.txt

---

Exceptionally, if the loop body may read from standard input, you can open the file using a different file descriptor:

while read -u 10 p; do
  ...
done 10<peptides.txt

Here, 10 is just an arbitrary number (different from 0, 1, 2).

how to loop through string for patterns from linux shell?

A pipe through tr can split those strings out to separate lines:

grep -hx -- ':[:[:alnum:]]*:' ~/Documents/wiki{,/diary}/*.mkd | tr -s ':' '\n'

This will also remove the colons and an empty line will be present in the output (easy to repair, note the empty line will always be the first one due to the leading :). Add sort -u to sort and remove duplicates, or awk '!seen[$0]++' to remove duplicates without sorting.

An approach with sed:

sed '/^:/!d;s///;/:$/!d;s///;y/:/\n/' ~/Documents/wiki{,/diary}/*.mkd

This also removes colons, but avoids adding empty lines (by removing the leading/trailing : with s before using y to transliterate remaining : to <newline>). sed could be combined with tr:

sed '/:$/!d;/^:/!d;s///' ~/Documents/wiki{,/diary}/*.mkd | tr -s ':' '\n'

Using awk to work with the : separated fields, removing duplicates:

awk -F: '/^:/ && /:$/ {for (i=2; i<NF; ++i) if (!seen[$i]++) print $i}' \
~/Documents/wiki{,/diary}/*.mkd

Save output command in a variable and write for loop

Better than storing in a variable, use this :

find . -type f -name "*.jpg" -exec command {} \;

Even, if you want, command can be a full bloated shell script.

A demo is better than an explanation, no ? Copy paste the whole lines in a terminal :

cat<<'EOF' >/tmp/test
#!/bin/bash

echo "I play with $1 and I can replay with $1, even 3 times: $1"
EOF
chmod +x /tmp/test
find . -type f -name "*.jpg" -exec /tmp/test {} \;

Edit: new demo (from new questions from comments)

find . -type f -name "*.jpg" | head -n 10 | xargs -n1 command

(this another solution doesn't take care of filenames with newlines or spaces)

This one take care :

#!/bin/bash

shopt -s globstar

count=0

for file in **/*.jpg; do
    if ((++count < 10)); then
        echo "process file $file number $count"
    else
        break
    fi
done

how can i loop through the coming frequency of the keyword

This sounds very much like a homework assignment.

c.f. BashFAQ for better reads; keeping this simple to focus on what you asked for.

Rewritten for more precise formatting -

while read key                          # read each search key 
do cnt=$(grep "$key" test|wc -l)        # count the hits
   pad="$key,searchall,$cnt,"           # build the "header" fields
   while read line                      # read the input from grep
   do if [[ "$line" =~ ^-- ]]           # treat hits separately
      then pad="$key,searchall,$cnt,"   # reset the "header"
           echo                         # add the blank line
           continue                     # skip to next line of data
      fi
      echo "$pad$line"                  # echo "header" and data
      pad="${pad//?/ }"                 # convert header to spacving
   done < <( grep -B1 -A1 "$key" test ) # pull hits for this key
   echo                                 # add blank lines between
done < word.txt                         # set stdin for the outer read                       

$: cat word.txt
course
red hat

$: ./tst
course,searchall,1,centos is my bro
                   red hat is my course
                   ubuntu is my OS

red hat,searchall,2,centos is my bro
                    red hat is my course
                    ubuntu is my OS

red hat,searchall,2,tom outsmart jerry
                    red hat is my boy
                    jerry is samall

---

Related Topics