Bash Loop to Compare Files

How would you loop through files in a directory and compare them to each other?

#!/usr/bin/env bash

typeset -r diffs=diffs.txt
typeset -a allfiles=()
typeset -- filename=''

# fills the allfiles array with all *.txt files except the diffs.txt
# that can be found from the current directory and down all sub-directories
while IFS= read -r -d '' filename; do
  allfiles+=("$filename")
done < <(
  find . -type f -name '*.txt' -and -not -name "$diffs" -print0 2>/dev/null
)

[[ ${#allfiles[@]} -lt 2 ]] && exit 2 # Need at least 2 files to compare

typeset -i i=0 j=0
typeset -- file_a='' file_b=''
export LC_MESSAGES=POSIX
# for all files except last
for ((i = 0; i < ${#allfiles[@]} - 1; i++)); do
  file_a="${allfiles[$i]}"
  # for next file to last file
  for ((j = i + 1; j < ${#allfiles[@]}; j++)); do
    file_b="${allfiles[$j]}"
    diff --report-identical-files --unified=0 --minimal -- \
      "$file_a" "$file_b" 2>/dev/null
    echo
  done
done >"$diffs" # all output to the diffs file

Bash loop to compare files

I'm not going to argue. I simply try to avoid double read from inside loops.

#! /bin/bash

cat md5.txt | while read sum file
do
    prev_sum=$(md5sum $file | awk '{print $1}')
    if [ "$sum" != "$prev_sum" ]
    then
        echo "md5 of file $file does not match"
    else
        echo "$file is fine"
    fi
done

Looping through the content of a file in Bash

One way to do it is:

while read p; do
  echo "$p"
done <peptides.txt

As pointed out in the comments, this has the side effects of trimming leading whitespace, interpreting backslash sequences, and skipping the last line if it's missing a terminating linefeed. If these are concerns, you can do:

while IFS="" read -r p || [ -n "$p" ]
do
  printf '%s\n' "$p"
done < peptides.txt

---

Exceptionally, if the loop body may read from standard input, you can open the file using a different file descriptor:

while read -u 10 p; do
  ...
done 10<peptides.txt

Here, 10 is just an arbitrary number (different from 0, 1, 2).

Nested for loop comparing files

try this

#!/bin/bash
cd /Path/dir1/
for f in *; do
    comm -13 <(sort $f) <(sort /Path/dir2/$f) > /Path/dir3/$f
done

if syntax in shell is

if test_command;then commands;fi

commands are executed if test_command exit code is 0

if [ $f = $i ] ; then  grep ... ; fi

but in your case it will be more efficient to get the file name

for i in $FILES; do
    f=/Path/dir2/`basename $i`
    grep
done

finally, maybe this will be more efficient than grep -v

comm -13 <(sort $f) <(sort $i)

comm -13 will get everything which is in the second and not in first ; comm without arguments generates 3 columns of output : first is only in first, second only in second and third what is common.
-13 or -1 -3 removes first and third column

Bash script to compare files

You have the {} in the wrong places:

if [ $( diff {$file} {$filex} ) == 0 ]

They should be at:

if [ $( diff ${file} ${filex} ) == 0 ]

(though the braces are optional now), but you should allow for spaces in the file names:

if [ $( diff "${file}" "${filex}" ) == 0 ]

Now it simply doesn't work properly because when diff finds no differences, it generates no output (and you get errors because the == operator doesn't expect nothing on its left-side). You could sort of fix it by double quoting the value from $(…) (if [ "$( diff … )" == "" ]), but you should simply and directly test the exit status of diff:

if diff "${file}" "${filex}"
then : no difference
else : there is a difference
fi

and maybe for comparing images you should be using cmp (in silent mode) rather than diff:

if cmp -s "$file" "$filex"
then : no difference
else : there is a difference
fi

---

Related Topics