Iterating Over File (And Directory) Names with Bash

How to loop over files in directory and change path and add suffix to filename

A couple of notes first: when you use Data/data1.txt as an argument, should it really be /Data/data1.txt (with a leading slash)? Also, should the outer loop scan only for .txt files, or all files in /Data? Here's an answer, assuming /Data/data1.txt and .txt files only:

#!/bin/bash
for filename in /Data/*.txt; do
    for ((i=0; i<=3; i++)); do
        ./MyProgram.exe "$filename" "Logs/$(basename "$filename" .txt)_Log$i.txt"
    done
done

Notes:

• /Data/*.txt expands to the paths of the text files in /Data (including the /Data/ part)
• $( ... ) runs a shell command and inserts its output at that point in the command line
• basename somepath .txt outputs the base part of somepath, with .txt removed from the end (e.g. /Data/file.txt -> file)

If you needed to run MyProgram with Data/file.txt instead of /Data/file.txt, use "${filename#/}" to remove the leading slash. On the other hand, if it's really Data not /Data you want to scan, just use for filename in Data/*.txt.

Loop through files in directory specified using argument

This would happen if the directory is empty, or misspelled. The shell (in its default configuration) simply doesn't expand a wildcard if it has no matches. (You can control this in Bash with shopt -s nullglob; with this option, wildcards which don't match anything are simply removed.)

You can verify this easily for yourself. In a directory with four files,

sh$ echo *
a       file    or      two

sh$ echo [ot]*
or      two

sh$ echo n*
n*

And in Bash,

bash$ echo n*
n*

bash$ shopt -s nullglob

bash$ echo n*

I'm guessing you are confused about how the current working directory affects the resolution of directory names; maybe read Difference between ./ and ~/

How to iterate files in directory with for loop in Bash

The for-loop is basically sound. However, if the directory is empty, the loop will be executed once, with the variable file containing the literal text /var/spool/bandit24/*.

The stat message is not from the for-loop, but from one of the commands in the loop.

The correct way would be to test if the directory is empty before you continue. You could put something like

if [ $(find . -type f | wc -l) -eq 0 ] ; then
    echo "Nothing to do"
    exit 0
fi

right after the cd.

Some other comments on your script.

• If you do a cd in the script, you don't need to specify the full path anymore.
• Your quoting is not really consistent. That may not be a problem if your file names never contain spaces or strange characters, but I would, for example timeout -s 9 60 "./$file" and rm -f "./file"
• /var/spool/bandit/* will never contain . or .., so that test is useless.
• You could also replace the test with if [ -f "$file" ] ; then

Iterating over file (and directory) names with bash

Use a wild card: for file in *; do …; done. That keeps the spaces in the names correct. Consider shopt -s nullglob too. Neither your code nor my suggestion lists names starting with a dot ..

Also, use if [ -d "$file" ] with double quotes around the variable value to avoid spacing problems.

Hence:

#!/bin/bash

shopt -s nullglob
files=0
dir=0
for file in *
do
    if [ -d "$file" ]
    then
        dir=$(($dir+1))
    else
        files=$(($files+1))
    fi
done 
echo "files=$files, directories=$dir"

In Bash, there are also other ways of writing the arithmetic, such as ((files++)).

Loop through files in a given directory

If you only want the files non-recursively in the current directory, combine what you have:

read -p 'Enter the directory path: ' directory
for file in "$directory"/*; do
  echo "$file"
done

If you want to loop recursively and you have bash 4, it's not much harder:

shopt -s globstar
for file in "$directory"/**/*; do …

But if you only have bash 3, you'd be better off using find.

find "$directory"

loop over files and extract part of filename

Just re-assign the loop variable at the beginning of each iteration:

for sample in *.vcf; do
    sample=${sample%_*}
    # do stuff here
done

bash iterate over a directory sorted by file size

You can use a mix of find (to print file sizes and names), sort (to sort the output of find) and cut (to remove the sizes). In case you have very unusual file names containing any possible character including newlines, it is safer to separate the files by a character that cannot be part of a name: NUL.

#/bin/bash

if [ -z "$1" ]; then
echo "Please supply the filename suffixes to delete.";
exit;
fi;

filter=$1;

while IFS= read -r -d '' -u 3 FILE; do
  clear
  cat "$FILE"
  printf '\n\n'
  rm -i "$FILE"
done 3< <(find . -mindepth 1 -maxdepth 1 -type f -name "*.$filter" \
  -printf '%s\t%p\0' | sort -zn | cut -zf 2-)

Note that we must use a different file descriptor than stdin (3 in this example) to pass the file names to the loop. Else, if we use stdin, it will also be used to provide the answers to rm -i.

---

Related Topics