Linux Iterate Over Files in Directory

How to loop over files in directory and change path and add suffix to filename

A couple of notes first: when you use Data/data1.txt as an argument, should it really be /Data/data1.txt (with a leading slash)? Also, should the outer loop scan only for .txt files, or all files in /Data? Here's an answer, assuming /Data/data1.txt and .txt files only:

#!/bin/bash
for filename in /Data/*.txt; do
    for ((i=0; i<=3; i++)); do
        ./MyProgram.exe "$filename" "Logs/$(basename "$filename" .txt)_Log$i.txt"
    done
done

Notes:

/Data/*.txt expands to the paths of the text files in /Data (including the /Data/ part)
$( ... ) runs a shell command and inserts its output at that point in the command line
basename somepath .txt outputs the base part of somepath, with .txt removed from the end (e.g. /Data/file.txt -> file)

If you needed to run MyProgram with Data/file.txt instead of /Data/file.txt, use "${filename#/}" to remove the leading slash. On the other hand, if it's really Data not /Data you want to scan, just use for filename in Data/*.txt.

Loop through files in directory specified using argument

This would happen if the directory is empty, or misspelled. The shell (in its default configuration) simply doesn't expand a wildcard if it has no matches. (You can control this in Bash with shopt -s nullglob; with this option, wildcards which don't match anything are simply removed.)

You can verify this easily for yourself. In a directory with four files,

sh$ echo *
a       file    or      two

sh$ echo [ot]*
or      two

sh$ echo n*
n*

And in Bash,

bash$ echo n*
n*

bash$ shopt -s nullglob

bash$ echo n*

I'm guessing you are confused about how the current working directory affects the resolution of directory names; maybe read Difference between ./ and ~/

Loop through files in a given directory

If you only want the files non-recursively in the current directory, combine what you have:

read -p 'Enter the directory path: ' directory
for file in "$directory"/*; do
  echo "$file"
done

If you want to loop recursively and you have bash 4, it's not much harder:

shopt -s globstar
for file in "$directory"/**/*; do …

But if you only have bash 3, you'd be better off using find.

find "$directory"

How to iterate files in directory with for loop in Bash

The for-loop is basically sound. However, if the directory is empty, the loop will be executed once, with the variable file containing the literal text /var/spool/bandit24/*.

The stat message is not from the for-loop, but from one of the commands in the loop.

The correct way would be to test if the directory is empty before you continue. You could put something like

if [ $(find . -type f | wc -l) -eq 0 ] ; then
    echo "Nothing to do"
    exit 0
fi

right after the cd.

Some other comments on your script.

If you do a cd in the script, you don't need to specify the full path anymore.
Your quoting is not really consistent. That may not be a problem if your file names never contain spaces or strange characters, but I would, for example timeout -s 9 60 "./$file" and rm -f "./file"
/var/spool/bandit/* will never contain . or .., so that test is useless.
You could also replace the test with if [ -f "$file" ] ; then

Loop over files in a directory not working

I assume that you expect to loop through all files within the directory you pass. To do that, you need to change your loop:

for file in "$1"/*

It's worth mentioning that for doesn't have any built-in behaviour to enumerate items in a directory, it simply iterates over the list of words that you pass it. The *, expanded by the shell, is what results in the loop iterating over a list of files.

Your condition would need modifying too, as the * needs to be outside of quotes (and the rest doesn't need to be inside them, either):

if [[ $f = *.txt ]]

But you can avoid the need for the conditional by directly looping through all files ending in .txt:

for file in "$1"/*.txt

You may also want to consider the case where there are no matches, in which case I guess you expect the loop not to run. One way to do that in bash would be:

# failing glob expands to nothing, rather than itself
shopt -s nullglob 

for file in "$1"/*.txt
    # ...
done

# unset this behaviour if you don't want it in the rest of the script
shopt -u nullglob

Expect: How to iterate over files in a directory?

You would write:

#!/usr/bin/env expect
foreach f [glob /path/*.csv] {
    puts $f
}

Expect is an extension of Tcl. In addition to the expect man page, a link to the Tcl command documentation will be helpful: https://www.tcl.tk/man/tcl8.6/TclCmd/contents.htm

Iterate Over Files in Variable Path (Bash)

Glob expansion doesn't happen inside quotes (both single and double) in shell.

You should be using this code:

for file in "$path"/*; do
     echo "INFO - Checking $file"
     [[ -e $file ]] || continue
done