For Loop for Files in Multiple Folders - Bash Shell

For loop for files in multiple folders - bash shell

Try for f in ./{test1,test2,test3}/* or for f in ./*/* depending on what you want.

bash - for loop through multiple directories and their files

Would you please try the following:

#!/bin/bash

for i in my_path/*/; do
    year=${i%/}; year=${year##*/}       # extract year
    year2=$(( year + 19 ))              # add 19
    for j in "$i"*.nc; do
        echo cdo "selyear,${year}/${year2}" "$j" "$j"2
    done
done

It outputs command lines as a dry run. If it looks good, drop echo and run.

Bash loop through multiple directories

Just find the actual files.

find "${arr[@]}" -maxdepth 1 -type f >> test

You could depend on shell filename expansion:

for dir in "${arr[@]}" # properly handle spaces in array values
do
      for file in "$dir"/*; do
          # check for empty dir
          if [ -f "$file" ]; then
              # Use printf, in case file is named ex. `-e`
              printf "%s\n" "$file"
          fi
      done
# don't reopen the file on each loop, just open it once
done >> test

but that's a lot more, just find it.

Interacting with files from multiple directories via bash script

You already accepted the answer of @Charles Duffy but (if I understood well) your question is about having files in different directories so if you need to work with multiple csv files on multiple directories you can use the following snippet:

# array containing the different directories to work with
pathDir=("/foo/bar/dir1" "/buzz/fizz/dir2")

for dir in "${pathDir[@]}" # For each directory
do
    for file in "$dir/"*.csv; do # For each csv file of the directory

        if [[ $(file -i "$file") == "$file: $charset" ]]; then
            iconv -f ISO-8859-1 -t UTF-8 "$file" > "$file.new";
            mv -f "$file.new" "$file";
        fi

    done
done

The pathDir variable is an array which contains the path of different directories.

The first for loop iterate through this array to get all the paths to check.

The second for loop as in the previous answer iterate through the files of the current tested directory.

How to use a for loop to create folders in bash

The biggest problem here is that you're doing things inside the loop that really only should be done once. Specifically, the error you're getting is because it tries to create the directory every time through the loop, but you can only create it once. Also, if the user enters too large a number, it'll print multiple error messages (e.g. if num2 is entered as 500, it'll print 500 error messages). You need to do both the error check and creating the directory once, before the loop.

A second problem is that you don't add $counter to the filename, so if the user enters "2 9", it'll create a file named 29.txt nine times.

You also have some more minor issues: in general, error messages should be printed to standard error instead of standard output (you can redirect them with >&2), and if there's an error the script should exit with a nonzero status. Also, you should (almost always) put double-quotes around variable references, to avoid weird results if the variables are blank or contain whitespace or some other things. You also don't need to touch files before writing into them (using > somefile will create the file if it doesn't exist).

With these things fixed (and some stylistic tweaks), here's what I get:

#!/bin/bash

echo "enter the nums: "
read num1 num2

if ! [ "$num1" -lt 10 ] || ! [ "$num2" -lt 10 ]; then
    echo "you weren't supposed to do that" >&2    # message send to stderr
    exit 1    # exit with error status
fi

mkdir "$num1" || exit $?    # if mkdir fails, exit with its error status

for (( counter=0; counter<$num2; counter++ )); do
    echo "${num1}${num2}" > "${num1}/${num1}${num2}${counter}.txt"
done

BTW, the ! [ "$num1" -lt 10 ] tests may look a little weird; why not just use [ "$num" -ge 10 ]? I did it that way in case $num1 and/or $num2 isn't a valid number, in which case both -lt and -ge tests would fail; using a negated test makes that an error rather than a success.

Nested for loop to enter and exit multiple directories Bash script

It looks like /completepath/ contains some entries that are not directories. When the loop over /completepath/* sees something that's not a directory, it doesn't enter it, thanks to the [ -d $f ] check.
But it still continues to run the next for y in ... loop.
At that point the script is still in the previous directory it has seen.

One way to solve that is to skip the rest of the loop when $f is not a directory:

if [ -d $f ]; then
    cd "$f" && echo Entering into $f
else
    continue
fi

There's an even better way. By writing /completepath/*/ only directory entries will be matched, so you can simplify your loop to this:

for f in /completepath/*/
do
    cd "$f" && echo "Entering into $f" || { echo "Error: could not enter into $f"; continue; }
    for y in $(ls *.fastq.gz | sed 's/_L00[1234]_R1_001.fastq.gz//g' | sort -u)
    do
        echo ${y}
    done
done

How to loop over files in directory and change path and add suffix to filename

A couple of notes first: when you use Data/data1.txt as an argument, should it really be /Data/data1.txt (with a leading slash)? Also, should the outer loop scan only for .txt files, or all files in /Data? Here's an answer, assuming /Data/data1.txt and .txt files only:

#!/bin/bash
for filename in /Data/*.txt; do
    for ((i=0; i<=3; i++)); do
        ./MyProgram.exe "$filename" "Logs/$(basename "$filename" .txt)_Log$i.txt"
    done
done

Notes:

• /Data/*.txt expands to the paths of the text files in /Data (including the /Data/ part)
• $( ... ) runs a shell command and inserts its output at that point in the command line
• basename somepath .txt outputs the base part of somepath, with .txt removed from the end (e.g. /Data/file.txt -> file)

If you needed to run MyProgram with Data/file.txt instead of /Data/file.txt, use "${filename#/}" to remove the leading slash. On the other hand, if it's really Data not /Data you want to scan, just use for filename in Data/*.txt.

---

Related Topics