Linux Shell Script - Find All Files and Run a Command on Each One of Them

Execute command on all files in a directory

The following bash code will pass $file to command where $file will represent every file in /dir

for file in /dir/*
do
  cmd [option] "$file" >> results.out
done

Example

el@defiant ~/foo $ touch foo.txt bar.txt baz.txt
el@defiant ~/foo $ for i in *.txt; do echo "hello $i"; done
hello bar.txt
hello baz.txt
hello foo.txt

Linux shell script - find all files and run a command on each one of them

You could process the results from find line by line and transform <file>.xml into <file>.csv:

find /results/ -type f -name "*Metadata*.xml" | while read file; do java -jar $SAXON/saxon9h3.jar -o:${file%.xml}.csv $file $WORKDIR/transform.XMI.xsl; done

This simple approach fails in case the file names have spaces in their paths/names.

Find files and execute command

You can use the find command:

find -name '*.rar' -exec unrar x {} \;

find offers the option exec which will execute that command on every file that was found.

Recursively read folders and executes command on each of them

If you want to recurse into directories, executing a command on each file found in those, I would use the find command, instead of writing anything using shell-script, I think.

That command can receive lots of parameters, like type to filter the types of files returned, or exec to execute a command on each result.

For instance, to find directories that are under the one I'm currently in :

find . -type d -exec echo "Hello, '{}'" \;

Which will get me somehthing like :

Hello, '.'
Hello, './.libs'
Hello, './include'
Hello, './autom4te.cache'
Hello, './build'
Hello, './modules'

Same to find the files under the current directory :

find . -type f -exec echo "Hello, '{}'" \;

which will get me something like this :

Hello, './config.guess'
Hello, './config.sub'
Hello, './.libs/memcache_session.o'
Hello, './.libs/memcache_standard_hash.o'
Hello, './.libs/memcache_consistent_hash.o'
Hello, './.libs/memcache.so'
Hello, './.libs/memcache.lai'
Hello, './.libs/memcache.o'
Hello, './.libs/memcache_queue.o'
Hello, './install-sh'
Hello, './config.h.in'
Hello, './php_memcache.h'
...

Some would say "it's not shell"... But why re-invent the wheel ?

(And, in a way, it is shell ^^ )

For more informations, you can take a look at :

• man find
• lots of tutorials found with google, like, for instance, Unix Find Command Tutorial

How to go to each directory and execute a command?

You can do the following, when your current directory is parent_directory:

for d in [0-9][0-9][0-9]
do
    ( cd "$d" && your-command-here )
done

The ( and ) create a subshell, so the current directory isn't changed in the main script.

Find and execute command on all files in directory that have a specific folder name

-exec runs the command in the directory from which you start find.

-execdir runs the command in the matching file's directory.

To only find *.txt* files whose parents contain a specific file, you could use:

find "$PWD" -path "*keyword*/*.txt*" -execdir command {} new_file \;

This will run the command for foo/bar/some-keyword-dir/baz/etc/file.txts but not for foo/bar/baz/file.txts (no keyword in parent directory names) or foo/bar/some-keyword-dir/baz/file.tar (not *.txt*)

Run a single command for every files in a folder- LINUX

Maybe this solution would be easier:

find <path> -maxdepth 1 -type f -exec sed -i "s/\$/\t\$f/" '{}' \;

If you omit or increase -maxdepth the command will search in subfolders too.

Read list of files on unix and run command

There are a couple of ways to do it. Since the names are 'one per line' in the data file, we can assume there are no newlines in the file names.

for loop

for file in $(<list.txt)
do
    sickle pe -f "${file}_file1.fastq" -r "${file}_file2.fastq" -t sanger
done

while loop with read

while read file
do
    sickle pe -f "${file}_file1.fastq" -r "${file}_file2.fastq" -t sanger
done < list.txt

The for loop only works if there are no blanks in the names (nor other white-space characters such as tabs). The while loop is clean as long as you don't have newlines in the names, though using while read -r file would give you even better protection against the unexpected. The double quotes around the file name in the for loop are decorative (but harmless) because the file names cannot contain blanks, but those in the while loop prevent file names containing blanks from being split when they should not be split. It's often a good idea to quote variables every time you use them, though it strictly only matters when the variable might contain blanks but you don't want the value split up.

I've had to guess what names should be passed to the sickle command since your question is not clear about it — I'm 99% sure I've guessed wrong, but it matches the different suffixes in your sample command assuming the base name of file is input. I've omitted the trailing backslash; it is the 'escape' character and it is not clear what you really want there.

How to loop over files in directory and change path and add suffix to filename

A couple of notes first: when you use Data/data1.txt as an argument, should it really be /Data/data1.txt (with a leading slash)? Also, should the outer loop scan only for .txt files, or all files in /Data? Here's an answer, assuming /Data/data1.txt and .txt files only:

#!/bin/bash
for filename in /Data/*.txt; do
    for ((i=0; i<=3; i++)); do
        ./MyProgram.exe "$filename" "Logs/$(basename "$filename" .txt)_Log$i.txt"
    done
done

Notes:

• /Data/*.txt expands to the paths of the text files in /Data (including the /Data/ part)
• $( ... ) runs a shell command and inserts its output at that point in the command line
• basename somepath .txt outputs the base part of somepath, with .txt removed from the end (e.g. /Data/file.txt -> file)

If you needed to run MyProgram with Data/file.txt instead of /Data/file.txt, use "${filename#/}" to remove the leading slash. On the other hand, if it's really Data not /Data you want to scan, just use for filename in Data/*.txt.

---

Related Topics