Linux Command to Check New Files in File System

Linux command to check new files in file system

There are bunch of ways for doing that.

First one:

start_date=201105040000

end_date=201105042359

touch -t ${start_date} start

touch -t ${end_date} end

find /you/path -type f -name '*you*pattern*' -newer start ! -newer end -exec ls -s {} \;

Second one:
find files modified between 20 and 21 days ago:

find -ctime +20 -ctime -21

finds files modified between 2500 and 2800 minutes ago:

find -cmin +2500 -cmin -2800

And read this topic too.

Get most recent file in a directory on Linux

ls -Art | tail -n 1

This will return the latest modified file or directory. Not very elegant, but it works.

Used flags:

-A list all files except . and ..

-r reverse order while sorting

-t sort by time, newest first

Find the files that have been changed in last 24 hours

To find all files modified in the last 24 hours (last full day) in a particular specific directory and its sub-directories:

find /directory_path -mtime -1 -ls

Should be to your liking

The - before 1 is important - it means anything changed one day or less ago.
A + before 1 would instead mean anything changed at least one day ago, while having nothing before the 1 would have meant it was changed exacted one day ago, no more, no less.

Bash function to find newest file matching pattern

The ls command has a parameter -t to sort by time. You can then grab the first (newest) with head -1.

ls -t b2* | head -1

But beware: Why you shouldn't parse the output of ls

My personal opinion: parsing ls is dangerous when the filenames can contain funny characters like spaces or newlines.

If you can guarantee that the filenames will not contain funny characters (maybe because you are in control of how the files are generated) then parsing ls is quite safe.

If you are developing a script which is meant to be run by many people on many systems in many different situations then do not parse ls.

Here is how to do it safe: How can I find the latest (newest, earliest, oldest) file in a directory?

unset -v latest
for file in "$dir"/*; do
[[ $file -nt $latest ]] && latest=$file
done

How to get new files coming into a folder while processing files for the same folder in shell script

What about something like this:

#!/bin/sh -xe

# create some dummy files to start with
touch filea
touch fileb

function analyzeFile() {
echo "analyzing $1"
sleep 10 # dummy for the real stuff you need to do
}

declare stillGettingSomething
declare -A alreadyAnalyzed

stillGettingSomething=true
while [ $stillGettingSomething ]; do
stillGettingSomething=false # prevent endless looping

for i in ./file*; do
# idea: see also http://superuser.com/questions/195598/test-if-element-is-in-array-in-bash

if [[ ${alreadyAnalyzed[$i]} ]]; then
echo "$i was already analyzed before; skipping it immediately"
continue
fi

alreadyAnalyzed[$i]=true # Memorize the file which we visited
stillGettingSomething=true # We found some new file; we have to run another scan iteration later on

analyzeFile $i

# create some new files for the purpose of demonstration
echo "creating file $i-latecreate"
touch $i-latecreate
done

done

The result of this script is

+ declare stillGettingSomething
+ declare -A alreadyAnalyzed
+ stillGettingSomething=true
+ '[' true ']'
+ stillGettingSomething=false
+ for i in './file*'
+ [[ -n '' ]]
+ alreadyAnalyzed[$i]=true
+ stillGettingSomething=true
+ analyzeFile ./filea
+ echo 'analyzing ./filea'
analyzing ./filea
+ sleep 10
+ echo 'creating file ./filea-latecreate'
creating file ./filea-latecreate
+ touch ./filea-latecreate
+ for i in './file*'
+ [[ -n '' ]]
+ alreadyAnalyzed[$i]=true
+ stillGettingSomething=true
+ analyzeFile ./fileb
+ echo 'analyzing ./fileb'
analyzing ./fileb
+ sleep 10
+ echo 'creating file ./fileb-latecreate'
creating file ./fileb-latecreate
+ touch ./fileb-latecreate
+ '[' true ']'
+ stillGettingSomething=false
+ for i in './file*'
+ [[ -n true ]]
+ echo './filea was already analyzed before; skipping it immediately'
./filea was already analyzed before; skipping it immediately
+ continue
+ for i in './file*'
+ [[ -n '' ]]
+ alreadyAnalyzed[$i]=true
+ stillGettingSomething=true
+ analyzeFile ./filea-latecreate
+ echo 'analyzing ./filea-latecreate'
analyzing ./filea-latecreate
+ sleep 10

The idea behind it is to use an associative array, which memorizes those files already processed. If a file was already processed, it is skipped the next time we step over it. We do this as long as we are getting at least one new file in a scan iteration.

EDIT: Cleaned-up coding

Here's a cleaned-up variant of the coding above, pruning the demo-purpose coding, trying to come as close as possible to the original requirement.

#!/bin/sh

function analyzeFile() {
echo "analyzing $1"
sleep 10 # dummy for the real stuff you need to do
}

declare stillGettingSomething
declare -A alreadyAnalyzed

stillGettingSomething=true
while [ $stillGettingSomething ]; do
stillGettingSomething=false # prevent endless looping

for i in "$mydir"/*; do

if [[ ${alreadyAnalyzed[$i]} ]]; then
echo "$i was already analyzed before; skipping it immediately"
continue
fi

alreadyAnalyzed[$i]=true # Memorize the file which we visited
stillGettingSomething=true # We found some new file; we have to run another scan iteration later on

analyzeFile $i
done
done


Related Topics



Leave a reply



Submit