How to Create a File Listener in Linux

How do I create a file listener in linux?

To get notified about events like file creation, opening, modifying etc. look into inotify. A good way to use it from bash is with the inotifywait command - here is its man page. It will block until an event you care about happens. For example:

inotifywait -e create /path/to/watch
echo "ding!"

will ding when a file or directory gets created in that path. See the man page for more details.

Linux add listener to the log file

Inotify is fine for getting notified about file events such as writing etc but how will you know how much data has been appended? If you know the log files in advance you might as well read until end of file and just sleep for a short while and try again to read (something like "tail -f" does). That way you still have the pointer to where you start to read the newly written data. You could even combine that with Inotify to know when to pick up reading. If you want to just use Inotify, you probably will have to store the pointers to the last read position somewhere.

Detect when new file is added to a directory

In case you are working on Unix like OS you should consider using inotifywait tool.
I just found nice Erlang wrapper around useful system tools: https://github.com/sdanzan/erlang-systools

How to create a copy of a file after observing the event using File Listener (watchdog) in Python?

Not sure why this is a problem. But if I understand correctly, you only need to

import os, shutil

and in

def on_created(self, event):
    self.process(event)
    destination = os.path.join(os.getcwd(), 'today', event.src_path.split('\\')[-1])
    shutil.copyfile(event.src_path, destination)

if you want "today" to be a subfolder of your curernt script's working directory. If not set the path to something else

UPDATE

Problem 1: Pausing the watchdog, wasn't so hard. It queues consequtive triggers (Events) and executes them

Problem 2: The trigger is executed, when the file is created. That doesn't mean the OS is finished writing it, hence the I/O Exception when trying to access / copy it. You have to wait for the file to be completely written. The EventHandler has no means of telling, so you have to implement a workaround. I have added the code below, that should do the trick.

from watchdog.observers import Observer  
from watchdog.events import PatternMatchingEventHandler
import time, os, shutil

class FileWatcher(PatternMatchingEventHandler):
    patterns = ["*.xlsx"] 

   def process(self, event):
        
        # event.src_path will be the full file path
        # event.event_type will be 'created', 'moved', etc.
        print('{} observed on {}'.format(event.event_type, event.src_path))

    def on_created(self, event):
        self._is_paused = True
        
        # WAITING FOR FILE TRANSFER
        file = None
        while file is None:
            try:
                file = open(event.src_path)
            except OSError:
                file = None                    
                time.sleep(1) # WAITING FOR FILE TRANSFER
                continue
        
        self.process(event)
        destination = os.path.join(os.getcwd(), 'today', event.src_path.split('\\')[-1])
        shutil.copyfile(event.src_path, destination)        
        self._is_paused = False

if __name__ == '__main__':
    
    obs = Observer() 
    obs.schedule(FileWatcher(), path='.\\')
    print("Monitoring started....")
    obs.start() # Start watching

    try:
        while obs.isAlive():
            obs.join()
    finally:
        obs.stop()
        obs.join()

file listener process on tomcat

since you refined the question, here comes another answer: how to start a daemon in tomcat:

first, register your Daemons in web.xml:

< listener >
my.package.servlet.Daemons
< /listener >

then implement the Daemons class as an implementation of ServletContextListener like this:

the code will be called every 5 seconds, tomcat will call contextDestroyed when your app shuts down. note that the variable is volatile, otherwise you may have troubles on shutdown on multi-core systems

import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;

public class Daemons implements ServletContextListener {
private volatile boolean active = true;

Runnable myDeamon = new Runnable() {

    public void run() {
        while (active) {
            try {
                System.out.println("checking changed files...");
                Thread.sleep(5000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }
};

public void contextInitialized(ServletContextEvent servletContextEvent) {
    new Thread(myDeamon).start();
}

public void contextDestroyed(ServletContextEvent servletContextEvent) {
    active = false;
}
}

How to monitor a complete directory tree for changes in Linux?

To my knowledge, there's no other way than recursively setting an inotify watch on each directory.

That said, you won't run out of file descriptors because inotify does not have to reserve an fd to watch a file or a directory (its predecessor, dnotify, did suffer from this limitation). inotify uses "watch descriptors" instead.

According to the documentation for inotifywatch, the default limit is 8192 watch descriptors, and you can increase it by writing the new value to /proc/sys/fs/inotify/max_user_watches.

---

Related Topics