Detect File Change Without Polling

Detect file change in PHP without polling

Check the filemtime(). You can poll it every so often and it's much easier than looking for changes in the file itself.

Detect File Change Without Polling

For linux, there is pyinotify.

From the homepage:

Pyinotify is a Python module for
monitoring filesystems changes.
Pyinotify relies on a Linux Kernel
feature (merged in kernel 2.6.13)
called inotify. inotify is an
event-driven notifier, its
notifications are exported from kernel
space to user space through three
system calls. pyinotify binds these
system calls and provides an
implementation on top of them offering
a generic and abstract way to
manipulate those functionalities.

Thus it is obviously not cross-platform and relies on a new enough kernel version. However, as far as I can see, requiring kernel support would be true about any non-polling mechanism.

Can I detect a file change without an infinite interval or reloading the page?

Below describes two ways on how you can improve your polling:

1) More efficient AJAX Polling

Instead of constantly performing an ajax request every second (whether or not you get the data), you can create a hacky open connection with the server to poll once the request is completed as shown below. The important part of the $.ajax() call is the complete option. The timeout (i.e. 30 seconds) is to make sure the polling continues if the cycle ever breaks.

(function betterPoll(){
    $.ajax({ url: 'http://music.wickedradionet.com/api/live-info/', success: function (data) {
        // do something with "data"
    }, dataType: "json", complete: betterPoll, timeout: 30000 });
})();

2) WebSocket Connection

An even better alternative is WebSocket. This allows for a full-duplex, low-latency connection to stay open between the client and server, enabling "real time" communication.

If you're using Node.js, the docs provide examples on how you can use Socket.IO for WebSocket communication. If you're using PHP, a quick search led me to Ratchet and PHP WebSockets.

Below is an example using Socket.IO:

In your HTML:

<script src="https://cdn.socket.io/socket.io-1.3.5.js"></script>
<script src="example.js"></script>

In "example.js", you can have something like this:

var socket = io.connect('http://music.wickedradionet.com/api/live-info/');
socket.on('an event', function (data) {
  // do something with "data"
  socket.emit('another event', { other: 'data' });
});

On the server, you would do something similar using .on('another event', function () {}) and .emit('an event', function () {}).

FileSystemWatcher vs polling to watch for file changes

I have seen the file system watcher fail in production and test environments. I now consider it a convenience, but I do not consider it reliable. My pattern has been to watch for changes with the files system watcher, but poll occasionally to catch missing file changes.

Edit: If you have a UI, you can also give your user the ability to "refresh" for changes instead of polling. I would combine this with a file system watcher.

Monitoring a directory for new file creation without FileSystemWatcher

Using @Petoj's answer I've included a full windows service that polls every five minutes for new files. Its contrained so only one thread polls, accounts for processing time and supports pause and timely stopping. It also supports easy attaching of a debbugger on system.start

 public partial class Service : ServiceBase{

    List<string> fileList = new List<string>();

    System.Timers.Timer timer;

    public Service()
    {
        timer = new System.Timers.Timer();
        //When autoreset is True there are reentrancy problems.
        timer.AutoReset = false;

        timer.Elapsed += new System.Timers.ElapsedEventHandler(DoStuff);
    }

    private void DoStuff(object sender, System.Timers.ElapsedEventArgs e)
    {
       LastChecked = DateTime.Now;

       string[] files = System.IO.Directory.GetFiles("c:\\", "*", System.IO.SearchOption.AllDirectories);

       foreach (string file in files)
       {
           if (!fileList.Contains(file))
           {
               fileList.Add(file);

               do_some_processing();
           }
       }

       TimeSpan ts = DateTime.Now.Subtract(LastChecked);
       TimeSpan MaxWaitTime = TimeSpan.FromMinutes(5);

       if (MaxWaitTime.Subtract(ts).CompareTo(TimeSpan.Zero) > -1)
           timer.Interval = MaxWaitTime.Subtract(ts).TotalMilliseconds;
       else
           timer.Interval = 1;

       timer.Start();
    }

    protected override void OnPause()
    {
        base.OnPause();
        this.timer.Stop();
    }

    protected override void OnContinue()
    {
        base.OnContinue();
        this.timer.Interval = 1;
        this.timer.Start();
    }

    protected override void OnStop()
    {
        base.OnStop();
        this.timer.Stop();
    }

    protected override void OnStart(string[] args)
    {
       foreach (string arg in args)
       {
           if (arg == "DEBUG_SERVICE")
                   DebugMode();

       }

        #if DEBUG
            DebugMode();
        #endif

        timer.Interval = 1;
        timer.Start();
   }

   private static void DebugMode()
   {
       Debugger.Break();
   }

 }

What does polling mean in the context of watching for file changes?

Polling means here that you regularly read for instance the last write-time of the files you want to watch and test if there is a difference. you could even read the file contents and compare it with a previous verison. Polling just means that you actively do the comparison instead of being notified.

Polling is best avoided for its cost. But if needed then it is needed.

https://en.wikipedia.org/wiki/Polling_(computer_science)

Regarding FileSystemwatcher. It is not perfect but in my experience correct enough in the majority of the cases . I assume that all watchers in developer tools use that mechanism and it is for instance good enough there. I suggest to try it first for your purpose before polling.

File changed listener in Java

Since JDK 1.7, the canonical way to have an application be notified of changes to a file is using the WatchService API. The WatchService is event-driven. The official tutorial provides an example:

/*
 * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 *   - Redistributions of source code must retain the above copyright
 *     notice, this list of conditions and the following disclaimer.
 *
 *   - Redistributions in binary form must reproduce the above copyright
 *     notice, this list of conditions and the following disclaimer in the
 *     documentation and/or other materials provided with the distribution.
 *
 *   - Neither the name of Oracle nor the names of its
 *     contributors may be used to endorse or promote products derived
 *     from this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
 * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
 * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

import java.nio.file.*;
import static java.nio.file.StandardWatchEventKinds.*;
import static java.nio.file.LinkOption.*;
import java.nio.file.attribute.*;
import java.io.*;
import java.util.*;

/**
 * Example to watch a directory (or tree) for changes to files.
 */

public class WatchDir {

    private final WatchService watcher;
    private final Map<WatchKey,Path> keys;
    private final boolean recursive;
    private boolean trace = false;

    @SuppressWarnings("unchecked")
    static <T> WatchEvent<T> cast(WatchEvent<?> event) {
        return (WatchEvent<T>)event;
    }

    /**
     * Register the given directory with the WatchService
     */
    private void register(Path dir) throws IOException {
        WatchKey key = dir.register(watcher, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
        if (trace) {
            Path prev = keys.get(key);
            if (prev == null) {
                System.out.format("register: %s\n", dir);
            } else {
                if (!dir.equals(prev)) {
                    System.out.format("update: %s -> %s\n", prev, dir);
                }
            }
        }
        keys.put(key, dir);
    }

    /**
     * Register the given directory, and all its sub-directories, with the
     * WatchService.
     */
    private void registerAll(final Path start) throws IOException {
        // register directory and sub-directories
        Files.walkFileTree(start, new SimpleFileVisitor<Path>() {
            @Override
            public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
                throws IOException
            {
                register(dir);
                return FileVisitResult.CONTINUE;
            }
        });
    }

    /**
     * Creates a WatchService and registers the given directory
     */
    WatchDir(Path dir, boolean recursive) throws IOException {
        this.watcher = FileSystems.getDefault().newWatchService();
        this.keys = new HashMap<WatchKey,Path>();
        this.recursive = recursive;

        if (recursive) {
            System.out.format("Scanning %s ...\n", dir);
            registerAll(dir);
            System.out.println("Done.");
        } else {
            register(dir);
        }

        // enable trace after initial registration
        this.trace = true;
    }

    /**
     * Process all events for keys queued to the watcher
     */
    void processEvents() {
        for (;;) {

            // wait for key to be signalled
            WatchKey key;
            try {
                key = watcher.take();
            } catch (InterruptedException x) {
                return;
            }

            Path dir = keys.get(key);
            if (dir == null) {
                System.err.println("WatchKey not recognized!!");
                continue;
            }

            for (WatchEvent<?> event: key.pollEvents()) {
                WatchEvent.Kind kind = event.kind();

                // TBD - provide example of how OVERFLOW event is handled
                if (kind == OVERFLOW) {
                    continue;
                }

                // Context for directory entry event is the file name of entry
                WatchEvent<Path> ev = cast(event);
                Path name = ev.context();
                Path child = dir.resolve(name);

                // print out event
                System.out.format("%s: %s\n", event.kind().name(), child);

                // if directory is created, and watching recursively, then
                // register it and its sub-directories
                if (recursive && (kind == ENTRY_CREATE)) {
                    try {
                        if (Files.isDirectory(child, NOFOLLOW_LINKS)) {
                            registerAll(child);
                        }
                    } catch (IOException x) {
                        // ignore to keep sample readbale
                    }
                }
            }

            // reset key and remove from set if directory no longer accessible
            boolean valid = key.reset();
            if (!valid) {
                keys.remove(key);

                // all directories are inaccessible
                if (keys.isEmpty()) {
                    break;
                }
            }
        }
    }

    static void usage() {
        System.err.println("usage: java WatchDir [-r] dir");
        System.exit(-1);
    }

    public static void main(String[] args) throws IOException {
        // parse arguments
        if (args.length == 0 || args.length > 2)
            usage();
        boolean recursive = false;
        int dirArg = 0;
        if (args[0].equals("-r")) {
            if (args.length < 2)
                usage();
            recursive = true;
            dirArg++;
        }

        // register directory and process its events
        Path dir = Paths.get(args[dirArg]);
        new WatchDir(dir, recursive).processEvents();
    }
}

For individual files, various solutions exist, such as:

• https://dzone.com/articles/listening-to-fileevents-with-java-nio

Note that Apache VFS uses a polling algorithm, although it may offer greater functionality. Also note that the API does not offer a way to determine whether a file has been closed.

---

Related Topics