Using Filesystemwatcher to Monitor a Directory

Using FileSystemWatcher to monitor a directory

The problem was the notify filters. The program was trying to open a file that was still copying. I removed all of the notify filters except for LastWrite.

private void watch()
{
  FileSystemWatcher watcher = new FileSystemWatcher();
  watcher.Path = path;
  watcher.NotifyFilter = NotifyFilters.LastWrite;
  watcher.Filter = "*.*";
  watcher.Changed += new FileSystemEventHandler(OnChanged);
  watcher.EnableRaisingEvents = true;
}

Monitoring a directory for new file creation without FileSystemWatcher

Using @Petoj's answer I've included a full windows service that polls every five minutes for new files. Its contrained so only one thread polls, accounts for processing time and supports pause and timely stopping. It also supports easy attaching of a debbugger on system.start

 public partial class Service : ServiceBase{


    List<string> fileList = new List<string>();

    System.Timers.Timer timer;


    public Service()
    {
        timer = new System.Timers.Timer();
        //When autoreset is True there are reentrancy problems.
        timer.AutoReset = false;

        timer.Elapsed += new System.Timers.ElapsedEventHandler(DoStuff);
    }


    private void DoStuff(object sender, System.Timers.ElapsedEventArgs e)
    {
       LastChecked = DateTime.Now;

       string[] files = System.IO.Directory.GetFiles("c:\\", "*", System.IO.SearchOption.AllDirectories);

       foreach (string file in files)
       {
           if (!fileList.Contains(file))
           {
               fileList.Add(file);

               do_some_processing();
           }
       }


       TimeSpan ts = DateTime.Now.Subtract(LastChecked);
       TimeSpan MaxWaitTime = TimeSpan.FromMinutes(5);

       if (MaxWaitTime.Subtract(ts).CompareTo(TimeSpan.Zero) > -1)
           timer.Interval = MaxWaitTime.Subtract(ts).TotalMilliseconds;
       else
           timer.Interval = 1;

       timer.Start();
    }

    protected override void OnPause()
    {
        base.OnPause();
        this.timer.Stop();
    }

    protected override void OnContinue()
    {
        base.OnContinue();
        this.timer.Interval = 1;
        this.timer.Start();
    }

    protected override void OnStop()
    {
        base.OnStop();
        this.timer.Stop();
    }

    protected override void OnStart(string[] args)
    {
       foreach (string arg in args)
       {
           if (arg == "DEBUG_SERVICE")
                   DebugMode();

       }

        #if DEBUG
            DebugMode();
        #endif

        timer.Interval = 1;
        timer.Start();
   }

   private static void DebugMode()
   {
       Debugger.Break();
   }

 }

FileSystemWatcher monitoring 2 directories. Change log increased by 1 per change, so when one file changes in console it pops up many times?

In the MonitorDirectory-method you're just adding more delegates to the FileSystemWatcher_Changed-method on fileSystemWatcher.Changed:

fileSystemWatcher.Changed += FileSystemWatcher_Changed;

That's why you get a linear increase of notifications everytime the current job document is changed.

FileSystemWatcher to monitor directory size

You're using the folder path provided by the FileSystemEventArgs so that will be the folder that has changed. Instead, create an object for each directory root that you are monitoring and, inside it, store the root path and use that instead of the EventArgs.

You may find an easy way to create this object is just to use a lambda function for the event handler as follows. This effectively wraps up the path from the outer scope into a different object for each path you are monitoring.

 FileSystemWatcher watcher = new FileSystemWatcher();
 watcher.Path = dirPaths[i].ToString();
 watcher.NotifyFilter = NotifyFilters.Size;
 watcher.EnableRaisingEvents = true;
 watcher.Changed += delegate (object source, FileSystemEventArgs e)
 {
    float dirSize = CalculateFolderSize(watcher.Path); // using the path from the outer scope

    float limitSize = int.Parse(_config.TargetSize);//getting the limit size 

    if (dirSize > limitSize)
    {
        eventLogCheck.WriteEntry("the folloing path has crossed the limit " + directory);
        //TODO: mail sending
    }
 };

How to watch all the folders of a specific name under a parent directory with FileSystemWatcher?

FileSystemWatcher has a property called IncludeSubdirectories, When true, IncludeSubdirectories is recursive through the entire sub tree, not just the immediate child directories.

So an example would be something like this (modified MSDN example):

private static void Watch(string watch_folder)
{
    FileSystemWatcher watcher = new FileSystemWatcher();
    watcher.IncludeSubdirectories = true;
    watcher.Path = watch_folder;
    /* Watch for changes in LastAccess and LastWrite times, and
       the renaming of files or directories. */
    watcher.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite
       | NotifyFilters.FileName | NotifyFilters.DirectoryName;

    // Add event handlers.
    watcher.Changed += new FileSystemEventHandler(OnChanged);
    watcher.Created += new FileSystemEventHandler(OnChanged);
    watcher.Deleted += new FileSystemEventHandler(OnChanged);
    watcher.Renamed += new RenamedEventHandler(OnRenamed);

    // Begin watching.
    watcher.EnableRaisingEvents = true;
}

// Define the event handlers.
private static void OnChanged(object source, FileSystemEventArgs e)
{
    // Specify what is done when a file is changed, created, or deleted.
}

private static void OnRenamed(object source, RenamedEventArgs e)
{
    // Specify what is done when a file is renamed.
}

Update

This is a small console example on how to filter notifications to only react to the Processed folders based on @MickyD comment:

class Program
{
    static void Main(string[] args)
    {

        try
        {
            string path = Path.Combine(Directory.GetCurrentDirectory(), "Order");
            Console.WriteLine(path);
            FileSystemWatcher fileSystemWatcher = new FileSystemWatcher(path);
            fileSystemWatcher.IncludeSubdirectories = true;
            fileSystemWatcher.Changed += FileSystemWatcher_Changed;
            fileSystemWatcher.EnableRaisingEvents = true;

            Process.GetCurrentProcess().WaitForExit();
            fileSystemWatcher.Dispose();
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.Message);
        }
    }

    private static void FileSystemWatcher_Changed(object sender, FileSystemEventArgs e)
    {
        if (e.Name.Contains("Processed"))
            Console.WriteLine("Processed folder has changed");
    }
}

System.IO.FileSystemWatcher to monitor a network-server folder - Performance considerations

From a server load point of view, using the IO.FileSystemWatcher for remote change notifications in the scenario you describe is probably the most efficient method possible. It uses the FindFirstChangeNotification and ReadDirectoryChangesW Win32 API functions internally, which in turn communicate with the network redirector in an optimized way (assuming standard Windows networking: if a third-party redirector is used, and it doesn't support the required functionality, things won't work at all). The .NET wrapper also uses async I/O and everything, further assuring maximum efficiency.

The only problem with this solution is that it's not very reliable. Other than having to deal with network connections going away temporarily (which isn't too much of a problem, since IO.FileSystemWatcher will fire an error event in this case which you can handle), the underlying mechanism has certain fundamental limitations. From the MSDN documentation for the Win32 API functions:

ReadDirectoryChangesW fails with ERROR_INVALID_PARAMETER when the buffer length is greater than 64 KB and the application is monitoring a directory over the network. This is due to a packet size limitation with the underlying file sharing protocols
Notifications may not be returned when calling FindFirstChangeNotification for a remote file system

In other words: under high load (when you would need a large buffer) or, worse, under random unspecified circumstances, you may not get the notifications you expect. This is even an issue with local file system watchers, but it's much more of a problem over the network. Another question here on SO details the inherent reliability problems with the API in a bit more detail.

When using file system watchers, your application should be able to deal with these limitations. For example:

If the files you're looking for have sequence numbers, store the last sequence number you got notified about, so you can look for 'gaps' on future notifications and process the files for which you didn't get notified;
On receiving a notification, always do a full directory scan. This may sound really bad, but since the scan is event-driven, it's still much more efficient than dumb polling. Also, as long as you keep the total number of files in a single directory, as well as the number of directories to scan, under a thousand or so, the impact of this operation on performance should be pretty minimal anyway.

Setting up multiple listeners is something you should avoid as much as possible: if anything, this will make things even less reliable...

Anyway, if you absolutely have to use file system watchers, things can work OK as long as you're aware of the limitations, and don't expect 1:1 notification for every file modified/created.

So, if you have other options (essentially, having the process writing the files notify you in a non-file system based way: any regular RPC method will be an improvement...), those are definitely worth looking into from a reliability point of view.

Using FileSystemWatcher to monitor file creation and copy it before deleted

Options 1:
Instead of looking at FileSystemWatcher, you should look to hook your code to delete event.
You can look this this comment on Stack Overflow: https://stackoverflow.com/a/4395147/442470

Option 2:
As soon as your FileSystemWatcher realizes that a file is created, change permission of the file so that it cannot be deleted.

Using Filesystemwatcher to Monitor a Directory