Monitoring Contents of Files/Directories

Monitoring a folder for files

Based on your script:

import time
import os
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class EventHandler(FileSystemEventHandler):

    @staticmethod
    def on_created(event):
        action, path = (event.event_type, event.src_path)
        if not "." in path.split(os.sep)[-1]:
            return
        path = ".".join(path.split("."))
        extension = path.split(".")[-1]
        dirname = os.path.dirname(path)

        files = os.listdir(dirname)
        same_ext_files = [os.path.join(dirname, f) for f in files if f.endswith("." + extension)]
        creation_date = [os.path.getmtime(f) for f in same_ext_files]

        print(f"{action = }")
        print(f"{path = }")
        print(f"{extension = }")
        print(f"{dirname = }")
        print(f"{files = }")
        print(f"{same_ext_files = }")
        print(f"{creation_date = }")

if __name__ == "__main__":

    path = os.path.join("/root", "test/")
    if not os.path.isdir(path):
        raise FileNotFoundError
    observer = Observer()
    observer.schedule(EventHandler(), path, recursive=True)
    observer.start()
    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

Creating file Fichier texte (2).txt:

python test.py 
action = 'created'
path = '/home/vince/Bureau/stackoverflow/Fichier texte (2).txt'
extension = 'txt'
dirname = '/home/vince/Bureau/stackoverflow'
files = ['Fichier texte (2).txt', 'Fichier texte (1).txt', 'Fichier texte.txt', 'test.py']
same_ext_files = ['/home/vince/Bureau/stackoverflow/Fichier texte (2).txt', '/home/vince/Bureau/stackoverflow/Fichier texte (1).txt', '/home/vince/Bureau/stackoverflow/Fichier texte.txt']
creation_date = [1638378453.7645514, 1638378430.8448474, 1638378409.4784563]

You have now two lists: same_ext_files and creation_date to pipe to anything you want.

code fixed to working

Monitoring contents of files/directories?

For Unix/Linux based systems, you should use File Alteration Monitor Python bindings to libfam.

For Windows based systems, you should tie into the Win32 API FindFirstChangeNotification and related functions.

As for a cross platform way, I don't know about a good cross platform way. I think it would be best to build a module yourself that works on either OS that uses one of the 2 above methods after detecting what OS it is.

What's a good way to monitor the number of files/folders in a directory in a reactive context?

You can use reactivePoll() to create a reactive file count that is checked regularly (here 1 second):

library(shiny)
library(tidyverse)

ui <- fluidRow(verbatimTextOutput("result"))

server <- function(input, output, session) {
  file_count <- function() {
    list.dirs(".", full.names = FALSE, recursive = FALSE) %>% 
      length()
    }
  current_file_count <- reactivePoll(1000, session, file_count, file_count)
  output$result <- renderText(current_file_count())
}

shinyApp(ui = ui, server = server)

Powershell - Monitoring folder Content

Insted of the while($true) loop, have you tried the "Register-ObjectEvent"?

I just tested one of my script using this method and could easily take 2000 empty files (generated in powershell). Unfortunatly, this was on a local machine.

Instructions: Define function like normal and off you go.
The command you use is:

Start-BackupScript -WatchFolder "C:\temp\my watch folder\" -DestinationFolder "C:\temp\backup\"

The script now monitors "C:\temp\my watch folder\" for new files created in that specific folder and will move them to "C:\temp\backup\". It will also append the date and time to the file.

Lets say you have started the script with the parameters above. You now place "hello_world.txt" in the watch folder. The script will move the file to "C:\temp\backup\" with the new filename being: "hello_world_2016-02-10_10-00-00.txt"

The script runs in the background. If you want to know how it's doing, then use the command:

Get-Job $backupscript -Keep

There you can see what it has been doing when. Please note that the -Keep parameter keeps the output in the "log", so you can check it later.

Script:

function Start-BackupScript
{
  [CmdletBinding()]
  Param
  (
    [Parameter()]
    [String]$WatchFolder,
    [Parameter()]
    [String]$DestinationFolder
    )
  Process 
  {
    $filter = '*.*'                           
    $fsw = New-Object IO.FileSystemWatcher $WatchFolder, $filter -Property @{IncludeSubdirectories = $false;NotifyFilter = [IO.NotifyFilters]'FileName,     LastWrite'} 
    $action = {
      $fileMissing = $false 
      $FileInUseMessage = $false 
      $copied = $false 
      $file = Get-Item $Args.FullPath 
      $dateString = Get-Date -format "_yyyy-MM-dd_HH-mm-ss" 
      $DestinationFolder = $event.MessageData 
      $DestinationFileName = $file.basename + $dateString + $file.extension 
      $resultfilename = Join-Path $DestinationFolder $DestinationFileName 
      Write-Output ""
      while(!$copied) { 
        try { 
          Move-Item -Path $file.FullName -Destination $resultfilename -ErrorAction Stop
          $copied = $true 
        }  
        catch [System.IO.IOException] { 
          if(!$FileInUseMessage) { 
            Write-Output "$(Get-Date -Format "yyyy-MM-dd @ HH:mm:ss") - $file in use. Waiting to move file"
            $FileInUseMessage = $true 
          } 
          Start-Sleep -s 1 
        }  
        catch [System.Management.Automation.ItemNotFoundException] { 
          $fileMissing = $true 
          $copied = $true 
        } 
      } 
      if($fileMissing) { 
        Write-Output "$(Get-Date -Format "yyyy-MM-dd @ HH:mm:ss") - $file not found!"
      } else { 
        Write-Output "$(Get-Date -Format "yyyy-MM-dd @ HH:mm:ss") - Moved $file to backup! `n`tFilename: `"$resultfilename`""
      }
    }
    $backupscript = Register-ObjectEvent -InputObject $fsw -EventName "Created" -Action $action -MessageData $DestinationFolder
    Write-Host "Started. WatchFolder: `"$($WatchFolder)`" DestinationFolder: `"$($DestinationFolder)`". Job is in: `$backupscript"
  }
}

monitoring files and directories in C under windows

See FindFirstChangeNotification or ReadDirectoryChangesW.

---

Related Topics