Locking a File in Python

Locking a file in Python

Alright, so I ended up going with the code I wrote here, on my website ^{link is dead, view on archive.org} (also available on GitHub). I can use it in the following fashion:

from filelock import FileLock

with FileLock("myfile.txt"):
    # work with the file as it is now locked
    print("Lock acquired.")

Lock file for access on windows

You could use subprocess to open the file in notepad or excel:

import subprocess, time

subprocess.call('start excel.exe "\lockThisFile.txt\"', shell = True)

time.sleep(10) # if you need the file locked before executing the next commands, you may need to sleep it for a few seconds

or

subprocess.call('notepad > lockThisFile.txt', shell = True)

As written you need shell = True, otherwise windows will give you a syntax error.

(subprocess.Popen() works as well)

You can then close the process later using:

subprocess.call('taskkill /f /im notepad.exe') # or excel.exe

Other options include

-write some C++ code and call it from python (https://msdn.microsoft.com/en-us/library/windows/desktop/aa365203(v=vs.85).aspx)

-call 3rd party programs with subprocess.call():

FileLocker http://www.jensscheffler.de/filelocker (https://superuser.com/questions/294826/how-to-purposefully-exclusively-lock-a-file)

Easy File Locker http://www.xoslab.com/efl.html and Dispatch (from win32com.client import Dispatch), although last choice is the most complex

Locking a txt file with python

I think lockfile should meet your needs

How To Lock Files Using Python

Use os module for this purpose.

import os
file_path = 'D:\\file.txt'    # your file path
os.system(f'echo y| cacls {file_path} /P everyone:n')

I hope you got it.

Opening a file on Windows with exclusive locking in Python

For those who are interested in a Windows specific solution:

import os
import ctypes
import msvcrt
import pathlib

# Windows constants for file operations
NULL = 0x00000000
CREATE_ALWAYS = 0x00000002
OPEN_EXISTING = 0x00000003
FILE_SHARE_READ = 0x00000001
FILE_ATTRIBUTE_READONLY = 0x00000001  # strictly for file reading
FILE_ATTRIBUTE_NORMAL = 0x00000080  # strictly for file writing
FILE_FLAG_SEQUENTIAL_SCAN = 0x08000000
GENERIC_READ = 0x80000000
GENERIC_WRITE = 0x40000000

_ACCESS_MASK = os.O_RDONLY | os.O_WRONLY
_ACCESS_MAP = {os.O_RDONLY: GENERIC_READ,
               os.O_WRONLY: GENERIC_WRITE
               }

_CREATE_MASK = os.O_CREAT | os.O_TRUNC
_CREATE_MAP = {NULL: OPEN_EXISTING,
               os.O_CREAT | os.O_TRUNC: CREATE_ALWAYS
               }

win32 = ctypes.WinDLL('kernel32.dll', use_last_error=True)
win32.CreateFileW.restype = ctypes.c_void_p
INVALID_FILE_HANDLE = ctypes.c_void_p(-1).value


def _opener(path: pathlib.Path, flags: int) -> int:

    access_flags = _ACCESS_MAP[flags & _ACCESS_MASK]
    create_flags = _CREATE_MAP[flags & _CREATE_MASK]

    if flags & os.O_WRONLY:
        share_flags = NULL
        attr_flags = FILE_ATTRIBUTE_NORMAL
    else:
        share_flags = FILE_SHARE_READ
        attr_flags = FILE_ATTRIBUTE_READONLY

    attr_flags |= FILE_FLAG_SEQUENTIAL_SCAN

    h = win32.CreateFileW(path, access_flags, share_flags, NULL, create_flags, attr_flags, NULL)

    if h == INVALID_FILE_HANDLE:
        raise ctypes.WinError(ctypes.get_last_error())

    return msvcrt.open_osfhandle(h, flags)


class _FileControlAccessor(pathlib._NormalAccessor):

    open = staticmethod(_opener)


_control_accessor = _FileControlAccessor()


class Path(pathlib.WindowsPath):

    def _init(self) -> None:

        self._closed = False
        self._accessor = _control_accessor

    def _opener(self, name, flags) -> int:

        return self._accessor.open(name, flags)

How to filelock an entire directory?

This is an issue with how portalocker deals with files - it tries to open them, which works when opening non-existing file in 'w' mode, and fails in 'r' mode.

The solution in your case is to manually create the file (and never remove it, it's a bad idea).

Writer:

with pl.Lock(p_lock, 'w'):
    # do things
    # don't remove the p_lock file

Reader:

p_lock.touch(exist_ok=True)
with pl.Lock(p_lock, 'r', flags=pl.LockFlags.SHARED | pl.LockFlags.NON_BLOCKING):
    # do things
    # don't remove the p_lock file

(and don't bother writing PID into the file, unless it's for your own debugging purpuses)

Python script to write(+lock) / read a file between 3 processes

If I am not misunderstanding, the task could be achieved by using a Lock for gaining writing access, a Semaphore to notify the processes which, having failed to gain the write Lock, will want to read the file and finally a Barrier to re-align the processes.
An Example:

import multiprocessing as mp
import time
from random import randint


def fun(lock_w, sem_r, barrier, task, filename):
  me = mp.current_process()
  for i in range(3):
    time.sleep(randint(1, 4))
    if(lock_w.acquire(block=False)):
      print(me.pid, "write access")
      task()
      sem_r.release()
      sem_r.release()
    else:
      sem_r.acquire()
      print(me.pid, "read access")
      task()
    if barrier.wait() == 0:
      print(me.pid, "releasing Lock")
      lock_w.release()

def task1():
  print("\tPerform Task 1")

def task2():
  print("\tPerform Task 2")

def task3():
  print("\tPerform Task 3")

lock_w = mp.Lock()
sem_r = mp.Semaphore(0)
bar = mp.Barrier(3)
t1 = mp.Process(target=fun, args=(lock_w, sem_r, bar, task1, "foo.txt", ))
t2 = mp.Process(target=fun, args=(lock_w, sem_r, bar, task2, "foo.txt", ))
t3 = mp.Process(target=fun, args=(lock_w, sem_r, bar, task3, "foo.txt", ))
t1.start()
t2.start()
t3.start()
t1.join()
t2.join()
t3.join()

Example putput:

585 write access
    Perform Task 1
586 read access
    Perform Task 2
587 read access
    Perform Task 3
585 releasing Lock
587 write access
    Perform Task 3
586 read access
    Perform Task 2
585 read access
    Perform Task 1
587 releasing Lock
586 write access
    Perform Task 2
585 read access
    Perform Task 1
587 read access
    Perform Task 3
586 releasing Lock

Python locking and unlocking file from multiple processes

1. Ok first, locking a file is a platform-specific operation, so you will need to run different code for different operating systems.
2. Secondly, as @Kevin said here, - "coordinating access to a single file at the OS level is fraught with all kinds of issues that you probably don't want to solve. Your best bet is have a separate process that coordinates read/write access to that file."

---

Related Topics