Safely Create a File If and Only If It Does Not Exist with Python

Safely create a file if and only if it does not exist with Python

Edit: See also Dave Jones' answer: from Python 3.3, you can use the x flag to open() to provide this function.

Original answer below

Yes, but not using Python's standard open() call. You'll need to use os.open() instead, which allows you to specify flags to the underlying C code.

In particular, you want to use O_CREAT | O_EXCL. From the man page for open(2) under O_EXCL on my Unix system:

Ensure that this call creates the file: if this flag is specified in conjunction with O_CREAT, and pathname already exists, then open() will fail. The behavior of O_EXCL is undefined if O_CREAT is not specified.

When these two flags are specified, symbolic links are not followed: if pathname is a symbolic link, then open() fails regardless of where the symbolic link points to.

O_EXCL is only supported on NFS when using NFSv3 or later on kernel 2.6 or later. In environments where NFS O_EXCL support is not provided, programs that rely on it for performing locking tasks will contain a race condition.

So it's not perfect, but AFAIK it's the closest you can get to avoiding this race condition.

Edit: the other rules of using os.open() instead of open() still apply. In particular, if you want use the returned file descriptor for reading or writing, you'll need one of the O_RDONLY, O_WRONLY or O_RDWR flags as well.

All the O_* flags are in Python's os module, so you'll need to import os and use os.O_CREAT etc.

Example:

import os
import errno

flags = os.O_CREAT | os.O_EXCL | os.O_WRONLY

try:
    file_handle = os.open('filename', flags)
except OSError as e:
    if e.errno == errno.EEXIST:  # Failed as the file already exists.
        pass
    else:  # Something unexpected went wrong so reraise the exception.
        raise
else:  # No exception, so the file must have been created successfully.
    with os.fdopen(file_handle, 'w') as file_obj:
        # Using `os.fdopen` converts the handle to an object that acts like a
        # regular Python file object, and the `with` context manager means the
        # file will be automatically closed when we're done with it.
        file_obj.write("Look, ma, I'm writing to a new file!")

How can I safely create a directory (possibly including intermediate directories)?

On Python ≥ 3.5, use pathlib.Path.mkdir:

from pathlib import Path
Path("/my/directory").mkdir(parents=True, exist_ok=True)

For older versions of Python, I see two answers with good qualities, each with a small flaw, so I will give my take on it:

Try os.path.exists, and consider os.makedirs for the creation.

import os
if not os.path.exists(directory):
    os.makedirs(directory)

As noted in comments and elsewhere, there's a race condition – if the directory is created between the os.path.exists and the os.makedirs calls, the os.makedirs will fail with an OSError. Unfortunately, blanket-catching OSError and continuing is not foolproof, as it will ignore a failure to create the directory due to other factors, such as insufficient permissions, full disk, etc.

One option would be to trap the OSError and examine the embedded error code (see Is there a cross-platform way of getting information from Python’s OSError):

import os, errno

try:
    os.makedirs(directory)
except OSError as e:
    if e.errno != errno.EEXIST:
        raise

Alternatively, there could be a second os.path.exists, but suppose another created the directory after the first check, then removed it before the second one – we could still be fooled.

Depending on the application, the danger of concurrent operations may be more or less than the danger posed by other factors such as file permissions. The developer would have to know more about the particular application being developed and its expected environment before choosing an implementation.

Modern versions of Python improve this code quite a bit, both by exposing FileExistsError (in 3.3+)...

try:
    os.makedirs("path/to/directory")
except FileExistsError:
    # directory already exists
    pass

...and by allowing a keyword argument to os.makedirs called exist_ok (in 3.2+).

os.makedirs("path/to/directory", exist_ok=True)  # succeeds even if directory exists.

Writing to a new file if it doesn't exist, and appending to a file if it does

It's not clear to me exactly where the high-score that you're interested in is stored, but the code below should be what you need to check if the file exists and append to it if desired. I prefer this method to the "try/except".

import os
player = 'bob'

filename = player+'.txt'

if os.path.exists(filename):
    append_write = 'a' # append if already exists
else:
    append_write = 'w' # make a new file if not

highscore = open(filename,append_write)
highscore.write("Username: " + player + '\n')
highscore.close()

How to have Python check if a file exists and create it if it doesn't?

Similar question

This is the best way:

try:
    with open(filename) as file:
        # do whatever
except IOError:
    # generate the file

There's also os.path.exists(), but this can be a security concern.

open() in Python does not create a file if it doesn't exist

You should use open with the w+ mode:

file = open('myfile.dat', 'w+')

Create a file if it doesn't exist, otherwise do nothing

You can just catch the FileExistsError and try opening the file with the exclusive creation mode.

open for exclusive creation, failing if the file already exists

def users_csv_file_init():
    try:
        with open("users.csv", 'x') as output_file:
            writer = csv.writer(output_file)
            writer.writerow(["userid", "username", "fname", "lname", "uuid"])
    except FileExistsError:
        pass

how to open (create if not exists) a file while acquiring exclusive lock avoiding races

No, it is not possible as a basic operation supported by Linux/UNIX.

The O_CREAT|O_EXCL technique in the answer you referenced can work here. Instead of exclusively creating the target file, you exclusively create a lockfile whose name is predictably derived from the target file. E.g., os.path.join("/tmp", hashlib.md5(target_filename).hexdigest() + ".lock").

However, as others have suggested, it's not clear that you need to protect both the target file creation and its checksumming + possible replacement. An fcntl advisory lock will suit your needs.

Pythonic way to check if a file exists?

To check if a path is an existing file:

os.path.isfile(path)

Return True if path is an existing
regular file. This follows symbolic
links, so both islink() and
isfile() can be true for the same
path.

Create file but if name exists add number

In a way, Python has this functionality built into the tempfile module. Unfortunately, you have to tap into a private global variable, tempfile._name_sequence. This means that officially, tempfile makes no guarantee that in future versions _name_sequence even exists -- it is an implementation detail.
But if you are okay with using it anyway, this shows how you can create uniquely named files of the form file#.pdf in a specified directory such as /tmp:

import tempfile
import itertools as IT
import os

def uniquify(path, sep = ''):
    def name_sequence():
        count = IT.count()
        yield ''
        while True:
            yield '{s}{n:d}'.format(s = sep, n = next(count))
    orig = tempfile._name_sequence 
    with tempfile._once_lock:
        tempfile._name_sequence = name_sequence()
        path = os.path.normpath(path)
        dirname, basename = os.path.split(path)
        filename, ext = os.path.splitext(basename)
        fd, filename = tempfile.mkstemp(dir = dirname, prefix = filename, suffix = ext)
        tempfile._name_sequence = orig
    return filename

print(uniquify('/tmp/file.pdf'))

---

Related Topics