What Is Different Between Makedirs and Mkdir of Os

the difference between os.mkdir() and os.makedirs()

os.makedirs() : Recursive directory creation function. Like os.mkdir(), but makes all intermediate-level directories needed to contain the leaf directory.

What this means is that you should not try to create nested directories with os.mkdir() but use os.makedirs() instead.

In your case, I am guessing that you want to create a directory under your home directory, in which case you would need something like os.mkdir("/home/img"), which will fail if you do not have enough permissions.

You could try and do something like: os.chdir('/home') and after that os.mkdir('img') so you create home/img in steps! Good luck!

When should I use pathlib.Path.mkdir() vs os.mkdir() or os.makedirs()?

mkdir does not create intermediate-level directories that are not existent at the time of function calling. makedirs does.

Path.mkdir also does, but it's called as a method of a Path object (whereas the other two are called receiving the path, be it by a string with the path or a Path object (starting on Python 3.6), as an argument to the function).

Otherwise, behavior is the same.

Benefit of using os.mkdir vs os.system(mkdir)

Correctness

Think about what happens if your directory name contains spaces:

mkdir hello world

...creates two directories, hello and world. And if you just blindly substitute in quotes, that won't work if your filename contains that quoting type:

'mkdir "' + somedir + '"'

...does very little good when somedir contains hello "cruel world".d.


Security

In the case of:

os.system('mkdir somedir')

...consider what happens if the variable you're substituting for somedir is called ./$(rm -rf /)/hello.

Also, calling os.system() (or subprocess.call() with shell=True) invokes a shell, which means that you can be open to bugs such as ShellShock; if your /bin/sh were provided by a ShellShock-vulnerable bash, and your code provided any mechanism for arbitrary environment variables to be present (as is the case with HTTP headers via CGI), this would provide an opportunity for code injection.


Performance

os.system('mkdir somedir')

...starts a shell:

/bin/sh -c 'mkdir somedir'

...which then needs to be linked and loaded; needs to parse its arguments; and needs to invoke the external command mkdir (meaning another link and load cycle).


A significant improvement is the following:

subprocess.call(['mkdir', '--', somedir], shell=False)

...which only invokes the external mkdir command, with no shell; however, as it involves a fork()/exec() cycle, this is still a significant performance penalty over the C-library mkdir() call.

In the case of os.mkdir(somedir), the Python interpreter directly invokes the appropriate syscall -- no external commands at all.


Error Handling

If you call os.mkdir('somedir') and it fails, you get an IOError with the appropriate errno thrown, and can trivially determine the type of the error.

If the mkdir external command fails, you get a failed exit status, but no handle on the actual underlying problem without parsing its stderr (which is written for humans, not machine readability, and which will vary in contents depending on the system's current locale).

mkdir -p functionality in Python

For Python ≥ 3.5, use pathlib.Path.mkdir:

import pathlib
pathlib.Path("/tmp/path/to/desired/directory").mkdir(parents=True, exist_ok=True)

The exist_ok parameter was added in Python 3.5.


For Python ≥ 3.2, os.makedirs has an optional third argument exist_ok that, when True, enables the mkdir -p functionality—unless mode is provided and the existing directory has different permissions than the intended ones; in that case, OSError is raised as previously:

import os
os.makedirs("/tmp/path/to/desired/directory", exist_ok=True)

For even older versions of Python, you can use os.makedirs and ignore the error:

import errno    
import os

def mkdir_p(path):
try:
os.makedirs(path)
except OSError as exc: # Python ≥ 2.5
if exc.errno == errno.EEXIST and os.path.isdir(path):
pass
# possibly handle other errno cases here, otherwise finally:
else:
raise

Passing commands to OS: What is wrong here?

You don't need the double quotes. subprocess passes the parameters directly to the process, so you don't need to prepare them for parsing by a shell. You also don't need the trailing slash, and should use os.path.join to combine path components:

path = os.path.join(destination, filename)

EDIT: You should accept @Fabian's answer, which explains that you don't need subprocess at all (I knew that).

How can I safely create a nested directory?

On Python ≥ 3.5, use pathlib.Path.mkdir:

from pathlib import Path
Path("/my/directory").mkdir(parents=True, exist_ok=True)

For older versions of Python, I see two answers with good qualities, each with a small flaw, so I will give my take on it:

Try os.path.exists, and consider os.makedirs for the creation.

import os
if not os.path.exists(directory):
os.makedirs(directory)

As noted in comments and elsewhere, there's a race condition – if the directory is created between the os.path.exists and the os.makedirs calls, the os.makedirs will fail with an OSError. Unfortunately, blanket-catching OSError and continuing is not foolproof, as it will ignore a failure to create the directory due to other factors, such as insufficient permissions, full disk, etc.

One option would be to trap the OSError and examine the embedded error code (see Is there a cross-platform way of getting information from Python’s OSError):

import os, errno

try:
os.makedirs(directory)
except OSError as e:
if e.errno != errno.EEXIST:
raise

Alternatively, there could be a second os.path.exists, but suppose another created the directory after the first check, then removed it before the second one – we could still be fooled.

Depending on the application, the danger of concurrent operations may be more or less than the danger posed by other factors such as file permissions. The developer would have to know more about the particular application being developed and its expected environment before choosing an implementation.

Modern versions of Python improve this code quite a bit, both by exposing FileExistsError (in 3.3+)...

try:
os.makedirs("path/to/directory")
except FileExistsError:
# directory already exists
pass

...and by allowing a keyword argument to os.makedirs called exist_ok (in 3.2+).

os.makedirs("path/to/directory", exist_ok=True)  # succeeds even if directory exists.


Related Topics



Leave a reply



Submit