the difference between os.mkdir() and os.makedirs()
os.makedirs() : Recursive directory creation function. Like os.mkdir(), but makes all intermediate-level directories needed to contain the leaf directory.
What this means is that you should not try to create nested directories with os.mkdir() but use os.makedirs() instead.
In your case, I am guessing that you want to create a directory under your home directory, in which case you would need something like os.mkdir("/home/img"), which will fail if you do not have enough permissions.
You could try and do something like: os.chdir('/home') and after that os.mkdir('img') so you create home/img in steps! Good luck!
When should I use pathlib.Path.mkdir() vs os.mkdir() or os.makedirs()?
mkdir
does not create intermediate-level directories that are not existent at the time of function calling. makedirs
does.
Path.mkdir
also does, but it's called as a method of a Path object (whereas the other two are called receiving the path, be it by a string with the path or a Path object (starting on Python 3.6), as an argument to the function).
Otherwise, behavior is the same.
Benefit of using os.mkdir vs os.system(mkdir)
Correctness
Think about what happens if your directory name contains spaces:
mkdir hello world
...creates two directories, hello
and world
. And if you just blindly substitute in quotes, that won't work if your filename contains that quoting type:
'mkdir "' + somedir + '"'
...does very little good when somedir
contains hello "cruel world".d
.
Security
In the case of:
os.system('mkdir somedir')
...consider what happens if the variable you're substituting for somedir
is called ./$(rm -rf /)/hello
.
Also, calling os.system()
(or subprocess.call()
with shell=True
) invokes a shell, which means that you can be open to bugs such as ShellShock; if your /bin/sh
were provided by a ShellShock-vulnerable bash, and your code provided any mechanism for arbitrary environment variables to be present (as is the case with HTTP headers via CGI), this would provide an opportunity for code injection.
Performance
os.system('mkdir somedir')
...starts a shell:
/bin/sh -c 'mkdir somedir'
...which then needs to be linked and loaded; needs to parse its arguments; and needs to invoke the external command mkdir
(meaning another link and load cycle).
A significant improvement is the following:
subprocess.call(['mkdir', '--', somedir], shell=False)
...which only invokes the external mkdir
command, with no shell; however, as it involves a fork()/exec() cycle, this is still a significant performance penalty over the C-library mkdir()
call.
In the case of os.mkdir(somedir)
, the Python interpreter directly invokes the appropriate syscall -- no external commands at all.
Error Handling
If you call os.mkdir('somedir')
and it fails, you get an IOError
with the appropriate errno
thrown, and can trivially determine the type of the error.
If the mkdir
external command fails, you get a failed exit status, but no handle on the actual underlying problem without parsing its stderr (which is written for humans, not machine readability, and which will vary in contents depending on the system's current locale).
mkdir -p functionality in Python
For Python ≥ 3.5, use pathlib.Path.mkdir
:
import pathlib
pathlib.Path("/tmp/path/to/desired/directory").mkdir(parents=True, exist_ok=True)
The exist_ok
parameter was added in Python 3.5.
For Python ≥ 3.2, os.makedirs
has an optional third argument exist_ok
that, when True
, enables the mkdir -p
functionality—unless mode
is provided and the existing directory has different permissions than the intended ones; in that case, OSError
is raised as previously:
import os
os.makedirs("/tmp/path/to/desired/directory", exist_ok=True)
For even older versions of Python, you can use os.makedirs
and ignore the error:
import errno
import os
def mkdir_p(path):
try:
os.makedirs(path)
except OSError as exc: # Python ≥ 2.5
if exc.errno == errno.EEXIST and os.path.isdir(path):
pass
# possibly handle other errno cases here, otherwise finally:
else:
raise
Passing commands to OS: What is wrong here?
You don't need the double quotes. subprocess
passes the parameters directly to the process, so you don't need to prepare them for parsing by a shell. You also don't need the trailing slash, and should use os.path.join to combine path components:
path = os.path.join(destination, filename)
EDIT: You should accept @Fabian's answer, which explains that you don't need subprocess at all (I knew that).
How can I safely create a nested directory?
On Python ≥ 3.5, use pathlib.Path.mkdir
:
from pathlib import Path
Path("/my/directory").mkdir(parents=True, exist_ok=True)
For older versions of Python, I see two answers with good qualities, each with a small flaw, so I will give my take on it:
Try os.path.exists
, and consider os.makedirs
for the creation.
import os
if not os.path.exists(directory):
os.makedirs(directory)
As noted in comments and elsewhere, there's a race condition – if the directory is created between the os.path.exists
and the os.makedirs
calls, the os.makedirs
will fail with an OSError
. Unfortunately, blanket-catching OSError
and continuing is not foolproof, as it will ignore a failure to create the directory due to other factors, such as insufficient permissions, full disk, etc.
One option would be to trap the OSError
and examine the embedded error code (see Is there a cross-platform way of getting information from Python’s OSError):
import os, errno
try:
os.makedirs(directory)
except OSError as e:
if e.errno != errno.EEXIST:
raise
Alternatively, there could be a second os.path.exists
, but suppose another created the directory after the first check, then removed it before the second one – we could still be fooled.
Depending on the application, the danger of concurrent operations may be more or less than the danger posed by other factors such as file permissions. The developer would have to know more about the particular application being developed and its expected environment before choosing an implementation.
Modern versions of Python improve this code quite a bit, both by exposing FileExistsError
(in 3.3+)...
try:
os.makedirs("path/to/directory")
except FileExistsError:
# directory already exists
pass
...and by allowing a keyword argument to os.makedirs
called exist_ok
(in 3.2+).
os.makedirs("path/to/directory", exist_ok=True) # succeeds even if directory exists.
Related Topics
How to Exit Linux Terminal Using Python Script
How I Open Remote Server Folder Using Python
Priority of the Logical Operators Not, And, or in Python
How to Do Parallel Programming in Python
Add Leading Zeros to Strings in Pandas Dataframe
Detecting Consecutive Integers in a List
Syntaxerror: Non-Ascii Character '\Xa3' in File When Function Returns '£'
What Do Square Brackets, "[]", Mean in Function/Class Documentation
Convert Floats to Ints in Pandas
Python Multiprocessing Linux Windows Difference
Pil Installation Fails Missing:Stdarg.H
Calling Java/Scala Function from a Task
How to Get a List of All the Duplicate Items Using Pandas in Python
Matplotlib Plots: Removing Axis, Legends and White Spaces
Polling the Keyboard (Detect a Keypress) in Python
How to Run a Python Program in the Command Prompt in Windows 7