What's the Difference Between Tempfile and Mktemp

What's the difference between tempfile and mktemp?

I suspect there's some interesting Unix lore behind this ...

The history of mktemp can be traced to OpenBSD 2.1. However, it became a part of GNU coreutils much later. This post announced the inclusion of mktemp for coreutils.

Until then, tempfile was being used by a number of programs. There was also a proposal to make tempfile a wrapper around mktemp, which was rejected to discourage use of tempfile.

However, the following was added to tempfile manual:

Exclusive creation is not guaranteed when creating files on NFS partitions. tempfile cannot make temporary directories. tempfile is deprecated; you should use mktemp(1) instead.

What's the difference between tempfile and mktemp?

I suspect there's some interesting Unix lore behind this ...

The history of mktemp can be traced to OpenBSD 2.1. However, it became a part of GNU coreutils much later. This post announced the inclusion of mktemp for coreutils.

Until then, tempfile was being used by a number of programs. There was also a proposal to make tempfile a wrapper around mktemp, which was rejected to discourage use of tempfile.

However, the following was added to tempfile manual:

Exclusive creation is not guaranteed when creating files on NFS partitions. tempfile cannot make temporary directories. tempfile is deprecated; you should use mktemp(1) instead.

Python: differences between tempfile.mkdtemp and tempfile.TemporaryDirectory

You are correct in that the only real difference is that TemporaryDirectory will delete itself when it is done. It will let you do something like:

with tempfile.TemporaryDirectory() as dir:
   do_stuff_with(dir)

when you leave the scope of the with, the temporary directory will be deleted. With mkdtemp, you would need to do that manually.

Creating temp files in scripts: Advantages of mktemp over touch-ing a file?

mktemp randomizes the name. It is very important from the security point of view.

Just imagine that you do something like:

echo something > /tmp/temporary-file

in your root-running script.

And someone (who has read your script) does

ln -s /etc/passwd /tmp/temporary-file

before.

This results in /etc/passwd being overwritten, and potentially it can mean different unpleasant things starting from the system becomes broken, and ending with the system becomes hacked (when the input something could be carefully crafted).

The mktemp command could help you in this situation:

TEMP=$(mktemp /tmp/temporary-file.XXXXXXXX)
echo something > ${TEMP}

Now this ln /etc/passwd attack will not work.

A brief insight into the history of mktemp: The mktemp command was invented by the OpenBSD folks, and first appeared in OpenBSD 2.1 back in 1997. Their goal was to improve the security of shell scripts. Previously the norm had been to add $$ to temporary file names, which was absolutely insecure. Now all UNIX/Linux systems have either mktemp or its alternatives, and it became standard de-facto. Funny enough, the mktemp C function was deprecated for being unsecure.

tempfile.mkdtemp() difference on osx?

/tmp is a symlink to /private/tmp on macOS (and same goes for /var). See cross-site Q&A Why is /tmp a symlink to /private/tmp?

So to write it cross-platform, you could just resolve links:

assert os.path.realpath(curdir) == os.path.realpath(abspath_tmpdir)

What is the OS-level handle of tempfile.mkstemp good for?

Your suspicion is correct. The os.remove only removes the directory entry that contains the name of the file. However, the file data remains intact and continues to consume space on the disk until the last open descriptor on the file is closed. During that time normal operations on the file through existing descriptors continue to work, which means you could still use the _ descriptor to seek in, read from, or write to the file after os.remove has returned.

In fact it's common practice to immediately os.remove the file before moving on to using the descriptor to operate on the file contents. This prevents the file from being opened by any other process, and also means that the file won't be left hanging around if this program dies unexpectedly before reaching a later os.remove.

Of course that only works if you're willing and able to use the low-level descriptor for all of your operations on the file, or if you use the os.fdopen method to construct a file object on top of the descriptor and use that new object for all operations. Obviously you only want to do one of those things; mixing descriptor access and file-object access to the same underlying file can produce unexpected results.

os.fdopen(_) should execute faster than open(path) but it doesn't have the context manager integration that open has, so it's not directly usable in a with construct. I think you can use contextlib.closing to get around that.

What effect does this line have in a shell script?

It creates a temporary file and puts the path to it in the $tempfile variable.

`tempfile 2>/dev/null`

runs the tempfile command (man tempfile) and discards any error messages. If it succeeds, it returns the name of the newly created temporary file. If it fails, it returns non-zero, in which case the next part of the command runs.

For a command this || that, that only runs if this fails, i.e. returns non-zero.

$$ is a variable in bash that expands to the process ID of the shell. (Compare the results of ps and echo $$.) So tempfile=/tmp/test$$ will expand to something like tempfile=/tmp/test2278.

Presumably, later in the script, something writes to $tempfile.

Python temporary directory to execute other processes?

tempfile.mkstemp creates a file that is normally visible in the filesystem and returns you the path as well. You should be able to use this to create your input and output files - assuming javac will atomically overwrite the output file if it exists there should be no race condition if other processes on your system don't misbehave.

Why mktemp with -u option is stated as unsafe in its manual?

When you use -u, no file is created, so using the name later doesn't guarantee to access a temporary file created by you.

There's a window of opportunity for another process to create a file of that name between invoking mktemp and using the result. That file may be a symbolic link, enabling another user to abuse your permissions to write somewhere.

If you use mktemp -u, you need to very carefully ensure that such a race is not exploitable.

Usually, it's better to create a temporary directory (mktemp -d), and use names of your choice within that directory.

---

Related Topics