What Happens Internally When Deleting an Opened File in Linux

Will we ever be able to delete an open file in Windows?

Your initial statement is not correct. Windows does allow open files to be deleted. You just have to specify FILE_SHARE_DELETE and you're all set. Careful programmers should sensibly decide if that flag (or sharing for reading/writing) make sense and pass it.

An anti virus product that does not open files with full sharing (including deletion) enabled is buggy.

Windows does, however, remember the current working directory of any process and prevents it from being deleted. This working directory is independent of the location of any files opened by the process.

Will blocks of a recently-created subsequently deleted file ever be written back to disc?

I agree with Jonathan Leffler, but not only for classic Unix file systems: There has been a discussion with a similar topic concerning the ext4 file system.

In a comment, Theodore Ts'o (one of the main developers of the ext4 file system) states: ``... for example, if you create a scratch file, and then delete it 20 seconds later, it will probably never hit the disk.''

What happens to already opened files when you change process ownership (uid/gid) in Linux?

When you open a file, your ability to do so is determined by your effective uid and gid at the time you open the file.

When you change your effective uid or gid, it has no effect on any open file descriptors that you may have.

In most cases, if you have a valid file descriptor, that's all you need to read or write the resource that descriptor is connected to. The fact that you hold the valid file descriptor is supposed to be all the proof you need that you have permission to read/write the underlying resource.

When you read or write using an ordinary file descriptor, no additional authorization checks are performed. This is partly for efficiency (because those authentication checks would be expensive to perform each time), and partly so that -- this may be exactly what you are trying to do -- you can open a privileged resource, downgrade your process's privileges, and continue to access the open resource.

Bottom line: Yes, it's entirely possible for a process to use an open file descriptor to read or write a file which (based on its current uid/gid) it would not be able to open.

---

Footnote: What I've said is true for ordinary Unix file descriptors connected to ordinary resources (files, devices, pipes, network streams, etc.). But as @Mark Plotnick reminds in a comment, some file descriptors and underlying resources are "different" -- NFS and Linux /proc files are two examples. For those, it's possible for additional checks to be performed at the time of read/write.

How does Linux manage deleting temporary files?

There's no bit that marks a file as a temporary file.

Every inode has a link count field, which is the number of directory entries that refer to the file. Every time you make a hard link to a file this count is increased, and when you remove a name it's decreased; when the count goes to zero, the file is deleted (the inode is marked as available, and all the data blocks are put on the free list).

When a file is opened in a process, a copy of the inode is kept in the kernel's file table, and the number of file handles that refer to it are added into the link count in this copy. When a process closes its file descriptor, the link count is decremented. The file isn't actually removed until this in-memory link count drops to zero. This is what keeps a file on disk while it's open, even if all the names are removed.

So when you create a temporary file, it performs the following steps:

1. Creates the file. The on-disk inode link count = 1.
2. Opens the file. The kernel inode link count = 2.
3. Removes the filename. The kernel inode link count = 1.

At this point, the process can keep using the temporary file, but it can't be opened by another process because it has no name.

When the process closes the file handle, the link count goes to 0, and the file is deleted.

Recent versions of Linux have an O_TMPFILE flag to open(2) that automates this. Instead of specifying a filename, you just specify the directory, which is just used to find a filesystem to hold the file data. When this is used, it effectively does all 3 steps above in one call, but it never actually creates the filename anywhere (so race conditions and name conflicts are avoided).

Linux vs. Windows File.delete()

AFAIK, No OS supports deleting part of a file (except the end)

You can't delete a file you have open so you have to be sure you closed it everywhere, but you can create a temporary file and rename it as the original. (No need to copy it back)

Here is how I might write it

public static void removeLine(String filename, String line) {
    File from = new File(filename);
    File tmp = new File(filename + ".tmp");
    PrintWriter pw = null;
    BufferedReader br = null;
    try {
        pw = new PrintWriter(tmp);
        br = new BufferedReader(new FileReader(from));
        boolean found = false;
        for (String line2; (line2 = br.readLine()) != null; )
            if (line2.equals(line))
                found = true;
            else
                pw.println(line2);
        pw.close();
        br.close();
        if (found) {
            from.delete();
            tmp.renameTo(from);
        } else {
            tmp.delete();
        }
    } catch (IOException e) {
        // log error.
        try { if (br != null) br.close(); } catch (IOException ignored) { }
        if (pw != null) pw.close();
    }
}

Is fread possible after a file is removed?

In Unix and Unix-like operating systems, the file doesn't actually go away until the last open file handle on it is closed. This is a very useful trick for temporary files - if you unlink it as soon as you open it, the file won't be visible to other processes, and it will be removed from the system as soon as your program closes it, ends or crashes. That helps prevent the proliferation of orphan temp files.

Practically (glossing over some technical details here) what happens is that Unix file systems are reference counted. When you open the file, you actually get connected to the file's inode (which is the real indication of where the actual content of the file lives). But unlinking the file just removes the directory entry, so the file doesn't have a name any more. The file system will only reclaim the file space (ie the inode) if it isn't in any directory entries, AND nobody has it open. The other processes can't open it in the ordinary manner because they can't map a file name to the inode.

Note that Unix file systems allow multiple directory entries to point to the same inode - we call that a "hard link". If you do a "ls -l", one of the fields is the count of hard links to that same inode, and if you do an "ls -li", you can see the actual inode address.

Finding size of deleted open file in Linux?

This prints the /proc/[pid]/fd/[fd] path, its symlink target, and the size of the actual file:

find /proc/[0-9]*/fd -lname '*(deleted)' \
    -printf '%p => %l\t' -exec stat -Lc '%s' {} \; 2>/dev/null

There may be smarter ways to do it ;-)

Cygwin: Deleting file when handle is opened

under windows not always possible delete file. for example when some process mapping this file as an image

if try delete running EXE file with rm.exe - it first call ZwOpenFile with DesiredAccess = DELETE, ShareAccess = FILE_SHARE_DELETE and OpenOptions = FILE_OPEN_FOR_BACKUP_INTENT. this is ok. than called ZwSetInformationFile with FileDispositionInformation - the DeleteFile from FILE_DISPOSITION_INFORMATION set to TRUE. this call failed with status STATUS_CANNOT_DELETE.

filesystem return STATUS_CANNOT_DELETE exactly from this place:

        //  Make sure there is no process mapping this file as an image.

        if (!MmFlushImageSection( &Fcb->NonPaged->SectionObjectPointers,
                                  MmFlushForDelete )) {

            DebugTrace(-1, Dbg, "Cannot delete user mapped image\n", 0);

            return STATUS_CANNOT_DELETE;
}

than rm.exe again try open file, already with OpenOptions = FILE_OPEN_FOR_BACKUP_INTENT | FILE_DELETE_ON_CLOSE options. but this call of course fail with the same STATUS_CANNOT_DELETE. now error from this point:

//  If the user wants to delete on close, we must check at this
//  point though.
//

if (FlagOn(*DesiredAccess, FILE_WRITE_DATA) || DeleteOnClose) {

    Fcb->OpenCount += 1;
    DecrementFcbOpenCount = TRUE;

    if (!MmFlushImageSection( &Fcb->NonPaged->SectionObjectPointers,
                              MmFlushForWrite )) {

        Iosb.Status = DeleteOnClose ? STATUS_CANNOT_DELETE :
                                      STATUS_SHARING_VIOLATION;
        try_return( Iosb );
    }
}

after this rm.exe again call ZwSetInformationFile already with FileRenameInformation - where RootDirectory from FILE_RENAME_INFORMATION point to volume (on which file located) root (so like \Device\HarddiskVolume<N>\ and FileName point to some path in recycle bin. as result file actually moved but not deleted. rm.exe deceives you

---

Related Topics