Unix File System: How Are File Names Translated to Disk Sectors

Unix File System: How are file names translated to disk sectors?

In addition of tmp's answer:

Files really are inodes.

Usually, a given file has some entry in some directory pointing to its inode. Directories are mapping names to inodes, and directories are a kind of file. See stat(2) for what an inode contains (and can be queried by application code), in particular for the various file types (plain file, directory, char or block device, fifo, symlink, ....). So a directory is often a dictionnary (implemented in various, file system specific, ways) mapping strings to inodes. So in the directory /bin/ there is usually an entry associating bash to the inode for the ELF executable of the bash shell (i.e. /bin/bash). Use readdir(3) -which in turn calls getdents(2)- to read the entries in a directory.

It may happen that a given inode is not any more reachable by some name. This happens in particular when a process is open(2)-ing a file, then unlink(2)-ing it (while keep the opened file descriptor). This is the preferred way to make temporary files. (They will be released by the kernel when no more processes use them).

It may also happen that a given inode has several directories entries pointing to it. (i.e. a file has "several names") This happens with link(2) syscall. (symlink files are created with symlink(2)).

See also path_resolution(7) and unix file system & file system & ext2 & ext3 & ext4 & btrfs & comparison of file systems wikipages. Read also this old file system description (some details are rotten, but the general idea is here, notably the role of VFS)

Reading a sector on the boot disk

In Linux, you can read from the special device file /dev/sda, assuming the hard drive you want to read is the first one. You need to be root to read this file. To read sector 2, you just seek to offset 2*SECTOR_SIZE and read in SECTOR_SIZE bytes.

I don't know if this device file is available on OS X. Check for interestingly named files under /dev such as /dev/sda or /dev/hda.

inode - move a file which is already opened

Using rename() on a file within the same file system just changes the name that points to the inode. You can even use rename() to move that name into another directory - as long as it's in the same file system:

The rename() function shall change the name of a file. The old
argument points to the pathname of the file to be renamed. The new
argument points to the new pathname of the file. ...

Note that the POSIX specification for rename() is a lot more specific than the C Standard rename() specification:

7.21.4.2 The rename function

Synopsis

#include <stdio.h>
int rename(const char *old, const char *new);

Description

The rename function causes the file whose name is the string pointed
to by old to be henceforth known by the name given by the string
pointed to by new . The file named old is no longer accessible by
that name. If a file named by the string pointed to by new exists
prior to the call to the rename function, the behavior is
implementation-defined.

Returns

The
rename
function returns zero if the operation succeeds, nonzero if it fails,
269)
in
which case if the file existed previously it is still known by its original name.

(That's the entire C Standard specification for rename(). Read the POSIX link for a lot more information.)

So how can you rename a file while your watching it in an application? The underlying inode that your movie-watching process's open file descriptor uses to access the file that you just rename()'d doesn't change.

That's the same reason why you can unlink() (delete) a file while it's in use - all you're doing is removing the entry that points to the inode - one of the links to the inode. The space used by the file isn't freed until the last link to the inode is removed - and an open file descriptor counts as a link. That's also why the function that deletes the directory entry for a file is called unlink() - that's all it does. And yes, a file (inode) can have more than one link pointing to it.

File management systems: device drivers and basic file systems

Break this down into layer:

Layer 1) Physical I/O to a disk requires specifying the platter, sector and track to read or write to a block.

Layer 2) Logical I/O to a disk arranges the blocks in a numeric sequence and one reads or writes to a specific logical block number that gets translated into into the track/platter/sector.

Operating systems generally have support for a Logical I/O and physical I/O to the disk. That said, most disks these days do the logical to physical translation. O/S support for that is only needed for older disks.

If the device supports logical I/O the device driver performs the I/O. If the device only supports physical I/O the device driver usually handles both the Logical and Physical layers. Thus, the physical I/O layer only exists in drivers for disks that do not do logical I/O in hardware. If the disk supports logical I/O, there is no layer 1 in the driver.

All of the above is what is appears the your first quote is addressing.

Layer 3) Virtual I/O writes to a specific bytes or blocks (depending upon the O/S) to a file. This layer is usually handled outside the device driver. At this layer there are separate modules for each supported file system. Virtual I/O requests to all disks using the same file system go through the same module.

Handling Virtual I/O requires much more complexity than simply reading an writing disk blocks. The virtual I/O layer requires working with the underlying disk file system structure to allocate the blocks to a specific file.

This appears to be what is referred to in the second quote. What is confusing to me is why it is calling this the "physical I/O" layer instead of the "virtual I/O" layer.

Everywhere I have been Physical I/O and Logical I/O are the writing of raw blocks to a disk without regard to the file system on the disk.

search and print all the files and subfolders of a directory

In pure standard C, you cannot do that, since the C11 standard n1570 does not mention directories.

You need some operating system specific things. On Linux, look into nftw(3). It uses lower level stuff like opendir(3), readdir(3), closedir, stat(2) that you might use directly.

On Windows, the API to handle directories is very different.

Some frameworks like Poco, Boost, Qt, ... try to define some common abstraction on directories working on several OSes. But the precise notion of directory and file is different on Windows and on Unix or Linux. See also this.

Linux what is the benefit of having . and .. as hard link and not soft links?

How . and .. can be hard links, isn't it breaking the file system ?

The POSIX specification requires that, and Unix tradition had . and .. be hard links since several decades (if they did not exist, the file system could be corrupted or disappear or become broken without possibility of repair).

Read the wikipage on hard links to understand them.

Remember that (on Unix and Linux) files don't have necessarily names. On Unix and Linux, a file is an inode (see also inode(7)) and can have one (this is the usual case), zero or several names. A directory is a mapping between names and inodes and is conceptually required to have both . and .. entries. See also this and that answers to related questions.

Read also Operating Systems: Three Easy Pieces notably the chapters about files, directories, file systems. Read also about VFS.

Hard links are usually not allowed on directories (and link(2) would then fail) to facilitate a reference counting scheme in the kernel and forbid circular references in file trees.

The . and .. entries are created at mkdir(2) time. The .. entry could be changed by rename(2) (and conceptually by mount(2)). The rmdir(2) syscall requires the directory to be empty, i.e. to contain only those two  . & .. entries. See also path_resolution(7) (and this).

Finding goal in ext4 file system

ext4_ext_find_goal() is called when we need to allocate new block.

In here, this function is called as you said.

ar.goal = ext4_ext_find_goal(inode, path, map->m_lblk);

In this context, path is not NULL. So, ext4_ext_find_goal() will find block number of that path, and return this block number.

---

Related Topics