Project Quotas in Ext4

Is ext4 considered file storage or block storage?

ext4, and any filesystem, is by definition "file storage". That's what a filesystem is.

---

The terms "file storage", "block storage", and "object storage" refers to the API that the storage exports to its consumers, regardless of the transport (network, local, etc.).

ext4 exports a file API (create directories, open files within directories etc.), and is therefore file storage.

If ext4 let you issue inode-based syscalls, it could also count as object storage, because inodes are not aware of hierarchy - the directory hierarchy layer is implemented on top of inodes.

Hard drives, on the other hand, only provide "read at this offset" and "write at this offset" kind of API, and are therefore block storage.

This API is simple enough that it can be used by the lowest-level boot code in the firmware. If the firmware could speak file API or object API, you could boot from there... Which is a thing.

Netboot via PXE uses a file transfer protocol to download the bootloader (as opposed to a block storage protocol), so you can say file storage or object storage ARE capable of booting an OS if they can support that protocol.

---

A lot of confusion in the questions, comments, and answers comes from these storage types usually being stacked.

On Linux, both md and LVM implement block storage using other block storage. They do that by accepting a "write this block" request and translating it to another "write to this block" request to the correct disk.

You can also stack LVM on top of md on top of physical disks.

In fact, block storage API is the only thing that's economically simple enough to support in hardware, so effectively all storage is built on top of block storage.

ext4 is file storage implemented on top of block storage (your hard drive, or LVM). It decides on how files should look like on a block device based on how it's formatted (namely, how its skeleton data structures were written to the block storage), and then when it receives a file write request, it translates it to the appropriate "write this block" requests to the block storage.

WrapFS is file storage that uses another file storage rather than using block storage. Namely, when it receives a file write request, it translates it to another file write requests to the filesystem it's wrapping.

NFS is networked file storage implemented on top of local file storage implemented on top of local block storage. That is, it translates "write this file" requests to a different "write this file" RPC, and the server translates this RPC to a yet different "write this file" request to its local filesystem.

There are even loop devices, which implement block storage on top of file storage.

Many object stores are actually built on top of file stores that are built on top of block stores. Minio, for example, will take a regular filesystem, format it (namely, create the directory structure it needs), and then translate requests like "write this object" to "write into this file in the filesystem".

Likewise, Panasas' HPC cluster implements object storage on top of networked file storage (OSDFS).

Panasas also implements file storage on top of their object storage (translate file write request to object write request), which means it writes some skeleton data e.g. the root inode onto the object store (namely, it formats the object store).

I've seen a proprietary networked file stores that are implemented on top of proprietary local object storage that is implemented on top of networked file stores that are implemented on top of local block stores.

Quota doesn't report any warning even if the soft limit is crossed. Only reports the warning for hard limit

Answer from quota developer | creator, Jan Kara:

What you see is just a message from cp(1) that it got EDQUOT error from a write(2) syscall. The warnings about the softlimit (and the hardlimit) being reached are printed by the kernel to the controlling terminal of the process and you don't seem to see any of those. Now the question is what is the controlling terminal of the cp(1) process in your cloud machine and why can't you see the message. You can find controlling terminal e.g. from ps(1) output (TTY column). Also find these lines in /proc/config.gz from your machine:

CONFIG_QUOTA
CONFIG_QUOTA_NETLINK_INTERFACE
CONFIG_PRINT_QUOTA_WARNING

If CONFIG_PRINT_QUOTA_WARNING is not set , that your kernel is compiled without support for printing quota warnings to the console. If you have CONFIG_QUOTA_NETLINK_INTERFACE=y, that the kernel is compiled with CONFIG_QUOTA_NETLINK_INTERFACE so you can run quota_nld daemon (from quota-tools package) to print messages to console instead.

So I ran sudo quota_nld -b and put it in the cron and all users began to receive warnings.

Creating Directory Loops by Manipulating ext4

Okay, I finally figured out what the problem was.

Looking around in the syslog I found that whenever browsing the directory I would get the following message:

ext4_lookup:1376: inode #12: comm ls: 'bottom' linked to parent dir

After searching the internet for a while and asking ext4 developers on IRC I finally found this patch committed to the ext4 filesystem code that targets exactly what I tried to do.
Nice patch, but here's what you can do to get a directory loop anyhow:

"/"
 |--"top_1"
 |   \--"bottom_1" --> "top_2"
 | 
 \--"top_2"
     \--"bottom_2" --> "top_1"

Creating a new file in a new ext4 file system

The "~" should not be there. You mounted /ext4_filesystem, not ~/ext4_filesystem.

---

Related Topics