Maximum Number of Inodes in a Directory

what determines the maximum inodes of a single partition?

You did not mention a specific file system so I am going to assume ext4, although what I am saying should mostly apply to ext3 as well.

The number of inodes is determined when the file-system is created. File-systems are generally written to be flexible enough so that this number can be specified at creation to better suit the needs of the system. So if you have a lot of small files you can create more inodes and if you have a smaller number of large files you can create less inodes.

With mkfs.ext4 you can use the -i flag to specify the bytes per inode. The default value as of now is typically 16384 bytes per inode. This number is nothing specifically special but if you assume the typical 256 bytes for the inode size and 16384 bytes per inode you get approximately 1.56% of the disk space being used by inodes.

What is the max number of files that can be kept in a single folder, on Win7/Mac OS X/Ubuntu Filesystems?

In Windows (assuming NTFS): 4,294,967,295 files

In Linux (assuming ext4): also 4 billion files (but it can be less with some custom inode tables)

In Mac OS X (assuming HFS): 2.1 billion

But I have put around 65000 files into a single directory and I have to say just loading the file list can kill an average PC.

Maximum number of files/directories on Linux?

ext[234] filesystems have a fixed maximum number of inodes; every file or directory requires one inode. You can see the current count and limits with df -i. For example, on a 15GB ext3 filesystem, created with the default settings:

Filesystem           Inodes  IUsed   IFree IUse% Mounted on
/dev/xvda           1933312 134815 1798497    7% /

There's no limit on directories in particular beyond this; keep in mind that every file or directory requires at least one filesystem block (typically 4KB), though, even if it's a directory with only a single item in it.

As you can see, though, 80,000 inodes is unlikely to be a problem. And with the dir_index option (enablable with tune2fs), lookups in large directories aren't too much of a big deal. However, note that many administrative tools (such as ls or rm) can have a hard time dealing with directories with too many files in them. As such, it's recommended to split your files up so that you don't have more than a few hundred to a thousand items in any given directory. An easy way to do this is to hash whatever ID you're using, and use the first few hex digits as intermediate directories.

For example, say you have item ID 12345, and it hashes to 'DEADBEEF02842.......'. You might store your files under /storage/root/d/e/12345. You've now cut the number of files in each directory by 1/256th.

How many files can I put in a directory?

FAT32:

Maximum number of files: 268,173,300
Maximum number of files per directory: 2¹⁶ - 1 (65,535)
Maximum file size: 2 GiB - 1 without LFS, 4 GiB - 1 with

NTFS:

Maximum number of files: 2³² - 1 (4,294,967,295)
Maximum file size
- Implementation: 2⁴⁴ - 2⁶ bytes (16 TiB - 64 KiB)
- Theoretical: 2⁶⁴ - 2⁶ bytes (16 EiB - 64 KiB)
Maximum volume size
- Implementation: 2³² - 1 clusters (256 TiB - 64 KiB)
- Theoretical: 2⁶⁴ - 1 clusters (1 YiB - 64 KiB)

ext2:

Maximum number of files: 10¹⁸
Maximum number of files per directory: ~1.3 × 10²⁰ (performance issues past 10,000)
Maximum file size
- 16 GiB (block size of 1 KiB)
- 256 GiB (block size of 2 KiB)
- 2 TiB (block size of 4 KiB)
- 2 TiB (block size of 8 KiB)
Maximum volume size
- 4 TiB (block size of 1 KiB)
- 8 TiB (block size of 2 KiB)
- 16 TiB (block size of 4 KiB)
- 32 TiB (block size of 8 KiB)

ext3:

Maximum number of files: min(volumeSize / 2¹³, numberOfBlocks)
Maximum file size: same as ext2
Maximum volume size: same as ext2

ext4:

Maximum number of files: 2³² - 1 (4,294,967,295)
Maximum number of files per directory: unlimited
Maximum file size: 2⁴⁴ - 1 bytes (16 TiB - 1)
Maximum volume size: 2⁴⁸ - 1 bytes (256 TiB - 1)

what is the max files per directory in EXT4?

It depends upon the MKFS parameters used during the filesystem creation. Different Linux flavors have different defaults, so it's really impossible to answer your question definitively.

AWS EFS: Max number of files allowed in a folder

As you noted, I don't see any mention of file count limits in the AWS EFS Limits doc https://docs.aws.amazon.com/efs/latest/ug/limits.html

These limits guides are a good way to know about hard service limits that would affect any implementation. If it's not mentioned here, there likely is no such limit.

Also note, EFS implements NFSv4, which is a network file sharing protocol. There is some good discussion of why NFS itself would have no such inherent limit. See https://superuser.com/questions/700102/file-count-limit-for-nfs-mounted-drive and See also https://aws.amazon.com/efs/

Finally, here is a good discussion on file system limits How many files can I put in a directory?

I would assume that the file system back end on EFS has to be ext4. As noted above, an ext4 filesystem has no limit to the number of files in a directory.

Based on the above, I believe is there is no such limit in EFS. It should handle your file requirements.

Number of inodes in a partition not matching up to the maximum number of inodes the partition should support

I'm pretty sure that inodes are allocated statically when you create the volume (using mfs.ext3 in this case). For whatever reason, mkfs.ext3 decided to reserve 13 Million inodes and now you can't create any more files.

See this 2001 discussion of inodes

The Wikipedia ext3 page has a footnote explaining this more concisely: wiki link

Also, inodes are allocated per file (not block), which is why there are only 13M inodes - mkfs.ext3 must have been configured with an average file size of 8 KB which would account for the issue you're seeing.

How to obtain the maximum number of subdirectories in a directory from a C program on Linux?

There is no specific limit on the number of files or subdirectories within a given directory. There are limits on the total number of inodes in a file system depending on how the file system was built and (mostly) how much space there is in total in the file system. Each named object requires an inode (but, thanks to hard links, a single inode can have multiple names). Thus, the limit is primarily controlled by the space available in the file system.

There are usually limits on how deep a directory hierarchy can be — that's the POSIX constant {PATH_MAX} defined (or not) in <limits.h>, and the related lower-bounds on the minimum acceptable value for {PATH_MAX} — {_XOPEN_PATH_MAX} (1024) and {_POSIX_PATH_MAX} (256).

You can use the functions fpathconf() and pathconf() to find properties of file systems at run-time. The related function sysconf() handles other configuration properties.