Why Does Python Give "Oserror: [Errno 36] File Name Too Long" for Filename Shorter Than Filesystem's Limit

Why does Python give OSError: [Errno 36] File name too long for filename shorter than filesystem's limit?

I've found a solution. It turns out that I'm not dealing with pure ext4---it's actually ecryptfs layered atop ext4, carried over from a previous Ubuntu install. As suggested in an unaccepted answer to this related question the ecryptfs layer imposes a limit of 143 characters to the filename length. A definitive answer to the file name length limit under ecryptfs can be found here.

Ubuntu encrypted home directory | Errno 36 File Name too long

The fact that the filename that's too long starts with '.cache/www.example.com' explains the problem.

httplib2 optionally caches requests that you make. You've enabled caching, and you've given it .cache as the cache directory.

The easy solution is to put the cache directory somewhere else.

Without seeing your code, it's impossible to tell you how to fix it. But it should be trivial. The documentation for FileCache shows that it takes a dir_name as the first parameter.

Or, alternatively, you can pass a safe function that lets you generate a filename from the URI, overriding the default. That would allow you to generate filenames that fit within the 144-character limit for Ubuntu encrypted fs.

Or, alternatively, you can create your own object with the same interface as FileCache and pass that to the Http object to use as a cache. For example, you could use tempfile to create random filenames, and store a mapping of URLs to filenames in an anydbm or sqlite3 database.

A final alternative is to just turn off caching, of course.

What characters are forbidden in Windows and Linux directory names?

A “comprehensive guide” of forbidden filename characters is not going to work on Windows because it reserves filenames as well as characters. Yes, characters like
* " ? and others are forbidden, but there are a infinite number of names composed only of valid characters that are forbidden. For example, spaces and dots are valid filename characters, but names composed only of those characters are forbidden.

Windows does not distinguish between upper-case and lower-case characters, so you cannot create a folder named A if one named a already exists. Worse, seemingly-allowed names like PRN and CON, and many others, are reserved and not allowed. Windows also has several length restrictions; a filename valid in one folder may become invalid if moved to another folder. The rules for
naming files and folders
are on the Microsoft docs.

You cannot, in general, use user-generated text to create Windows directory names. If you want to allow users to name anything they want, you have to create safe names like A, AB, A2 et al., store user-generated names and their path equivalents in an application data file, and perform path mapping in your application.

If you absolutely must allow user-generated folder names, the only way to tell if they are invalid is to catch exceptions and assume the name is invalid. Even that is fraught with peril, as the exceptions thrown for denied access, offline drives, and out of drive space overlap with those that can be thrown for invalid names. You are opening up one huge can of hurt.

Find all files in a directory with extension .txt in Python

You can use glob:

import glob, os
os.chdir("/mydir")
for file in glob.glob("*.txt"):
    print(file)

or simply os.listdir:

import os
for file in os.listdir("/mydir"):
    if file.endswith(".txt"):
        print(os.path.join("/mydir", file))

or if you want to traverse directory, use os.walk:

import os
for root, dirs, files in os.walk("/mydir"):
    for file in files:
        if file.endswith(".txt"):
             print(os.path.join(root, file))

How do I get file creation and modification date/times?

In Python 3.4 and above, you can use the object oriented pathlib module interface which includes wrappers for much of the os module. Here is an example of getting the file stats.

>>> import pathlib
>>> fname = pathlib.Path('test.py')
>>> assert fname.exists(), f'No such file: {fname}'  # check that the file exists
>>> print(fname.stat())
os.stat_result(st_mode=33206, st_ino=5066549581564298, st_dev=573948050, st_nlink=1, st_uid=0, st_gid=0, st_size=413, st_atime=1523480272, st_mtime=1539787740, st_ctime=1523480272)

For more information about what os.stat_result contains, refer to the documentation. For the modification time you want fname.stat().st_mtime:

>>> import datetime
>>> mtime = datetime.datetime.fromtimestamp(fname.stat().st_mtime, tz=datetime.timezone.utc)
>>> print(mtime)
datetime.datetime(2018, 10, 17, 10, 49, 0, 249980)

If you want the creation time on Windows, or the most recent metadata change on Unix, you would use fname.stat().st_ctime:

>>> ctime = datetime.datetime.fromtimestamp(fname.stat().st_ctime, tz=datetime.timezone.utc)
>>> print(ctime)
datetime.datetime(2018, 4, 11, 16, 57, 52, 151953)

This article has more helpful info and examples for the pathlib module.

Best way to resolve file path too long exception

As the cause of the error is obvious, here's some information that should help you solve the problem:

See this MS article about Naming Files, Paths, and Namespaces

Here's a quote from the link:

Maximum Path Length Limitation In the Windows API (with some exceptions discussed in the following paragraphs), the maximum length
for a path is MAX_PATH, which is defined as 260 characters. A local
path is structured in the following order: drive letter, colon,
backslash, name components separated by backslashes, and a terminating
null character. For example, the maximum path on drive D is "D:\some
256-character path string<NUL>" where "<NUL>" represents the invisible
terminating null character for the current system codepage. (The
characters < > are used here for visual clarity and cannot be part of
a valid path string.)

And a few workarounds (taken from the comments):

There are ways to solve the various problems. The basic idea of the solutions listed below is always the same: Reduce the path-length in order to have path-length + name-length < MAX_PATH. You may:

• Share a subfolder
• Use the commandline to assign a drive letter by means of SUBST
• Use AddConnection under VB to assign a drive letter to a path

---

Related Topics