How to Ignore Hidden Files Using Os.Listdir()

How to ignore hidden files using os.listdir()?

You can write one yourself:

import os

def listdir_nohidden(path):
    for f in os.listdir(path):
        if not f.startswith('.'):
            yield f

Or you can use a glob:

import glob
import os

def listdir_nohidden(path):
    return glob.glob(os.path.join(path, '*'))

Either of these will ignore all filenames beginning with '.'.

Ignoring specific files with `os.listdir`

You can use an if-statement here by checking that the file name is not equal (!=) to "DS_Store":

class MySentences(object):
    def __init__(self, dirname):
        self.dirname = dirname

    def __iter__(self):
        for fname in os.listdir(self.dirname):
            if fname != '.DS_Store':
                for line in open(os.path.join(self.dirname, fname)):
                    yield line.split()

If the filename does equal this string, then that loop is simply passed-over without the nested for statement being touched.

How to ignore hidden files in python functions?

Assuming you want to ignore all files that start with .:

import os

root = "/Users/Siddhartha/Desktop/py scripts"
for item in os.listdir(root):
    if not item.startswith('.') and os.path.isfile(os.path.join(root, item)):
        print item

How to ignore hidden files when using os.stat() results in Python?

Your desired outcome is impossible. The most recent modification time of all non-hidden files doesn't necessarily correspond to the virtual "last modified time of a directory ignoring hidden files". The problem is that directories are modified when files are moved in and out of them, but the file timestamps aren't changed (the file was moved, but not modified). So your proposed solution is at best a heuristic; you can hope it's correct, but there is no way to be sure.

In any event, no, there is no built-in that provides this heuristic. The concept of hidden vs. non-hidden files is OS and file system dependent, and Python provides no built-in API that cares about the distinction. If you want to make a "last_modified_guess" function, you'll have to write it yourself (I recommend basing it on os.scandir for efficiency).

Something as simple as:

last_time = max(entry.stat().st_mtime for entry in os.scandir(somedir) if not entry.name.startswith('.'))

would get you the most recent last modified time (in seconds since the epoch) of your non-hidden directory entries.

Update: On further reflection, the glob module does include a concept of . prefix meaning "hidden", so you could use glob.glob/glob.iglob of os.path.join(somedir, '*') to have it filter out the "hidden" files for you. That said, by doing so, you give up some of the potential benefits of os.scandir (free or cached stat results, free type checks, etc.), so if all you need is "hidden" filtering, a simple .startswith('.') check is not worth giving that up.

python os.listdir() shows protected files

You can use with the pywin32 module, and manually test for FILE_ATTRIBUTE_HIDDEN or any number of attributes

FILE_ATTRIBUTE_ARCHIVE              = 32
FILE_ATTRIBUTE_ATOMIC_WRITE         = 512
FILE_ATTRIBUTE_COMPRESSED           = 2048
FILE_ATTRIBUTE_DEVICE               = 64
FILE_ATTRIBUTE_DIRECTORY            = 16
FILE_ATTRIBUTE_ENCRYPTED            = 16384
FILE_ATTRIBUTE_HIDDEN               = 2
FILE_ATTRIBUTE_NORMAL               = 128
FILE_ATTRIBUTE_NOT_CONTENT_INDEXED  = 8192
FILE_ATTRIBUTE_OFFLINE              = 4096
FILE_ATTRIBUTE_READONLY             = 1
FILE_ATTRIBUTE_REPARSE_POINT        = 1024
FILE_ATTRIBUTE_SPARSE_FILE          = 512
FILE_ATTRIBUTE_SYSTEM               = 4
FILE_ATTRIBUTE_TEMPORARY            = 256
FILE_ATTRIBUTE_VIRTUAL              = 65536
FILE_ATTRIBUTE_XACTION_WRITE        = 1024

like so:

import win32api, win32con

#test for a certain type of attribute
attribute = win32api.GetFileAttributes(filepath)
#The file attributes are bitflags, so you want to see if a given flag is 1.
# (AKA if it can fit inside the binary number or not) 
# 38 in binary is  100110 which means that 2, 4 and 32 are 'enabled', so we're checking for that
## Thanks to Nneoneo
if attribute & (win32con.FILE_ATTRIBUTE_HIDDEN | win32con.FILE_ATTRIBUTE_SYSTEM):
  raise Exception("hidden file") #or whatever

#or alter them
win32api.SetFileAttributes(filepath, win32con.FILE_ATTRIBUTE_NORMAL) #or FILE_ATTRIBUTE_HIDDEN

After you alter a file, take a look in the folder, it won't be hidden anymore.

Found this information here and here: Checking file attributes in python

---

Alternatively, you can try to use the os.stat function, whose docs here and then use the stat module to further understand what you're looking at.

Found these relevant questions. (python) meaning of st_mode and How can I get a file's permission mask?

Exclude hidden files in Python

In your original code, there are multiple boolean args creating different paths. Your extension == '.' path was the only one where check_attributes was being called from what I can tell, so that might have been the issue. I decided to take a crack at rewriting it. The way I rewrote it has 2 phases: 1. get the files, either recursively or not then 2. filter the files with the args provided. Here's what I came up with:

import argparse
import os
import win32api
import win32con

def count_files(args):
    files = []
    
    # Get the files differently based on whether recursive or not.
    if args.recursive:
        # Note here I changed how you're iterating. os.walk returns a list of tuples, so
        # you can unpack the tuple in your for. current_dir is the current dir it's in
        # while walking and found_files are all the files in that dir
        for current_dir, dirs, found_files in os.walk(top=args.path, topdown=True):
            files += [os.path.join(current_dir, found_file) for found_file in found_files]
    else
        # Note the os.path.join with the dir each file is in. It's important to store the
        # absolute path of each file.
        files += [os.path.join(args.path, found_file) for found_file in os.listdir(args.path)
                  if os.path.isfile(os.path.join(args.path, found_file))]
       
    filtered_files = []
    for found_file in files:
        print(found_file)
        if not args.hidden and (win32api.GetFileAttributes(found_file) & win32con.FILE_ATTRIBUTE_HIDDEN):
            continue  # hidden == False and file has hidden attribute, go to next one
        
        if args.extension and not found_file.endswith(args.extension):
            continue  # File doesn't end in provided extension
            
        filtered_files.append(found_file)
    
    print(f'Length: {len(filtered_files)}')
    return len(filtered_files)
    

if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='Process some integers.')
    # Note that I took advantage of some other argparse features here like
    # required vs optional arguments and boolean types
    parser.add_argument('path')
    parser.add_argument('--recursive', action='store_true', default=False)
    parser.add_argument('--hidden', action='store_true', default=False)
    parser.add_argument('--extension', type=str)

    args = parser.parse_args()
    count_files(args)
    

---

Related Topics