How to Search Sub-Folders Using Glob.Glob Module

How can I search sub-folders using glob.glob module?

In Python 3.5 and newer use the new recursive **/ functionality:

configfiles = glob.glob('C:/Users/sam/Desktop/file1/**/*.txt', recursive=True)

When recursive is set, ** followed by a path separator matches 0 or more subdirectories.

In earlier Python versions, glob.glob() cannot list files in subdirectories recursively.

In that case I'd use os.walk() combined with fnmatch.filter() instead:

import os
import fnmatch

path = 'C:/Users/sam/Desktop/file1'

configfiles = [os.path.join(dirpath, f)
    for dirpath, dirnames, files in os.walk(path)
    for f in fnmatch.filter(files, '*.txt')]

This'll walk your directories recursively and return all absolute pathnames to matching .txt files. In this specific case the fnmatch.filter() may be overkill, you could also use a .endswith() test:

import os

path = 'C:/Users/sam/Desktop/file1'

configfiles = [os.path.join(dirpath, f)
    for dirpath, dirnames, files in os.walk(path)
    for f in files if f.endswith('.txt')]

How to access files in multiple sub-directories using glob

Instead of specifying .jpg within the path itself, specify it in glob() function. So, you can try this:

import glob 
file = '/content/drive/My Drive/Jacob_Images/Substrate_Training/B_Substrate/Originals'
for f in glob.glob(file + "/*.jpg"):
    print(f)

Or you can use this code below borrowing code from this post.

for i in glob.glob('/home/studio-lab-user/sagemaker-studiolab-notebooks/Misc/tsv/**/*.csv', recursive=True):
    print(i)

Your code also works. I am certain the issue is with the path you are providing.

Using your way:

Using my way:

Listing files in a sub folder using Glob

Without any further information it's hard to say what the issue is. I tested your syntax, it works fine for me. Are you sure the extension is .html not .htm in your /users/name/Desktop/folder/subfolder/ directory?

Also, to further troubleshoot you can check what python can see in you directory by running:

os.listdir('/users/name/Desktop/folder/subfolder/')

This should get you started.

How to use glob() to find files recursively?

pathlib.Path.rglob

Use pathlib.Path.rglob from the the pathlib module, which was introduced in Python 3.5.

from pathlib import Path

for path in Path('src').rglob('*.c'):
    print(path.name)

If you don't want to use pathlib, use can use glob.glob('**/*.c'), but don't forget to pass in the recursive keyword parameter and it will use inordinate amount of time on large directories.

For cases where matching files beginning with a dot (.); like files in the current directory or hidden files on Unix based system, use the os.walk solution below.

os.walk

For older Python versions, use os.walk to recursively walk a directory and fnmatch.filter to match against a simple expression:

import fnmatch
import os

matches = []
for root, dirnames, filenames in os.walk('src'):
    for filename in fnmatch.filter(filenames, '*.c'):
        matches.append(os.path.join(root, filename))

Getting a list of all subdirectories in the current directory

Do you mean immediate subdirectories, or every directory right down the tree?

Either way, you could use os.walk to do this:

os.walk(directory)

will yield a tuple for each subdirectory. Ths first entry in the 3-tuple is a directory name, so

[x[0] for x in os.walk(directory)]

should give you all of the subdirectories, recursively.

Note that the second entry in the tuple is the list of child directories of the entry in the first position, so you could use this instead, but it's not likely to save you much.

However, you could use it just to give you the immediate child directories:

next(os.walk('.'))[1]

Or see the other solutions already posted, using os.listdir and os.path.isdir, including those at "How to get all of the immediate subdirectories in Python".

Find all files in a directory with extension .txt in Python

You can use glob:

import glob, os
os.chdir("/mydir")
for file in glob.glob("*.txt"):
    print(file)

or simply os.listdir:

import os
for file in os.listdir("/mydir"):
    if file.endswith(".txt"):
        print(os.path.join("/mydir", file))

or if you want to traverse directory, use os.walk:

import os
for root, dirs, files in os.walk("/mydir"):
    for file in files:
        if file.endswith(".txt"):
             print(os.path.join(root, file))

How to incorporate sub directories using import glob, os?

Here's a sketch using dictionaries and os.walk.

import os

extensions = {
    ".asm": "Assemblies",
    ".dft": "Drafts",
    ".prt": "Regular Parts",
    ".psm": "Sheet Metal Parts",
}
count = {x: 0 for x in extensions.keys()}

for dirname, dirs, files in os.walk(r"Z:\TopFolder"):
    for file in files:
        for ext in count.keys():
            if file.endswith(ext):
                count[ext] += 1

total = 0
for ext in count.keys():
    print("%s: %i" % (extensions[ext], count[ext]))
    total += count[ext]
print("Total:", total)

---

Related Topics