Iterating Through Directories with Python

Iterating through directories with Python

The actual walk through the directories works as you have coded it. If you replace the contents of the inner loop with a simple print statement you can see that each file is found:

import os
rootdir = 'C:/Users/sid/Desktop/test'

for subdir, dirs, files in os.walk(rootdir):
    for file in files:
        print(os.path.join(subdir, file))

---

If you still get errors when running the above, please provide the error message.

Iterate through directories, subdirectories, to get files with specialized ext. , copy in new directory

Use os.walk() and find all files. Then use endswith() to find the files ending with .trl.

Sample code:

import os, shutil;

directory = "/home/.../test_daten";
dest_dir = "/home/.../test_korpus";

filelist = [];

for root, dirs, files in os.walk(directory):
    for file in files:
        filelist.append(os.path.join(root,file));

for trlFile in filelist:
    if trlFile.endswith(".trl"):
        shutil.copy(trlFile, dest_dir);

Python loop through directories

A more efficient way to iterate through the folders and only select the files you are looking for is below:

source_folder = '<path-to-dir>/942ba956-8967-4bec-9540-fbd97441d17f/'
files = [os.path.normpath(os.path.join(root,f)) for root,dirs,files in os.walk(source_folder) for f in files if '000000' in f and not f.endswith('.gz')]
for file in files:
    os.rename(f, f"{f}.csv")

The list comprehension stores the full path to the files you are looking for. You can change the condition inside the comprehension to anything you need. I use this code snippet a lot to find just images of certain type, or remove unwanted files from the selected files.
In the for loop, files are renamed adding the .csv extension.

How can I iterate over files in a given directory?

Python 3.6 version of the above answer, using os - assuming that you have the directory path as a str object in a variable called directory_in_str:

import os

directory = os.fsencode(directory_in_str)
    
for file in os.listdir(directory):
     filename = os.fsdecode(file)
     if filename.endswith(".asm") or filename.endswith(".py"): 
         # print(os.path.join(directory, filename))
         continue
     else:
         continue

Or recursively, using pathlib:

from pathlib import Path

pathlist = Path(directory_in_str).glob('**/*.asm')
for path in pathlist:
     # because path is object not string
     path_in_str = str(path)
     # print(path_in_str)

• Use rglob to replace glob('**/*.asm') with rglob('*.asm')
  
  • This is like calling Path.glob() with '**/' added in front of the given relative pattern:

from pathlib import Path

pathlist = Path(directory_in_str).rglob('*.asm')
for path in pathlist:
     # because path is object not string
     path_in_str = str(path)
     # print(path_in_str)

---

Original answer:

import os

for filename in os.listdir("/path/to/dir/"):
    if filename.endswith(".asm") or filename.endswith(".py"): 
         # print(os.path.join(directory, filename))
        continue
    else:
        continue

Iterate through different folders

You can use Path.iterdir to iterate through the folders. Assuming the current working directory is the one with all the datetime folders and they are the only things there, you can loop over each folder like so:

from pathlib import Path

# ...

folder_names = []
spreadsheet_contents = []

current_directory = Path.cwd()
for folder in current_directory.iterdir():
    folder_names.append(folder.name)

    spreadsheet_path = folder / "spreadsheet.xlsx"
    spreadsheet_contents.append(pd.read_excel(spreadsheet_path, ...))

Note that the folder names will still be strings. If you want to parse the actual dates and times, you can use datetime.strptime.

Iterate through folders, then subfolders and print filenames with path to text file

Use os.walk(). The following will output a list of all files within the subdirectories of "dir". The results can be manipulated to suit you needs:

import os                                                                                                             
                                                                                                                      
def list_files(dir):                                                                                                  
    r = []                                                                                                            
    subdirs = [x[0] for x in os.walk(dir)]                                                                            
    for subdir in subdirs:                                                                                            
        files = os.walk(subdir).next()[2]                                                                             
        if (len(files) > 0):                                                                                          
            for file in files:                                                                                        
                r.append(os.path.join(subdir, file))                                                                         
    return r                                                                                                          

For python 3, change next() to __next__().

How to iterate over all files in multiple folders with python

You can use the below sample snippet both #1 or #2 works:

import os
path = "."
for (root, dirs, files) in os.walk(path, topdown=True):
    for file in files:
        if file.endswith(".html"):
            print(root+"/"+file)                #1
            print(os.path.join(root+"/"+file))  #2

Loop through each subdirectory in a main directory and run code against each file using OS

I like using pure os:

import os

for fname in os.listdir(src):

    # build the path to the folder
    folder_path = os.path.join(src, fname)
    
    if os.path.isdir(folder_path):
        # we are sure this is a folder; now lets iterate it
        for file_name in os.listdir(folder_path):
            file_path = os.path.join(folder_path, file_name)
            # now you can apply any function assuming it is a file
            # or double check it if needed as `os.path.isfile(file_path)`

Note that this function just iterate over the folder given at src and one more level:

src/foo.txt  # this file is ignored
src/foo/a.txt  # this file is processed
src/foo/foo_2/b.txt  # this file is ignored; too deep.
src/foo/foo_2/foo_3/c.txt  # this file is ignored; too deep.

In case you need to go as deep as possible, you can write a recursive function and apply it to every single file, as follows:

import os

def function_over_files(path):
    if os.path.isfile(path):
        # do whatever you need with file at path
    else:
        # this is a dir: we will list all files on it and call recursively
        for fname in os.listdir(path):
            f_path = os.path.join(path, fname)

            # here is the trick: recursive call to the same function
            function_over_files(f_path)

src = "path/to/your/dir"
function_over_files(src)

This way you can apply the function to any file under path, don't care how deep it is in the folder, now:

src/foo.txt  # this file is processed; as each file under src
src/foo/a.txt  # this file is processed
src/foo/foo_2/b.txt  # this file is processed
src/foo/foo_2/foo_3/c.txt  # this file is processed

how iterate through a folder by subfolder in python

You could try the following:

In case you want to give the function the base folder, in which all the customer folders are located, and then want for each of the customer folders a list of all .ai-files (from every sublevel):

from pathlib import Path

def folder_loop(folder):
    for path in Path(folder).iterdir():
        if path.is_dir():
            yield list(path.rglob("*.ai"))

Path.rglob("*.ai") is recursively globbing the the given Path with all its subfolders for .ai-files.

To use it:

the_folder = "..."
for file_list in folder_loop(the_folder):
    print(file_list)
    # do whatever you want to do with the files

If you want to give it a folder and want one list with all the .ai files in it:

def folder_loop(folder):
    return list(Path(folder).rglob("*.ai"))

The yielded/returned lists here contain Path-objects (which are pretty handy). If you want strings instead, then you could do

       ....
            yield list(map(str, path.rglob("*.ai")))

etc.

---

Related Topics