How to Iterate Over Files in a Given Directory

How can I iterate over files in a given directory?

Python 3.6 version of the above answer, using os - assuming that you have the directory path as a str object in a variable called directory_in_str:

import os

directory = os.fsencode(directory_in_str)
    
for file in os.listdir(directory):
     filename = os.fsdecode(file)
     if filename.endswith(".asm") or filename.endswith(".py"): 
         # print(os.path.join(directory, filename))
         continue
     else:
         continue

Or recursively, using pathlib:

from pathlib import Path

pathlist = Path(directory_in_str).glob('**/*.asm')
for path in pathlist:
     # because path is object not string
     path_in_str = str(path)
     # print(path_in_str)

• Use rglob to replace glob('**/*.asm') with rglob('*.asm')
  
  • This is like calling Path.glob() with '**/' added in front of the given relative pattern:

from pathlib import Path

pathlist = Path(directory_in_str).rglob('*.asm')
for path in pathlist:
     # because path is object not string
     path_in_str = str(path)
     # print(path_in_str)

---

Original answer:

import os

for filename in os.listdir("/path/to/dir/"):
    if filename.endswith(".asm") or filename.endswith(".py"): 
         # print(os.path.join(directory, filename))
        continue
    else:
        continue

How to loop over files in directory and change path and add suffix to filename

A couple of notes first: when you use Data/data1.txt as an argument, should it really be /Data/data1.txt (with a leading slash)? Also, should the outer loop scan only for .txt files, or all files in /Data? Here's an answer, assuming /Data/data1.txt and .txt files only:

#!/bin/bash
for filename in /Data/*.txt; do
    for ((i=0; i<=3; i++)); do
        ./MyProgram.exe "$filename" "Logs/$(basename "$filename" .txt)_Log$i.txt"
    done
done

Notes:

• /Data/*.txt expands to the paths of the text files in /Data (including the /Data/ part)
• $( ... ) runs a shell command and inserts its output at that point in the command line
• basename somepath .txt outputs the base part of somepath, with .txt removed from the end (e.g. /Data/file.txt -> file)

If you needed to run MyProgram with Data/file.txt instead of /Data/file.txt, use "${filename#/}" to remove the leading slash. On the other hand, if it's really Data not /Data you want to scan, just use for filename in Data/*.txt.

How to iterate over files in specific directories?

You can't pass two separate directories to Path(). You'll need to loop over them.

for dirpath in (root_dir_A, root_dir_B):
    for file in Path(dirpath).glob('**/*.json'):
         ...

According to the documentation, Path("foo", "bar") should produce "foo/bar"; but it seems to actually use only the second path segment if it is absolute. Either way, it doesn't do what you seemed to hope it would.

How to iterate over files in directory in python with respect to size

You have to iterate over all the files. You can load all the files with respect to size, sort, then rename. Thousands of files isn't much data in the grand scheme of things.

import os
sorting_data = []
for filename in files:
    sorting_data.append((filename,os.path.getsize(filename)))
# Sort data by size
sorting_data.sort(key=lambda x: x[1], reverse=true)
# Rename files
for i in range(0, len(sorting_data)):
    name = sorting_data[i][0]
    os.rename(name, str(i))

Iterate through directories, subdirectories, to get files with specialized ext. , copy in new directory

Use os.walk() and find all files. Then use endswith() to find the files ending with .trl.


Sample code:

import os, shutil;

directory = "/home/.../test_daten";
dest_dir = "/home/.../test_korpus";

filelist = [];

for root, dirs, files in os.walk(directory):
    for file in files:
        filelist.append(os.path.join(root,file));

for trlFile in filelist:
    if trlFile.endswith(".trl"):
        shutil.copy(trlFile, dest_dir);

Iterate over files over several directories to extract data

If you need to keep track of how far you've walked into the directory structure, you can count the file path delimiter (os.sep). In your case it's / because you're on a Mac.

for path, dirs, _ in os.walk("../kepler"):      
  if path.count(os.sep) == 2:
    # path should be ../kepler/0017  
    for dir in dirs:  
      filename = dir + ".csv"
      data_files = os.listdir(path + os.sep + dir)
      for file in data_files:
        if file.endswith(".fits"):
          # Extract data
          # Write to CSV file

As far as I can tell this meets your requirements, but let me know if I've missed something.

Loop through files in a given directory

If you only want the files non-recursively in the current directory, combine what you have:

read -p 'Enter the directory path: ' directory
for file in "$directory"/*; do
  echo "$file"
done

If you want to loop recursively and you have bash 4, it's not much harder:

shopt -s globstar
for file in "$directory"/**/*; do …

But if you only have bash 3, you'd be better off using find.

find "$directory"

How to iterate over all files in multiple folders with python

You can use the below sample snippet both #1 or #2 works:

import os
path = "."
for (root, dirs, files) in os.walk(path, topdown=True):
    for file in files:
        if file.endswith(".html"):
            print(root+"/"+file)                #1
            print(os.path.join(root+"/"+file))  #2

---

Related Topics