How to Skip Blank Line While Reading CSV File Using Python

how to skip blank line while reading CSV file using python

If you want to skip all whitespace lines, you should use this test: ' '.isspace().

Since you may want to do something more complicated than just printing the non-blank lines to the console(no need to use CSV module for that), here is an example that involves a DictReader:

#!/usr/bin/env python
# Tested with Python 2.7

# I prefer this style of importing - hides the csv module
# in case you do from this_file.py import * inside of __init__.py
import csv as _csv


# Real comments are more complicated ...
def is_comment(line):
    return line.startswith('#')


# Kind of sily wrapper
def is_whitespace(line):
    return line.isspace()


def iter_filtered(in_file, *filters):
    for line in in_file:
        if not any(fltr(line) for fltr in filters):
            yield line


# A dis-advantage of this approach is that it requires storing rows in RAM
# However, the largest CSV files I worked with were all under 100 Mb
def read_and_filter_csv(csv_path, *filters):
    with open(csv_path, 'rb') as fin:
        iter_clean_lines = iter_filtered(fin, *filters)
        reader = _csv.DictReader(iter_clean_lines, delimiter=';')
        return [row for row in reader]


# Stores all processed lines in RAM
def main_v1(csv_path):
    for row in read_and_filter_csv(csv_path, is_comment, is_whitespace):
        print(row)  # Or do something else with it


# Simpler, less refactored version, does not use with
def main_v2(csv_path):
    try:
        fin = open(csv_path, 'rb')
        reader = _csv.DictReader((line for line in fin if not
                                  line.startswith('#') and not line.isspace()),
                                  delimiter=';')
        for row in reader:
            print(row)  # Or do something else with it
    finally:
        fin.close()


if __name__ == '__main__':
    csv_path = "C:\Users\BKA4ABT\Desktop\Test_Specification\RDBI.csv"
    main_v1(csv_path)
    print('\n'*3)
    main_v2(csv_path)

ignore "blank" (unfilled) row with python csv reader

You can use any to check if any column in the row contains data:

for rowdata in reader:
    # how to check here?
    if any(x.strip() for x in rowdata):
        dataSet.append(rowdata)

skip blank lines with csv.reader

The quick way to solve the problem is a second function that strips empty lines. You can use itertools.ifilter to do the job:

import itertools

def get_labels(filename):
    index = 0    
    with open(filename, 'rb') as f:
        sample = ''.join(x[0] for x in zip(itertools.ifilter(strip, f), range(4)))
        dialect = csv.Sniffer().sniff(sample)        
        f.seek(0)        
        reader = csv.reader(itertools.ifilter(strip, f), dialect)
        for row in reader:
            if 'TimeStamp (s)' not in row:
                index += 1
            else:
                return row, index + 1

You could write your own strip function instead of using filter:

def strip_lines(iterable, maxlines=None):
    for i, line in enumerate(iterable):
        if line.strip() and (maxlines is None or maxlines > i):
            yield line

def get_labels(filename):
    index = 0    
    with open(filename, 'rb') as f:
        dialect = csv.Sniffer().sniff(''.join(strip_lines(f, 4))
        f.seek(0)        
        reader = csv.reader(strip_lines(f), dialect)
        for row in reader:
            if 'TimeStamp (s)' not in row:
                index += 1
            else:
                return row, index + 1

python csv reader ignore blank row

If your csv files start with a blank line, I think you should be able to skip that line with readline() before creating the csv reader:

with open("ted.csv") as f1, open("ted2.csv") as f2, open('foo.csv', 'w') as out:
    f1.readline()
    f2.readline()
    r1, r2 = csv.reader(f1), csv.reader(f2)

Removing blank lines while creating file

The issue is the csv module. You can see that there is no way to get rid of the end of line character from the output of writerow(). After all, the csv module assume you can repeatedly write rows:

 writer.writerow(row1)
 writer.writerow(row2)
 ...

so emitting a newline on every row output is necessary to make it work. But your question is to remove the newline at end of file. So you can consider adding an extra step in between: (using Python 3 io module)

with open(tempLoc + item[0] + r"\prm.263", "w+") as f:
    g = io.StringIO()
    csv.writer(g).writerow(item[1:])
    f.write(g.getvalue().rstrip())

Above, we make a StringIO as file-like object for csv.writer() and ask the writer to write to the buffer instead of the file. After you finish writing all the rows, we read the buffer by g.getvalue() and then strip out the whitespaces (i.e., the newline) at the end, and write to the file.

If you're interested, this is what the csv module will give you for each row:

>>> import io
>>> f = io.StringIO()
>>> import csv
>>> csv.writer(f).writerow([1,2,3,4])
9
>>> f.getvalue()
'1,2,3,4\r\n'

---

Related Topics