File Read Using "Open()" VS "With Open()"

File read using open() vs with open()

Using with statement is not for performance gain, I do not think there are any performance gains or loss associated with using with statement, as long as, you perform the same cleanup activity that using with statement would perform automatically.

When you use with statement with open function, you do not need to close the file at the end, because with would automatically close it for you.

Also, with statement is not just for openning files, with is used in conjuction with context managers. Basically, if you have an object that you want to make sure it is cleaned once you are done with it or some kind of errors occur, you can define it as a context manager and with statement will call its __enter__() and __exit__() methods on entry to and exit from the with block. According to PEP 0343 -

This PEP adds a new statement "with" to the Python language to make it possible to factor out standard uses of try/finally statements.

In this PEP, context managers provide __enter__() and __exit__() methods that are invoked on entry to and exit from the body of the with statement.

Also, performance testing of using with and not using it -

In [14]: def foo():
   ....:     f = open('a.txt','r')
   ....:     for l in f:
   ....:         pass
   ....:     f.close()
   ....:

In [15]: def foo1():
   ....:     with open('a.txt','r') as f:
   ....:         for l in f:
   ....:             pass
   ....:

In [17]: %timeit foo()
The slowest run took 41.91 times longer than the fastest. This could mean that an intermediate result is being cached
10000 loops, best of 3: 186 µs per loop

In [18]: %timeit foo1()
The slowest run took 206.14 times longer than the fastest. This could mean that an intermediate result is being cached
10000 loops, best of 3: 179 µs per loop

In [19]: %timeit foo()
The slowest run took 202.51 times longer than the fastest. This could mean that an intermediate result is being cached
10000 loops, best of 3: 180 µs per loop

In [20]: %timeit foo1()
10000 loops, best of 3: 193 µs per loop

In [21]: %timeit foo1()
10000 loops, best of 3: 194 µs per loop

When should I ever use file.read() or file.readlines()?

The short answer to your question is that each of these three methods of reading bits of a file have different use cases. As noted above, f.read() reads the file as an individual string, and so allows relatively easy file-wide manipulations, such as a file-wide regex search or substitution.

f.readline() reads a single line of the file, allowing the user to parse a single line without necessarily reading the entire file. Using f.readline() also allows easier application of logic in reading the file than a complete line by line iteration, such as when a file changes format partway through.

Using the syntax for line in f: allows the user to iterate over the file line by line as noted in the question.

(As noted in the other answer, this documentation is a very good read):

https://docs.python.org/3/tutorial/inputoutput.html#methods-of-file-objects

Note:
It was previously claimed that f.readline() could be used to skip a line during a for loop iteration. However, this doesn't work in Python 2.7, and is perhaps a questionable practice, so this claim has been removed.

Python, difference between 'open' and 'with open'

This error was caused by a previous version of the posted script. It looked like this:

if not(os.path.exists('teams')):
    os.makedirs('mydir')

This tests for the existence of the directory teams but tries to create a new directory mydir.

Suggested solution: use variable names for everything, don't hardwire strings for paths:

path = 'mydir'

if not(os.path.exists(path)):
    os.makedirs(path)

And yes, both #1 and #2 do essentially the same. But the with statement also closes the file in case of an exception during writing.

How to open a file using the open with statement

Python allows putting multiple open() statements in a single with. You comma-separate them. Your code would then be:

def filter(txt, oldfile, newfile):
    '''\
    Read a list of names from a file line by line into an output file.
    If a line begins with a particular name, insert a string of text
    after the name before appending the line to the output file.
    '''

    with open(newfile, 'w') as outfile, open(oldfile, 'r', encoding='utf-8') as infile:
        for line in infile:
            if line.startswith(txt):
                line = line[0:len(txt)] + ' - Truly a great person!\n'
            outfile.write(line)

# input the name you want to check against
text = input('Please enter the name of a great person: ')    
letsgo = filter(text,'Spanish', 'Spanish2')

And no, you don't gain anything by putting an explicit return at the end of your function. You can use return to exit early, but you had it at the end, and the function will exit without it. (Of course with functions that return a value, you use the return to specify the value to return.)

Using multiple open() items with with was not supported in Python 2.5 when the with statement was introduced, or in Python 2.6, but it is supported in Python 2.7 and Python 3.1 or newer.

http://docs.python.org/reference/compound_stmts.html#the-with-statement
http://docs.python.org/release/3.1/reference/compound_stmts.html#the-with-statement

If you are writing code that must run in Python 2.5, 2.6 or 3.0, nest the with statements as the other answers suggested or use contextlib.nested.

io.StringIO vs open() in Python 3

The difference is: open takes a file name (and some other arguments like mode or encoding), io.StringIO takes a plain string and both return file-like objects.

Hence:

• Use open to read files ;
• Use StringIO when you need a file-like object and you want to pass the content of a string.

An example with StringIO:

import csv
import io

reader = csv.reader(io.StringIO("a,b,c\n1,2,3"))
print ([r for r in reader])
# output [['a', 'b', 'c'], ['1', '2', '3']]

It's very useful because you can use a string where a file was expected.

In the usual case, with a csv file on your disk, you would write something like:

with open(<path/to/file.csv>, ...) as f:
    reader = csv.reader(f, ...)

Difference between modes a, a+, w, w+, and r+ in built-in open function?

The opening modes are exactly the same as those for the C standard library function fopen().

The BSD fopen manpage defines them as follows:

 The argument mode points to a string beginning with one of the following
 sequences (Additional characters may follow these sequences.):

 ``r''   Open text file for reading.  The stream is positioned at the
         beginning of the file.

 ``r+''  Open for reading and writing.  The stream is positioned at the
         beginning of the file.

 ``w''   Truncate file to zero length or create text file for writing.
         The stream is positioned at the beginning of the file.

 ``w+''  Open for reading and writing.  The file is created if it does not
         exist, otherwise it is truncated.  The stream is positioned at
         the beginning of the file.

 ``a''   Open for writing.  The file is created if it does not exist.  The
         stream is positioned at the end of the file.  Subsequent writes
         to the file will always end up at the then current end of file,
         irrespective of any intervening fseek(3) or similar.

 ``a+''  Open for reading and writing.  The file is created if it does not
         exist.  The stream is positioned at the end of the file.  Subse-
         quent writes to the file will always end up at the then current
         end of file, irrespective of any intervening fseek(3) or similar.

If you're opening a file using the 'with' statement, do you still need to close the file object?

The answer to your immediate question is "No". The with block ensures that the file will be closed when control leaves the block, for whatever reason that happens, including exceptions (well, excluding someone yanking the power cord to your computer and some other rare events).

So it's good practice to use a with block.

Now arguably, having opened a file only for reading and then failing to close it is not that much of a problem. When garbage collection comes around (whenever that may be), that file will be closed, too, if there are no references to it anymore; at the latest that will happen when your program exits. In fact, several code samples in the official docs neglect closing a file that has been opened only for read access. When writing a file or when using the "read plus" mode like in your example, you definitely need to close the file. There are many questions her on SO dealing with incomplete/corrupted files because of a failure to close them properly.

---

Related Topics