How to Open a File Using the Open with Statement

How to open a file using the open with statement

Python allows putting multiple open() statements in a single with. You comma-separate them. Your code would then be:

def filter(txt, oldfile, newfile):
    '''\
    Read a list of names from a file line by line into an output file.
    If a line begins with a particular name, insert a string of text
    after the name before appending the line to the output file.
    '''

    with open(newfile, 'w') as outfile, open(oldfile, 'r', encoding='utf-8') as infile:
        for line in infile:
            if line.startswith(txt):
                line = line[0:len(txt)] + ' - Truly a great person!\n'
            outfile.write(line)

# input the name you want to check against
text = input('Please enter the name of a great person: ')    
letsgo = filter(text,'Spanish', 'Spanish2')

And no, you don't gain anything by putting an explicit return at the end of your function. You can use return to exit early, but you had it at the end, and the function will exit without it. (Of course with functions that return a value, you use the return to specify the value to return.)

Using multiple open() items with with was not supported in Python 2.5 when the with statement was introduced, or in Python 2.6, but it is supported in Python 2.7 and Python 3.1 or newer.

http://docs.python.org/reference/compound_stmts.html#the-with-statement
http://docs.python.org/release/3.1/reference/compound_stmts.html#the-with-statement

If you are writing code that must run in Python 2.5, 2.6 or 3.0, nest the with statements as the other answers suggested or use contextlib.nested.

If you're opening a file using the 'with' statement, do you still need to close the file object?

The answer to your immediate question is "No". The with block ensures that the file will be closed when control leaves the block, for whatever reason that happens, including exceptions (well, excluding someone yanking the power cord to your computer and some other rare events).

So it's good practice to use a with block.

Now arguably, having opened a file only for reading and then failing to close it is not that much of a problem. When garbage collection comes around (whenever that may be), that file will be closed, too, if there are no references to it anymore; at the latest that will happen when your program exits. In fact, several code samples in the official docs neglect closing a file that has been opened only for read access. When writing a file or when using the "read plus" mode like in your example, you definitely need to close the file. There are many questions her on SO dealing with incomplete/corrupted files because of a failure to close them properly.

Python open or create file using with clause

You can open the file in a+ mode (append mode with the permission to read) instead, and do a file seek to position 0 so that it creates the file if it doesn't already exist, but also allows reading the file from the beginning:

with open('ReservationsManagerApp/logs/'+data['booking']+'.json', 'a+') as outfile:
    outfile.seek(0)

How can I open multiple files using with open in Python?

As of Python 2.7 (or 3.1 respectively) you can write

with open('a', 'w') as a, open('b', 'w') as b:
    do_something()

(Historical note: In earlier versions of Python, you can sometimes use
contextlib.nested() to nest context managers. This won't work as expected for opening multiples files, though -- see the linked documentation for details.)

---

In the rare case that you want to open a variable number of files all at the same time, you can use contextlib.ExitStack, starting from Python version 3.3:

with ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # Do something with "files"

Note that more commonly you want to process files sequentially rather than opening all of them at the same time, in particular if you have a variable number of files:

for fname in filenames:
    with open(fname) as f:
        # Process f

Using with open() as file method, how to write more than once?

The w flag means "open for writing and truncate the file"; you'd probably want to open the file with the a flag which means "open the file for appending".

Also, it seems that you're using Python 2. You shouldn't be using the b flag, except in case when you're writing binary as opposed to plain text content. In Python 3 your code would produce an error.

Thus:

with open('somefile.txt', 'a') as the_file:
    the_file.write("durin's day\n")

with open('somefile.txt', 'a') as the_file:
    the_file.write("legolas\n")

---

As for the input not showing in the file using the filehandle = open('file', 'w'), it is because the file output is buffered - only a bigger chunk is written at a time. To ensure that the file is flushed at the end of a cell, you can use filehandle.flush() as the last statement.

File read using open() vs with open()

Using with statement is not for performance gain, I do not think there are any performance gains or loss associated with using with statement, as long as, you perform the same cleanup activity that using with statement would perform automatically.

When you use with statement with open function, you do not need to close the file at the end, because with would automatically close it for you.

Also, with statement is not just for openning files, with is used in conjuction with context managers. Basically, if you have an object that you want to make sure it is cleaned once you are done with it or some kind of errors occur, you can define it as a context manager and with statement will call its __enter__() and __exit__() methods on entry to and exit from the with block. According to PEP 0343 -

This PEP adds a new statement "with" to the Python language to make it possible to factor out standard uses of try/finally statements.

In this PEP, context managers provide __enter__() and __exit__() methods that are invoked on entry to and exit from the body of the with statement.

Also, performance testing of using with and not using it -

In [14]: def foo():
   ....:     f = open('a.txt','r')
   ....:     for l in f:
   ....:         pass
   ....:     f.close()
   ....:

In [15]: def foo1():
   ....:     with open('a.txt','r') as f:
   ....:         for l in f:
   ....:             pass
   ....:

In [17]: %timeit foo()
The slowest run took 41.91 times longer than the fastest. This could mean that an intermediate result is being cached
10000 loops, best of 3: 186 µs per loop

In [18]: %timeit foo1()
The slowest run took 206.14 times longer than the fastest. This could mean that an intermediate result is being cached
10000 loops, best of 3: 179 µs per loop

In [19]: %timeit foo()
The slowest run took 202.51 times longer than the fastest. This could mean that an intermediate result is being cached
10000 loops, best of 3: 180 µs per loop

In [20]: %timeit foo1()
10000 loops, best of 3: 193 µs per loop

In [21]: %timeit foo1()
10000 loops, best of 3: 194 µs per loop

Open statement according to a file extension

why not:

with (gzip.open if ext==".gz" else open)(file_name, 'rt', encoding='utf-8') as f:

the first argument of with is a ternary expression, where you decide which function to use depending on the extension. I used 'rt' in both cases, it's default for standard open. That method has the advantage to avoid copy/paste and to be able to use context manager.

Maybe some generic function could be created with an helper function:

def myopen(file_name)
  return (gzip.open if os.path.splitext(file_name)[1]==".gz" else open)(file_name, 'rt', encoding='utf-8')

use like:

with myopen(file_name):

One context manager instead of two for open file?

You can put the conditional statement in the with line:

with gzip.open(file, 'rb') if file[-3:] == '.gz' else open(file) as f:
   processFile(f)

In python (2.7) when using 'with open', do I need a 'finally close' block in case of error thrown (see example)

The with open(filepath, mode) as a_file statement keeps the file open only inside of that segment - when it "exits" the segment there is no need for a_file.close(), regardless of where an exception is thrown - in open() or in yaml.load(...).

---

Related Topics