Is Explicitly Closing Files Important

Why should I close files in Python?

For the most part, not closing files is a bad idea, for the following reasons:

1. It puts your program in the garbage collectors hands - though the file in theory will be auto closed, it may not be closed. Python 3 and Cpython generally do a pretty good job at garbage collecting, but not always, and other variants generally suck at it.

2. It can slow down your program. Too many things open, and thus more used space in the RAM, will impact performance.

3. For the most part, many changes to files in python do not go into effect until after the file is closed, so if your script edits, leaves open, and reads a file, it won't see the edits.

4. You could, theoretically, run in to limits of how many files you can have open.

5. As @sai stated below, Windows treats open files as locked, so legit things like AV scanners or other python scripts can't read the file.

6. It is sloppy programming (then again, I'm not exactly the best at remembering to close files myself!)

Hope this helps!

Explicitly closing files in python

Instead of doing this

indata = open(from_file).read()

You should have tried using the with keyword

with open(from_file) as f: indata = f.read()

In the former, you still have the file descriptor with you, even though you do not have any references left for it and there is no guarantee when the file will be closed. In the second approach, the file will be closed as soon as you are done with the statement execution.

Is it necessary to close files after reading (only) in any programming language?

In general, you should always close a file after you are done using it.

Reason number 1: There are not unlimited available File Descriptors
(or in windows, the conceptually similar HANDLES).
Every time you access a file ressource, even for reading, you are reducing the number of handles (or FD's) available to other processes.
every time you close a handle, you release it and makes it available for other processes.

Now consider the consequences of a loop that opens a file, reads it, but doesn't close it...

http://en.wikipedia.org/wiki/File_descriptor

https://msdn.microsoft.com/en-us/library/windows/desktop/aa364225%28v=vs.85%29.aspx

Reason number 2: If you are doing anything else than reading a file, there are problems with race conditions, if multiple processes or threads accesses the same file..
To avoid this, you may find file locks in place.
http://en.wikipedia.org/wiki/File_locking

if you are reading a file, and not closing it afterward, other applications, that could try to obtain a file lock are denied access.

oh - and the file can't be deleted by anyone that doesn't have rights to kill your process..

Reason number 3: There is absolutely no reason to leave a file unclosed. In any language, which is why Python helps the lazy programmers, and automatically closes a handle that drops out of scope, in case the programmer forgot.

Is file closing necessary in this situation?

Use

with open(pathf, "r") as r:
    fdata = r.read().splitlines()
# as soon as you leave the with-scope, the file is autoclosed, even if exceptions happen.

Its not only about auto-closing, but also about correct closing in case of exceptions.

Doku: methods of file objects

It is good practice to use the with keyword when dealing with file
objects. The advantage is that the file is properly closed after its
suite finishes, even if an exception is raised at some point. Using
with is also much shorter than writing equivalent try-finally blocks:

If you’re not using the with keyword, then you should call f.close()
to close the file and immediately free up any system resources used by
it.

If you don’t explicitly close a file, Python’s garbage collector
will eventually destroy the object and close the open file for you,
but the file may stay open for a while. Another risk is that different
Python implementations will do this clean-up at different times.

If you're opening a file using the 'with' statement, do you still need to close the file object?

The answer to your immediate question is "No". The with block ensures that the file will be closed when control leaves the block, for whatever reason that happens, including exceptions (well, excluding someone yanking the power cord to your computer and some other rare events).

So it's good practice to use a with block.

Now arguably, having opened a file only for reading and then failing to close it is not that much of a problem. When garbage collection comes around (whenever that may be), that file will be closed, too, if there are no references to it anymore; at the latest that will happen when your program exits. In fact, several code samples in the official docs neglect closing a file that has been opened only for read access. When writing a file or when using the "read plus" mode like in your example, you definitely need to close the file. There are many questions her on SO dealing with incomplete/corrupted files because of a failure to close them properly.

Why should I close file

By not closing files you are possibly wasting system resources. Also, other processes that want to access the file after your code was executed might not be able to do so due to still being opened by your code.

Is there a need to close files that have no reference to them?

You asked about the "basic concepts", so let's take it from the top: When you open a file, your program gains access to a system resource, that is, to something outside the program's own memory space. This is basically a bit of magic provided by the operating system (a system call, in Unix terminology). Hidden inside the file object is a reference to a "file descriptor", the actual OS resource associated with the open file. Closing the file tells the system to release this resource.

As an OS resource, the number of files a process can keep open is limited: Long ago the per-process limit was about 20 on Unix. Right now my OS X box imposes a limit of 256 open files (though this is an imposed limit, and can be raised). Other systems might set limits of a few thousand, or in the tens of thousands (per user, not per process in this case). When your program ends, all resources are automatically released. So if your program opens a few files, does something with them and exits, you can be sloppy and you'll never know the difference. But if your program will be opening thousands of files, you'll do well to release open files to avoid exceeding OS limits.

There's another benefit to closing files before your process exits: If you opened a file for writing, closing it will first "flush its output buffer". This means that i/o libraries optimize disk use by collecting ("buffering") what you write out, and saving it to disk in batches. If you write text to a file and immediately try to reopen and read it without first closing the output handle, you'll find that not everything has been written out. Also, if your program is closed too abruptly (with a signal, or occasionally even through normal exit), the output might never be flushed.

There's already plenty of other answers on how to release files, so here's just a brief list of the approaches:

1. Explicitly with close(). (Note for python newbies: Don't forget the parens! My students like to write in_file.close, which does nothing.)

2. Recommended: Implicitly, by opening files with the with statement. The close() method will be called when the end of the with block is reached, even in the event of abnormal termination (from an exception).

   with open("data.txt") as in_file:
       data = in_file.read()
   

3. Implicitly by the reference manager or garbage collector, if your python engine implements it. This is not recommended since it's not entirely portable; see the other answers for details. That's why the with statement was added to python.

4. Implicitly, when your program ends. If a file is open for output, this may run a risk of the program exiting before everything has been flushed to disk.

No need to close the file if it is opened with 'print' in Python?

Yes, you still need to close the file. There is no difference with print.

Closing the file will flush the data to disk and free the file handle.

In CPython, the system will do this for you when the reference count for f drops to zero. In PyPy, IronPython, and Jython, you would need to wait for the garbage collector to run (for automatic file closing). Rather that adopt the fragile practice of relying on automatic closing by the memory manager, the preferred practice is for you to control the closing of the file.

Since explicit closing of files is a best practice, Python has provided a context manager for file objects that makes it very easy:

with open(path, 'w') as f:
    print >> f, string

This will close your file when you leave the body of the with-statement.

---

Related Topics