How to Mock an Open Used in a with Statement (Using the Mock Framework in Python)

How do I mock an open used in a with statement (using the Mock framework in Python)?

The way to do this has changed in mock 0.7.0 which finally supports mocking the python protocol methods (magic methods), particularly using the MagicMock:

http://www.voidspace.org.uk/python/mock/magicmock.html

An example of mocking open as a context manager (from the examples page in the mock documentation):

>>> open_name = '%s.open' % __name__
>>> with patch(open_name, create=True) as mock_open:
...     mock_open.return_value = MagicMock(spec=file)
...
...     with open('/some/path', 'w') as f:
...         f.write('something')
...
<mock.Mock object at 0x...>
>>> file_handle = mock_open.return_value.__enter__.return_value
>>> file_handle.write.assert_called_with('something')

Mock open() function used in a class method

TL;DR

The heart of your problem is that you should be also mocking json.dump to be able to properly test the data that is going to be written to your file. I actually had a hard time running your code until a few important adjustments were made to your test method.

Mock with builtins.open and not mymmodule.open
You are in a context manager, so you should be checking m.return_value.__enter__.write, however you are actually calling the write from json.dump which is where the write will be called. (Details below on a suggested solution)
You should also mock json.dump to simply validate it is called with your data

In short, with the issues mentioned above, the method can be re-written as:

Details about all this below

def test_save_data_to_file(self):
    with patch('builtins.open', new_callable=mock_open()) as m:
        with patch('json.dump') as m_json:
            self.mc.save_data_to_file(self.data)

            # simple assertion that your open was called 
            m.assert_called_with('/tmp/data.json', 'w')

            # assert that you called m_json with your data
            m_json.assert_called_with(self.data, m.return_value)

Detailed Explanation

To focus on the problems I see in your code, the first thing I strongly suggest doing, since open is a builtin, is to mock from builtins, furthermore, you can save yourself a line of code by making use of new_callable and as, so you can simply do this:

with patch('builtins.open', new_callable=mock_open()) as m:

The next problem that I see with your code as I had trouble running this until I actually made the following adjustment when you started looping over your calls:

m.return_value.__enter__.return_value.write.mock_calls

To dissect that, what you have to keep in mind is that your method is using a context manager. In using a context manager, the work of your write will actually be done inside your __enter__ method. So, from the return_value of your m, you want to then get the return_value of __enter__.

However, this brings us to the heart of the problem with what you are trying to test. Because of how the json.dump works when writing to the file, your mock_calls for your write after inspecting the code, will actually look like this:

<MagicMock name='open().write' id='4348414496'>
call('[')
call('{')
call('"name"')
call(': ')
call('"peter"')
call(', ')
call('"id"')
call(': ')
call('5414470')
call('}')
call(', ')
call('{')
call('"name"')
call(': ')
call('"tom"')
call(', ')
call('"id"')
call(': ')
call('5414472')
call('}')
call(', ')
call('{')
call('"name"')
call(': ')
call('"pit"')
call(', ')
call('"id"')
call(': ')
call('5414232')
call('}')
call(']')
call.__str__()

That is not going to be fun to test. So, this brings us to the next solution you can try out; Mock json.dump.

You shouldn't be testing json.dump, you should be testing calling it with the right parameters. With that being said, you can follow similar fashion with your mocking and do something like this:

with patch('json.dump') as m_json:

Now, with that, you can significantly simplify your test code, to simply validate that the method gets called with your data that you are testing with. So, with that, when you put it all together, you will have something like this:

def test_save_data_to_file(self):
    with patch('builtins.open', new_callable=mock_open()) as m:
        with patch('json.dump') as m_json:
            self.mc.save_data_to_file(self.data)

            # simple assertion that your open was called 
            m.assert_called_with('/tmp/data.json', 'w')

            # assert that you called m_json with your data
            m_json.assert_called_with(self.data, m.return_value.__enter__.return_value)

If you're interested in further refactoring to make your test method a bit cleaner, you could also set up your patching as a decorator, leaving your code cleaner inside the method:

@patch('json.dump')
@patch('builtins.open', new_callable=mock_open())
def test_save_data_to_file(self, m, m_json):
    self.mc.save_data_to_file(self.data)

    # simple assertion that your open was called
    m.assert_called_with('/tmp/data.json', 'w')

    # assert that you called m_json with your data
    m_json.assert_called_with(self.data, m.return_value.__enter__.return_value)

Inspecting is your best friend here, to see what methods are being called at what steps, to further help with the testing. Good luck.

How do I mock an open(...).write() without getting a 'No such file or directory' error?

Mock builtins.open (or module.open, module = the module name that contains WriteData) with the mock_open:

import builtins

class TestListWindowsPasswords(unittest.TestCase):
    def setUp(self):
        self._orig_pathexists = os.path.exists
        os.path.exists = MockPathExists(True)

    def test_dump(self):
        with patch('builtins.open', unittest.mock.mock_open()) as m:
            data_writer = WriteData(
                dir='/my/path/not/exists',
                name='Foo'
            )
            data_writer.dump()

        self.assertEqual(os.path.exists.received_args[0], '/my/path/not/exists')  # fixed
        m.assert_called_once_with('/my/path/not/exists/output.text', 'w+')
        handle = m()
        handle.write.assert_called_once_with('Hello, Foo!')

Python: Mocking a file for unittesting

May be you should specify read_data as binary?
Here is a working example, you can paste it to file and run with unittest:

import hashlib
from unittest import TestCase

import mock

def md5(file_name):
    hash_md5 = hashlib.md5()
    with open(file_name, "rb") as f:
        for chunk in iter(lambda: f.read(4096), b""):
            hash_md5.update(chunk)
    return hash_md5.hexdigest()

class MD5TestCase(TestCase):
    def test(self):
        with mock.patch('%s.open' % __name__, mock.mock_open(read_data=b'aaa'), create=True) as m:
            result = md5("lalala")
            self.assertEqual(result, '47bce5c74f589f4867dbd57e9ca9f808')

The answer is based on that answer: How do I mock an open used in a with statement (using the Mock framework in Python)?

Mocking with open()

The return value of mock_open (until Python 3.7.1) doesn't provide a working __iter__ method, which may make it unsuitable for testing code that iterates over an open file object.

Instead, I recommend refactoring your code to take an already opened file-like object. That is, instead of

def some_method(file_name):
    with open([file_name], 'r') as file_list:
        for line in file_list:
            # Do stuff

...

 some_method(file_name)

write it as

def some_method(file_obj):
    for line in file_obj:
        # Do stuff

...

with open(file_name, 'r') as file_obj:
    some_method(file_obj)

This turns a function that has to perform IO into a pure(r) function that simply iterates over any file-like object. To test it, you don't need to mock open or hit the file system in any way; just create a StringIO object to use as the argument:

def test_it(self):
    f = StringIO.StringIO("line1\nline2\n")
    some_method(f)

(If you still feel the need to write and test a wrapper like

def some_wrapper(file_name):
    with open(file_name, 'r') as file_obj:
        some_method(file_obj)

note that you don't need the mocked open to do anything in particular. You test some_method separately, so the only thing you need to do to test some_wrapper is verify that the return value of open is passed to some_method. open, in this case, can be a plain old mock with no special behavior.)

Python Mock - Mocking several open

Here's a quick way of getting what you want. It cheats a little bit because the two file objects in the method under test are the same object and we're just changing the return value of the read call after each read. You can use the same technique in multiple layers if you want the file objects to be different, but it will be quite messy and it may disguise the intent of the test unnecessarily.

Replace this line:


    m_file.read.return_value = 'text1'

with:


    reads = ['text1', 'text2']
    m_file.read.side_effect = lambda: reads.pop(0)

How to use mock_open() with patch.object() in test annotation

You can patch the open method in many ways. I prefer to patch the builtins.open and to pass the mocked object to the test method like this:

from unittest.mock import patch, mock_open
from mymodule import method_that_read_with_open

class TestPatch(unittest.TestCase):
    @patch('builtins.open', new_callable=mock_open, read_data='1')
    def test_open_file(self, m):
        string_read = method_that_read_with_open()
        self.assertEqual(string_read, '1')
        m.assert_called_with('filename', 'r')

Note that we are passing the mock_open function without calling it!

But because you are patching the builtin method, you can also do:

class TestPatch(unittest.TestCase):
    @patch('builtins.open', mock_open(read_data='1'))
    def test_open_file(self):
        string_read = method_that_read_with_open()
        self.assertEqual(string_read, '1')
        open.assert_called_with('filename', 'r')

This two examples are basically equivalent: in the first one we are giving to the patch method a factory function that he will invoke to create the mock object, in the second one we are using an already created object as argument.