How to Use Valgrind with Python C++ Extensions

How to use valgrind with python?

I found the answer here.

Python also needs to be compiled in debug mode, i.e.

./configure --prefix=/home/dejan/workspace/python --without-pymalloc --with-pydebug --with-valgrind

In addition, numpy has a suppresion file that gets rid of the extra valgrind warnings.

Valgrind error and memory leaks with Python/C API

I initially suggested this as a duplicate. I don't think that's the case any more, but it still gives useful advice on removing false positives from the Valgrind output for Python programs.

Beyond that there are a number of specific issues with your program:

• No error checking - the Python C API typically uses a NULL return value to indicate an error. There are clearly various ways of writing error checking code, but I'd be tempted to go for something like

  IA::IA() :
  pModule(NULL), pDict(NULL), pFunc(NULL), pName(NULL) // initially null initialize everything
  {
    setenv("PYTHONPATH",".",1);
    Py_Initialize();
  
    // I don't think you actually use this line, so maybe remove it
    if (PyRun_SimpleString("import sys") == -1) goto error;
  
    pName = PyBytes_FromString((char*)"Test");
    if (!pName) goto error;
    pModule = PyImport_Import(pName);
    if (!pModule) goto error;
    pDict = PyModule_GetDict(pModule);
    if (!pDict) goto error;
    pFunc = PyDict_GetItemString(pDict, "push_f");
    if (!pFunc) goto error;
  
    return; // completed OK
  
    error:
    Py_XDECREF(pName); // XDECREF is OK with NULL...
    Py_XDECREF(pModule);
    Py_XDECREF(pDict);
    Py_XDECREF(pFunc);
    PyErr_Print();
    throw std::runtime_error(""); // ??? - possibly get and use the error string
              // see https://stackoverflow.com/a/1418703/4657412
  }
  

  I'm aware people are suspicious of goto but in this case it's a reasonably clean way of jumping to an error handling block. You can structure it differently if you like.

• The destructor doesn't decref pFunc, which looks like a memory leak.

• IA::LaunchIA similarly is lacking working error checking.

• IA::LaunchIA never decrefs toSend, pResult, TlistMob, TlistPlayer, pDPosIA. Some of these are related to the incompleteness of the code you show, but if they aren't decrefed then you're leaking memory.

• You call PyErr_Print() without checking for an error. The documentation says:

  Call this function only when the error indicator is set. (Otherwise it will cause a fatal error!)

  I suspect that this might be your biggest problem.

---

Those are all the issues I can see. Without a minimal complete example it's impossible to actually check what you're seeing. Given you're using C++ you might be well advised to use/make a decent, object oriented wrapper for PyObject* to avoid having to worry about reference counting yourself - Boost Python has one.

Is it normal that running python under valgrind shows many errors with memory?

You could try using the suppression file that comes with the python source

Reading the Python Valgrind README is a good idea too!

Find memory leaks when extending python with C

if you using a linux environment we can easily find the memory leaks by using debugger named valgrind.
To get the valgrind first you have to install it from the internet by using command

sudo apt-get valgrind

after the installation is completed compile your c code using this debugger you can easily find the memory leaks. This debugger shows what is the reason for the memory leak and also specifies the line at which the leak has been occured.

Python memory leaks?

There's a whole README.valgrind in the Python sources that explains the various caveats trying to use Valgrind with Python:

http://svn.python.org/projects/python/trunk/Misc/README.valgrind

Python uses its own small-object allocation scheme on top of malloc,
called PyMalloc.

Valgrind may show some unexpected results when PyMalloc is used.
Starting with Python 2.3, PyMalloc is used by default.  You can disable
PyMalloc when configuring python by adding the --without-pymalloc option.
If you disable PyMalloc, most of the information in this document and
the supplied suppressions file will not be useful.  As discussed above,
disabling PyMalloc can catch more problems.

If you use valgrind on a default build of Python,  you will see
many errors like:

        ==6399== Use of uninitialised value of size 4
        ==6399== at 0x4A9BDE7E: PyObject_Free (obmalloc.c:711)
        ==6399== by 0x4A9B8198: dictresize (dictobject.c:477)

These are expected and not a problem. 

---

Related Topics