How to Use C++ Classes with Ctypes

How to use C++ classes with ctypes?

The short story is that there is no standard binary interface for C++ in the way that there is for C. Different compilers output different binaries for the same C++ dynamic libraries, due to name mangling and different ways to handle the stack between library function calls.

So, unfortunately, there really isn't a portable way to access C++ libraries in general. But, for one compiler at a time, it's no problem.

This blog post also has a short overview of why this currently won't work. Maybe after C++0x comes out, we'll have a standard ABI for C++? Until then, you're probably not going to have any way to access C++ classes through Python's ctypes.

Python, using ctypes to create C++ class wrapper

Everything works fine... but you forgot to pass along the return value of the C function in the method MyMath.FunAdd!

def FunAdd(self, a, b):
    return self.FunMath.MyAdd(self.obj, a, b)
    ^^^^^^

Using Ctypes to call complicated c++ code

(1) Did I use extern "C" correctly?

Don't put extern ”C" around the method definitions.

The purpose of extern "C" is to tell the compiler that some function needs to be accessible as a C function—that is, called by C code, or by ctypes or some other library that loads and calls functions out of shared objects.

The only function you want to call through ctypes is printStuff, so only that function should be extern "C".

Trying to extern "C" the methods will probably harmlessly do nothing with g++ (they'll still end up named something like __ZN3foo3barEv in the export table), but you may get a warning, or even a non-running library, with a different compiler.

---

(2) How should I compile these three files into a shared library? I use g++.

You probably want to write a Makefile or use some other build system, but if you want to do it manually:

g++ -shared -o foo.so foo.cpp mainFunc.cpp

Depending on your platform, you may also need to manually specify -fPIC or -fpic.¹

If you want to do the steps separately:

g++ -c foo.cpp
g++ -c mainFunc.cpp
g++ -shared -o foo.so foo.o mainFunc.o

In this case, if a PIC flag is needed, it goes on the first two lines.

---

And now, to use it from Python:

$ python3
>>> import ctypes
>>> foo = ctypes.cdll.LoadLibrary('foo.so')
>>> foo.printStuff(10)
Printing...
15
>>> foo.printStuff("abc")
Printing...
116480373

Of course it's generally better to set the argtypes and restype instead of making ctypes guess. It guessed that the Python int 10 should be converted to the C int 10, which worked great, but it also guessed that the Python str "abc" should be converted to a C const char *, and you end up with the low 32 bits of the pointer to the string buffer being used as an int. So:

>>> foo.printStuff.argtypes = [ctypes.c_int]
>>> foo.printStuff.restype = None
>>> foo.printStuff(10)
Printing...
15
>>> foo.printStuff("abc")
ArgumentError: argument 1: <class 'TypeError'>: wrong type

And you may want to write a foo.py that wraps that up:

import ctypes
foo = ctypes.cdll.LoadLibrary('foo.so')
foo.printStuff.argtypes = [ctypes.c_int]
foo.printStuff.restype = None
printStuff = foo.printStuff

---

_{1. From quick tests, I didn't need it for x86_64 macOS, x86 macOS, or x86_64 Linux, but PowerPC Linux needs -fpic and ARM64 Linux needs -fPIC. However, I didn't actually run all of those. And, other than macOS (where I made sure to test with Homebrew gcc 8.2 and Apple Clang 9.1) I don't know which compiler version I had.}

Calling Functions from within class using CTypes

ctypes understands C linkage and a C++ library normally needs to create extern "C" wrapper functions. In this special case you can force ctypes to load the name-decorated C++ symbol for the static method, but it isn't recommended.

Here's both demonstrated:

test.cpp

#include <stdio.h>

#ifdef _WIN32
#   define API __declspec(dllexport)  // Windows requires explicit export of functions.
#else
#   define API
#endif

class API Clibrary  // exported to see name decoration, but not needed
{
public:
    Clibrary() {};
    ~Clibrary() {};
    static void prompt() { printf("hello world\n"); }
};

// wrapper function with C linkage
extern "C" API void prompt() { Clibrary::prompt(); }

List of symbols exported from the DLL:

ordinal hint RVA      name

      1    0 00001020 ??0Clibrary@@QEAA@XZ
      2    1 00001030 ??1Clibrary@@QEAA@XZ
      3    2 00001040 ??4Clibrary@@QEAAAEAV0@AEBV0@@Z
      4    3 00001050 ?prompt@Clibrary@@SAXXZ
      5    4 00001000 prompt

test.py

import ctypes

libObject = ctypes.CDLL('./test')

# Hack but works.  Decorated name is not standard and could
# be different using a different compiler.
other = getattr(libObject,'?prompt@Clibrary@@SAXXZ')
other()

libObject.prompt()  # best and portable

Output:

hello world
hello world

Loading C++ class in Python

I've found a solution starting from @Scheff comment.

On Linux (I've tried on Ubuntu 16, GCC 4.4.0 and Python 3.6), my question code works well without modifications (both on the code and on the compilation instructions).

On Windows, I modified the extern "C" block in this way:

extern "C"
{
     __declspec(dllexport) Foo* Foo_new(int n) {return new Foo(n);}
     __declspec(dllexport) void Foo_bar(Foo* foo) {foo->bar();}
     __declspec(dllexport) int Foo_foobar(Foo* foo, int n) {return foo->foobar(n);}
}

and I recompiled as before.

Thereafter I was able to import the module and the C++ class as described in the question link.

For implementing the C++ print function please see this question.

Using ctypes to call a C++ method with parameters from Python results in Don't know how to convert parameter error

You have to match the arguments exactly. Set the .argtypes and .restype of every function you use so ctypes can properly marshal the parameters to C and back again. If you do not set .restype ctypes assumes the return value is c_int (typically a signed 32-bit integer) instead of a (possibly 64-bit) pointer.

Here's a working example. I didn't flesh out every function because one should be sufficient. Tested on both 32- and 64-bit Python.

test.cpp (built with MS compiler, cl /LD /EHsc /W4 test.cpp):

#include <stdio.h>

// Needed to export functions on Windows
#ifdef _WIN32
#   define API __declspec(dllexport)
#else
#   define API
#endif

class CMatrix
{
public:
    CMatrix(int d1) : s1(d1) { m = new float[d1]; }
    ~CMatrix() { delete [] m; }
    const float* Get(int& s) { s = s1; return m; }
    void Set(int x, float f) { m[x] = f; }
    int s1;
    float *m;
};

extern "C" {
    API CMatrix* CMatrix_new(int i) {return new CMatrix(i); }
    API const float* CMatrix_Get(CMatrix* cm, int& x) { return cm->Get(x); }
    API void CMatrix_Set(CMatrix* cm, int x, float f) { cm->Set(x, f); }
    API void CMatrix_delete(CMatrix* cm) { delete cm; }
}

test.py

import ctypes as ct

# For type checking the returned pointer.
class _CMatrix(ct.c_void_p) : pass
PCMatrix = ct.POINTER(_CMatrix)

class CMatrix:

    _dll = ct.CDLL('./test')
    _dll.CMatrix_new.argtypes = ct.c_int, 
    _dll.CMatrix_new.restype = PCMatrix
    _dll.CMatrix_Get.argtypes = PCMatrix, ct.POINTER(ct.c_int)
    _dll.CMatrix_Get.restype = ct.POINTER(ct.c_float)
    _dll.CMatrix_Set.argtypes = PCMatrix, ct.c_int, ct.c_float
    _dll.CMatrix_Set.restype = None
    _dll.CMatrix_delete.argtypes = PCMatrix, 
    _dll.CMatrix_delete.restype = None

    def __init__(self, i):
        self.obj = self._dll.CMatrix_new(i)

    def Set(self, x, f):
        self._dll.CMatrix_Set(self.obj, x, f)

    def Get(self):
        size = ct.c_int()
        m = self._dll.CMatrix_Get(self.obj, ct.byref(size))
        return m[:size.value]

    def __del__(self):
        self._dll.CMatrix_delete(self.obj)

cm = CMatrix(2)
cm.Set(0, 1.5)
cm.Set(1, 2.5)
print(cm.Get())

Output:

[1.5, 2.5]

ctypes wrapper for function returning by value objects of a C++ class with destructor

Compile with warnings enabled and I get:

x.cpp(17): warning C4190: 'init_point_by_value' has C-linkage specified, but returns UDT 'Point'
    which is incompatible with C

This is due to the object having a destructor. Remove the destructor and it should accept it.

Another issue is the return type of init_point_by_value is incorrect. It isn't a POINTER(Point) but just a Point:

lib.init_point_by_value.restype = Point

Finally, don't try to free the returned-by-value object.

Result with the fixes as follows (adapted slightly for my Windows system):

test.cpp

#include <iostream>

#define API __declspec(dllexport) // Windows-specific export
extern "C" {

using namespace std;

struct Point {
    int x;
    int y;
};

API Point init_point_by_value(int x, int y) {
    cout << "init_point_by_value called" << endl;
    Point p;
    p.x = x;
    p.y = y;
    return p;
}

API Point& init_point_by_ref(int x, int y) {
    cout << "init_point_by_ref called" << endl;
    Point* p = new Point;
    p->x = x;
    p->y = y;
    return *p;
}

API void cleanup_point(Point* point) {
    cout << "cleanup_point called" << endl;
    if (point) {
        delete point;
    }
}

}

test.py

import ctypes

class Point(ctypes.Structure):
    _fields_ = [
        ('x', ctypes.c_int),
        ('y', ctypes.c_int),
    ]

def setup_lib(lib_path):
    lib = ctypes.cdll.LoadLibrary(lib_path)
    lib.cleanup_point.argtypes = [ctypes.POINTER(Point)]

    lib.init_point_by_value.argtypes = [ctypes.c_int, ctypes.c_int]
    lib.init_point_by_value.restype = Point

    lib.init_point_by_ref.argtypes = [ctypes.c_int, ctypes.c_int]
    lib.init_point_by_ref.restype = ctypes.POINTER(Point)

    return lib

lib = setup_lib('test')

p1 = lib.init_point_by_ref(3, 4)
print(p1.contents.x,p1.contents.y)
lib.cleanup_point(p1)

p2 = lib.init_point_by_value(5, 6)
print(p2.x,p2.y)

Output

init_point_by_ref called
3 4
cleanup_point called
init_point_by_value called
5 6

Python ctypes: wraping c++ class with operators

Problem is, you're trying to cast a Python object to a void *, rather than the void * you already attached to that object's obj attribute.

It should be as simple as changing...

def __eq__(self, other):
    return lib.compare_edge(self.obj, c_void_p(other))

...to...

def __eq__(self, other):
    return lib.compare_edge(self.obj, other.obj)

The explicit call to c_void_p should be unnecessary, since you already declared the types in the line...

lib.compare_edge.argtypes = [c_void_p, c_void_p]

---

Related Topics