Boost-Python How to Pass a C++ Class Instance to a Python Class

Boost-python How to pass a c++ class instance to a python class

Pass the object pointer via boost::python::ptr to python. This will prevent the python interpreter from makeing a copy:

#include <boost/python.hpp>
#include <string>
#include <iostream>

using namespace boost::python;
using namespace std;

class World
{
private:
    string name;
public:
    void set(string name) {
        this->name = name;
    }
    void greet() {
        cout << "hello, I am " << name << endl;
    }
};

typedef boost::shared_ptr< World > world_ptr;

BOOST_PYTHON_MODULE(hello)
{
    class_<World>("World")
        .def("greet", &World::greet)
        .def("set", &World::set)
    ;
};

int main(int argc, char **argv)
{
    Py_Initialize();
    try {
        PyRun_SimpleString(
            "class Person:\n"
            "    def sayHi(self):\n"
            "        print 'hello from python'\n"
            "    def greetReset(self, instance):\n"
            "        instance.set('Python')\n"
          );

        world_ptr worldObjectPtr (new World);
        worldObjectPtr->set("C++!");

        inithello();
        object o_main 
            = object(handle<>(borrowed(PyImport_AddModule("__main__"))));
        object o_person_type = o_main.attr("Person");
        object o_person = o_person_type();
        object o_func1 = o_person.attr("sayHi");
        o_func1();
        object o_func2 = o_person.attr("greetReset");
        o_func2(boost::python::ptr(worldObjectPtr.get()));
        worldObjectPtr->greet();
    }
    catch (error_already_set) {
        PyErr_Print();
    }

    Py_Finalize();

    return 0;
}

passing C++ classes instances to python with boost::python

boost::python knows all about boost::shared_ptr, but you need to tell it that boost::shared_ptr<A> holds an instance of A, you do this by adding boost::shared_ptr<A> in the template argument list to class_, more information on this 'Held Type' is here in the boost documentation.

To prevent instances being created from python, you add boost::python::no_init to the class_ constructor, so you end up with:

boost::python::class_< A, boost::shared_ptr<A> >("A", boost::python::no_init)
    //... .def, etc
    ;

In general you should not pass around shared pointers by reference, since if the reference to the shared pointer is invalidated, then the reference to which the shared pointer is pointing to is also invalidated (since taking a reference of the shared pointer didn't increment the reference counter to the pointed to object).

It is perfectly safe to pass boost::shared_ptr objects around to and from python, reference counts (python and shared_ptr) will be correctly managed provided you don't change the return_value_policy. If you change the policy of a method exposed in python so that it returns a reference to a shared pointer then you can cause problems, just as passing shared pointers around by c++ references can cause problems.

(Also, you should use make_shared<A>(...) in preference to shared_ptr<A>(new A(...)).)

Pass a type object (class, not an instance) from python to c++

To pass a Python type object, one needs to create a C++ type and register a custom a custom converter. As a Python type object is a python object, creating a type that derives from boost::python::object is appropriate:

/// @brief boost::python::object that refers to a type.
struct type_object: 
  public boost::python::object
{
  /// @brief If the object is a type, then refer to it.  Otherwise,
  ///        refer to the instance's type.
  explicit
  type_object(boost::python::object object):
    boost::python::object(get_type(object))
  {}

private:

  /// @brief Get a type object from the given borrowed PyObject.
  static boost::python::object get_type(boost::python::object object)
  {
    return PyType_Check(object.ptr())
      ? object 
      : object.attr("__class__");
  }
};

// ... register custom converter for type_object.

However, the example code presents an additional problem. One cannot directly perform comparisons between a Python type object and a C++ type. Furthermore, The Python type object has no direct association with the C++ type. To perform comparisons, one needs to compare the Python type objects.

Boost.Python uses an internal registry to associate C++ type identity, in the form of boost::python::type_info, to a Python class object. This association is one-way, in that one can only lookup a Python class object. Lets expand the type_object class to allow to provide auxiliaries functions for checking against C++ types:

/// @brief boost::python::object that refers to a type.
struct type_object: 
  public boost::python::object
{
  ...

  /// @brief Type identity check.  Returns true if this is the object returned
  ///        returned from type() when passed an instance of an object created
  ///        from a C++ object with type T.
  template <typename T>
  bool is() const
  {
    // Perform an identity check that registartion for type T and type_object
    // are the same Python type object.
    return get_class_object<T>() == static_cast<void*>(ptr());
  }

  /// @brief Type identity check.  Returns true if this is the object is a
  ///        subclass of the type returned returned from type() when passed
  ///        an instance of an object created from a C++ object with type T.
  template <typename T>
  bool is_subclass() const
  {
    return PyType_IsSubtype(reinterpret_cast<PyTypeObject*>(ptr()),
                            get_class_object<T>());
  }

private:

  ...

  /// @brief Get the Python class object for C++ type T.
  template <typename T>
  static PyTypeObject* get_class_object()
  {
    namespace python = boost::python;
    // Locate registration based on the C++ type.
    const python::converter::registration* registration =
          python::converter::registry::query(python::type_id<T>());

    // If registration exists, then return the class object.  Otherwise,
    // return NULL.
    return (registration) ? registration->get_class_object()
                          : NULL;
  }
};

Now, if type is an instance of type_object, one could check:

• If type is the Python type associated with the C++ Spam type with type.is<Spam>().
• If type is a subclass of the Python type associated with the C++ Spam type with type.is_subclass<Spam>().

---

Here is a complete example based on the original code that demonstrates receiving type objects to functions, checking for type identity and subclasses:

#include <boost/python.hpp>

/// @brief boost::python::object that refers to a type.
struct type_object: 
  public boost::python::object
{
  /// @brief If the object is a type, then refer to it.  Otherwise,
  ///        refer to the instance's type.
  explicit
  type_object(boost::python::object object):
    boost::python::object(get_type(object))
  {}

  /// @brief Type identity check.  Returns true if this is the object returned
  ///        returned from type() when passed an instance of an object created
  ///        from a C++ object with type T.
  template <typename T>
  bool is() const
  {
    // Perform an identity check that registartion for type T and type_object
    // are the same Python type object.
    return get_class_object<T>() == static_cast<void*>(ptr());
  }

  /// @brief Type identity check.  Returns true if this is the object is a
  ///        subclass of the type returned returned from type() when passed
  ///        an instance of an object created from a C++ object with type T.
  template <typename T>
  bool is_subclass() const
  {
    return PyType_IsSubtype(reinterpret_cast<PyTypeObject*>(ptr()),
                            get_class_object<T>());
  }

private:

  /// @brief Get a type object from the given borrowed PyObject.
  static boost::python::object get_type(boost::python::object object)
  {
    return PyType_Check(object.ptr())
      ? object 
      : object.attr("__class__");
  }

  /// @brief Get the Python class object for C++ type T.
  template <typename T>
  static PyTypeObject* get_class_object()
  {
    namespace python = boost::python;
    // Locate registration based on the C++ type.
    const python::converter::registration* registration =
          python::converter::registry::query(python::type_id<T>());

    // If registration exists, then return the class object.  Otherwise,
    // return NULL.
    return (registration) ? registration->get_class_object()
                          : NULL;
  }
};

/// @brief Enable automatic conversions to type_object.
struct enable_type_object
{
  enable_type_object()
  {
    boost::python::converter::registry::push_back(
      &convertible,
      &construct,
      boost::python::type_id<type_object>());
  }

  static void* convertible(PyObject* object)
  {
    return (PyType_Check(object) || Py_TYPE(object)) ? object : NULL;
  }

  static void construct(
    PyObject* object,
    boost::python::converter::rvalue_from_python_stage1_data* data)
  {
    // Obtain a handle to the memory block that the converter has allocated
    // for the C++ type.
    namespace python = boost::python;
    typedef python::converter::rvalue_from_python_storage<type_object>
                                                                 storage_type;
    void* storage = reinterpret_cast<storage_type*>(data)->storage.bytes;

    // Construct the type object within the storage.  Object is a borrowed 
    // reference, so create a handle indicting it is borrowed for proper
    // reference counting.
    python::handle<> handle(python::borrowed(object));
    new (storage) type_object(python::object(handle));

    // Set convertible to indicate success. 
    data->convertible = storage;
  }
};

// Mockup types.
struct A {};
struct B: public A {};
struct C {};

/// Mockup function that receives an object's type.
int func(type_object type)
{
  if (type.is<A>()) return 0;
  if (type.is<B>()) return 1;
  return -1;
}

/// Mockup function that returns true if the provided object type is a
/// subclass of A.
bool isSubclassA(type_object type)
{
  return type.is_subclass<A>();
}

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;

  // Enable receiving type_object as arguments.
  enable_type_object();

  python::class_<A>("A");
  python::class_<B, python::bases<A> >("B");
  python::class_<C>("C");

  python::def("func", &func);
  python::def("isSubclassA", &isSubclassA);
}

Interactive usage:

>>> import example
>>> assert(example.func(type("test")) == -1)
>>> assert(example.func(example.A) == 0)
>>> assert(example.func(example.B) == 1)
>>> assert(example.isSubclassA(example.A))
>>> assert(example.isSubclassA(example.B))
>>> assert(not example.isSubclassA(example.C))
>>> assert(example.func("test") == -1)
>>> assert(example.func(example.A()) == 0)
>>> assert(example.func(example.B()) == 1)
>>> assert(example.isSubclassA(example.A()))
>>> assert(example.isSubclassA(example.B()))
>>> assert(not example.isSubclassA(example.C()))

Passing a Python class instance into a C++ function

There are two errors in the initial code:

• Boost.Python is attempting to pass two arguments (self and an instance of Foo) to the static Wine::do_something() C++ function that only accepts one argument. To resolve this, when exposing the Wine class, the Python Wine.do_something() member function needs to be set as static via the boost::python::class_::staticmethod() member function. When exposed as a static method, Boost.Python will no longer pass the self instance argument.
• Unlike the Python/C API, where pointers are often used as handles to objects (PyObject*), Boost.Python provides a higher-level notation boost::python::object class that is often passed around by value or reference. Internally, this class interacts with a boost::python::handle that performs smart pointer management for PyObject.

---

Here is a complete Python extension based on the original code:

#include <boost/python.hpp>

class Wine
{
public:  

  static void do_something(boost::python::object object)
  {
    int a = 1;
    int b = 2;
    int c = 3;

    object.attr("Bar1")(a, b, c);
    object.attr("Bar2")(a, b, c);
  };
};

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::class_<Wine>("Wine")
    .def("do_something", &Wine::do_something)
      .staticmethod("do_something")
    ;
};

Interactive usage:

>>> class Foo(object):
...     def Bar1(self, a, b, c):
...         print "Bar1", locals()
...     def Bar2(self, a, b, c):
...         print "Bar2", locals()
... 
>>> import example
>>> cheese = example.Wine()
>>> cheese.do_something(Foo())
Bar1 {'a': 1, 'c': 3, 'b': 2, 'self': <__main__.Foo object at 0xb6b0f2ac>}
Bar2 {'a': 1, 'c': 3, 'b': 2, 'self': <__main__.Foo object at 0xb6b0f2ac>}

pass C++ object to python function by boost::python

You need to expose your Test type to Python, as shown here: http://wiki.python.org/moin/boost.python/HowTo

Boost.Python - Passing boost::python::object as argument to python function?

Got this fantastic solution from the c++sig mailing list.

Implement a std::map<std::string, boost::python::object> in the C++ class, then overload __getattr__() and __setattr__() to read from and write to that std::map. Then just send it to the python with boost::python::ptr() as usual, no need to keep an object around on the C++ side or send one to the python. It works perfectly.

Edit: I also found I had to override the __setattr__() function in a special way as it was breaking things I added with add_property(). Those things worked fine when getting them, since python checks a class's attributes before calling __getattr__(), but there's no such check with __setattr__(). It just calls it directly. So I had to make some changes to turn this into a full solution. Here's the full implementation of the solution:

First create a global variable:

boost::python::object PyMyModule_global;

Create a class as follows (with whatever other information you want to add to it):

class MyClass
{
public:
   //Python checks the class attributes before it calls __getattr__ so we don't have to do anything special here.
   boost::python::object Py_GetAttr(std::string str)
   {
      if(dict.find(str) == dict.end())
      {
         PyErr_SetString(PyExc_AttributeError, JFormat::format("MyClass instance has no attribute '{0}'", str).c_str());
         throw boost::python::error_already_set();
      }
      return dict[str];
   }

   //However, with __setattr__, python doesn't do anything with the class attributes first, it just calls __setattr__.
   //Which means anything that's been defined as a class attribute won't be modified here - including things set with
   //add_property(), def_readwrite(), etc.
   void Py_SetAttr(std::string str, boost::python::object val)
   {
      try
      {
         //First we check to see if the class has an attribute by this name.
         boost::python::object obj = PyMyModule_global["MyClass"].attr(str.c_str());
         //If so, we call the old cached __setattr__ function.
         PyMyModule_global["MyClass"].attr("__setattr_old__")(ptr(this), str, val);
      }
      catch(boost::python::error_already_set &e)
      {
         //If it threw an exception, that means that there is no such attribute.
         //Put it on the persistent dict.
         PyErr_Clear();
         dict[str] = val;
      }
   }
private:
   std::map<std::string, boost::python::object> dict;
};

Then define the python module as follows, adding whatever other defs and properties you want:

BOOST_PYTHON_MODULE(MyModule)
{
   boost::python::class_<MyClass>("MyClass", boost::python::no_init)
      .def("__getattr__", &MyClass::Py_GetAttr)
      .def("__setattr_new__", &MyClass::Py_SetAttr);
}

Then initialize python:

void PyInit()
{
   //Initialize module
   PyImport_AppendInittab( "MyModule", &initMyModule );
   //Initialize Python
   Py_Initialize();

   //Grab __main__ and its globals
   boost::python::object main = boost::python::import("__main__");
   boost::python::object global = main.attr("__dict__");

   //Import the module and grab its globals
   boost::python::object PyMyModule = boost::python::import("MyModule");
   global["MyModule"] = PyMyModule;
   PyMyModule_global = PyMyModule.attr("__dict__");

   //Overload MyClass's setattr, so that it will work with already defined attributes while persisting new ones
   PyMyModule_global["MyClass"].attr("__setattr_old__") = PyMyModule_global["MyClass"].attr("__setattr__");
   PyMyModule_global["MyClass"].attr("__setattr__") = PyMyModule_global["MyClass"].attr("__setattr_new__");
}

Once you've done all of this, you'll be able to persist changes to the instance made in python over to the C++. Anything that's defined in C++ as an attribute will be handled properly, and anything that's not will be appended to dict instead of the class's __dict__.

How to Create and Use Instance of a Python Object with boost::python

This is what ultimately worked for me.

namespace python = boost::python;
python::object main = python::import("main");
python::object mainNamespace = main.attr("__dict__");

//add the contents of the script to the global namespace
python::object script = python::exec_file(path_to_my_script, mainNamespace);

//add an instance of the object to the global namespace
python::exec("foo = MyPythonClass()", mainNamespace);
//create boost::python::object that refers to the created object
python::object foo = main.attr("foo");

//call Func2 on the python::object via attr
//then extract the result into a const char* and assign it to a std::string
//the last bit could be done on multiple lines with more intermediate variables if desired
const std::string func2return = python::extract<const char*>(foo.attr("Func2")("hola"));
assert(func2return == "hola");

Feel free to comment if there is a better way.

Pass C++ object instance to Python function

I've written some sample code based on your PyData data object; this code uses the boost::python data structures (tuple and list) for exchanging data to/from Python, as this is their intended use, but these can be populated by copying data into them from std::tuple and std::vector as needed.

This works with Python 2.7 and boost 1.53. Hopefully you can use this to help; NB the call to initpydata() (generated function) is required after Py_Initialze().

C++ code:

#include <iostream>
#include <vector>
#include <tuple>
#include <boost/python.hpp>
#include <boost/python/list.hpp>

class PyData
{
    public:

    PyData() {}

    float m_slope;
    float m_compliance;

    boost::python::tuple    m_TORSO_LV;
    boost::python::list     m_DsOsAVS;
    boost::python::list     m_RF_FORCES;
    boost::python::list     m_LF_FORCES;

    void InitData()
    {
        // simulate setting up data
        m_slope = 1.0;
        m_compliance = 2.0;

        m_TORSO_LV = boost::python::make_tuple(3.0, 4.0, 5.0);

        m_DsOsAVS.append(boost::python::make_tuple(10.0, 11.0, 12.0));
        m_DsOsAVS.append(boost::python::make_tuple(20.0, 21.0, 22.0));

        // etc.
    }

    ~PyData() {}
};

BOOST_PYTHON_MODULE(pydata) {
boost::python::class_<PyData>("PyData")
    .def_readwrite("Torso_LV", &PyData::m_TORSO_LV)
    .def_readwrite("DsOsAVs", &PyData::m_DsOsAVS)
    .def_readwrite("RF_FORCES", &PyData::m_RF_FORCES)
    .def_readwrite("LF_FORCES", &PyData::m_LF_FORCES)
    .def_readwrite("slope", &PyData::m_slope)
    .def_readwrite("compliance", &PyData::m_compliance)
    ;
};

int main (int argc, char * argv[])
{
    Py_Initialize();

    initpydata();

    boost::python::object main=boost::python::import("__main__");
    boost::python::object global(main.attr("__dict__"));
    boost::python::object result = boost::python::exec_file("/home/andy/Python2.py", global, global);
    boost::python::object predict_on_data = global["predict_on_data"];
    if (!predict_on_data.is_none())
    {
        boost::shared_ptr<PyData> o(new PyData);
        o->InitData();
        predict_on_data(boost::python::ptr(o.get()));
        std::cout << "values in c++ object are now: " << o->m_slope << " and " << o->m_compliance << std::endl;
    }

    return 0;
}

Python code (Python2.py file in this example):

def predict_on_data(o):
    print "In Python:"
    print repr(o)
    # print the data members in o
    print "o.slope is " + repr(o.slope)
    print "o.compliance is " + repr(o.compliance)
    print "o.Torso_LV is " + repr(o.Torso_LV)
    print "o.m_DsOsAVs is " + repr(o.DsOsAVs)
    # modify some data
    o.slope = -1.0
    o.compliance = -2.0

Once running this should give output like this:

In Python:
<pydata.PyData object at 0x7f41200956e0>
o.slope is 1.0
o.compliance is 2.0
o.Torso_LV is (3.0, 4.0, 5.0)
o.m_DsOsAVs is [(10.0, 11.0, 12.0), (20.0, 21.0, 22.0)]
values in c++ object are now: -1 and -2

Hope this is useful.

---

Related Topics