std::vector to boost::python::list
I have this function using iterators to convert std::vector
to py::list
:
namespace py = boost::python;
template<class T>
py::list std_vector_to_py_list(const std::vector<T>& v)
{
py::object get_iter = py::iterator<std::vector<T> >();
py::object iter = get_iter(v);
py::list l(iter);
return l;
}
Using boost::python, how to return vector of structs as list of dicts to Python?
In the end I adopted a solution like the following, which I post here in case it is useful to somebody else in the future. This can be improved by adding boost's "readonly" qualifier, but I haven't done so yet.
#include <boost/python.hpp>
using namespace boost::python;
struct Point {
int x, y;
};
using Points = std::vector<Point>;
struct Converter
{
static PyObject* convert(const Points& v)
{
boost::python::list ret;
for (const auto& c : v) {
boost::python::dict *r = new boost::python::dict();
(*r)["x"] = c.x;
(*r)["y"] = c.y;
ret.append(boost::python::object(*r));
}
return boost::python::incref(ret.ptr());
}
};
BOOST_PYTHON_MODULE(mymodule)
{
boost::python::to_python_converter<Points, Converter>();
class_<MyClass, boost::noncopyable>("MyClass")
.def("get_data", &MyClass::get_data);
}
boost::python: Python list to std::vector
To make your C++ method accept Python lists you should use boost::python::list
void massadd(boost::python::list& ns)
{
for (int i = 0; i < len(ns); ++i)
{
add(boost::python::extract<double>(ns[i]));
}
}
Exposing std::vector struct with boost.python
Method .def("get_vector",&myName::myfoo::get_vector)
is not working because it returns a pointer to a vector, so it's necessary to inform the policy that defines how object ownership should be managed:
class_<myName::myfoo>("myfoo", no_init)
// current code
.def("get_vector", &myfoo::get_vector, return_value_policy<reference_existing_object>())
;
In order to use vector_indexing_suite, it is necessary to implement the equal to operator
to the class that it holds:
struct sFOO
{
unsigned int start = 3 ;
double foo = 20.0 ;
bool operator==(const sFOO& rhs)
{
return this == &rhs; //< implement your own rules.
}
};
then you can export the vector:
#include <boost/python/suite/indexing/vector_indexing_suite.hpp>
class_<std::vector<sFOO>>("vector_sFOO_")
.def(vector_indexing_suite<std::vector<sFOO>>())
;
Passing Python list to C++ vector using Boost.python
Assuming you have function that takes a std::vector<Foo>
void bar (std::vector<Foo> arg)
The easiest way to handle this is to expose the vector
to python.
BOOST_PYTHON_MODULE(awesome_module)
{
class_<Foo>("Foo")
//methods and attrs here
;
class_<std::vector<Foo> >("VectorOfFoo")
.def(vector_indexing_suite<std::vector<foo> >() )
;
.def("bar", &bar)
}
So now in python we can stick Foo
s into a vector
and pass the vector to bar
from awesome_module import *
foo_vector = VectorOfFoo()
foo_vector.extend(Foo(arg) for arg in arglist)
bar(foo_vector)
Wrapping an std::vector using boost::python vector_indexing_suite
Due to the semantic differences between the languages, it is often very difficult to apply a single reusable solution to all scenarios when collections are involved. The largest issue is that the while Python collections directly support references, C++ collections require a level of indirection, such as by having shared_ptr
element types. Without this indirection, C++ collections will not be able to support the same functionality as Python collections. For instance, consider two indexes that refer to the same object:
s = Spam()
spams = []
spams.append(s)
spams.append(s)
Without pointer-like element types, a C++ collection could not have two indexes referring to the same object. Nevertheless, depending on usage and needs, there may be options that allow for a Pythonic-ish interface for the Python users while still maintaining a single implementation for C++.
- The most Pythonic solution would be to use a custom converter that would convert a Python iterable object to a C++ collection. See this answer for implementation details. Consider this option if:
- The collection's elements are cheap to copy.
- The C++ functions operate only on rvalue types (i.e.,
std::vector<>
orconst std::vector<>&
). This limitation prevents C++ from making changes to the Python collection or its elements.
- Enhance
vector_indexing_suite
capabilities, reusing as many capabilities as possible, such as its proxies for safely handling index deletion and reallocation of the underlying collection:- Expose the model with a custom
HeldType
that functions as a smart pointer and delegate to either the instance or the element proxy objects returned fromvector_indexing_suite
. - Monkey patch the collection's methods that insert elements into the collection so that the custom
HeldType
will be set to delegate to a element proxy.
- Expose the model with a custom
When exposing a class to Boost.Python, the HeldType
is the type of object that gets embedded within a Boost.Python object. When accessing the wrapped types object, Boost.Python invokes get_pointer()
for the HeldType
. The object_holder
class below provides the ability to return a handle to either an instance it owns or to an element proxy:
/// @brief smart pointer type that will delegate to a python
/// object if one is set.
template <typename T>
class object_holder
{
public:
typedef T element_type;
object_holder(element_type* ptr)
: ptr_(ptr),
object_()
{}
element_type* get() const
{
if (!object_.is_none())
{
return boost::python::extract<element_type*>(object_)();
}
return ptr_ ? ptr_.get() : NULL;
}
void reset(boost::python::object object)
{
// Verify the object holds the expected element.
boost::python::extract<element_type*> extractor(object_);
if (!extractor.check()) return;
object_ = object;
ptr_.reset();
}
private:
boost::shared_ptr<element_type> ptr_;
boost::python::object object_;
};
/// @brief Helper function used to extract the pointed to object from
/// an object_holder. Boost.Python will use this through ADL.
template <typename T>
T* get_pointer(const object_holder<T>& holder)
{
return holder.get();
}
With the indirection supported, the only thing remaining is patching the collection to set the object_holder
. One clean and reusable way to support this is to use def_visitor
. This is a generic interface that allows for class_
objects to be extended non-intrusively. For instance, the vector_indexing_suite
uses this capability.
The custom_vector_indexing_suite
class below monkey patches the append()
method to delegate to the original method, and then invokes object_holder.reset()
with a proxy to the newly set element. This results in the object_holder
referring to the element contained within the collection.
/// @brief Indexing suite that will resets the element's HeldType to
/// that of the proxy during element insertion.
template <typename Container,
typename HeldType>
class custom_vector_indexing_suite
: public boost::python::def_visitor<
custom_vector_indexing_suite<Container, HeldType>>
{
private:
friend class boost::python::def_visitor_access;
template <typename ClassT>
void visit(ClassT& cls) const
{
// Define vector indexing support.
cls.def(boost::python::vector_indexing_suite<Container>());
// Monkey patch element setters with custom functions that
// delegate to the original implementation then obtain a
// handle to the proxy.
cls
.def("append", make_append_wrapper(cls.attr("append")))
// repeat for __setitem__ (slice and non-slice) and extend
;
}
/// @brief Returned a patched 'append' function.
static boost::python::object make_append_wrapper(
boost::python::object original_fn)
{
namespace python = boost::python;
return python::make_function([original_fn](
python::object self,
HeldType& value)
{
// Copy into the collection.
original_fn(self, value.get());
// Reset handle to delegate to a proxy for the newly copied element.
value.reset(self[-1]);
},
// Call policies.
python::default_call_policies(),
// Describe the signature.
boost::mpl::vector<
void, // return
python::object, // self (collection)
HeldType>() // value
);
}
};
Wrapping needs to occur at runtime and custom functor objects cannot be directly defined on the class via def()
, so the make_function()
function must be used. For functors, it requires both CallPolicies and a MPL front-extensible sequence representing the signature.
Here is a complete example that demonstrates using the object_holder
to delegate to proxies and custom_vector_indexing_suite
to patch the collection.
#include <boost/python.hpp>
#include <boost/python/suite/indexing/vector_indexing_suite.hpp>
/// @brief Mockup type.
struct spam
{
int val;
spam(int val) : val(val) {}
bool operator==(const spam& rhs) { return val == rhs.val; }
};
/// @brief Mockup function that operations on a collection of spam instances.
void modify_spams(std::vector<spam>& spams)
{
for (auto& spam : spams)
spam.val *= 2;
}
/// @brief smart pointer type that will delegate to a python
/// object if one is set.
template <typename T>
class object_holder
{
public:
typedef T element_type;
object_holder(element_type* ptr)
: ptr_(ptr),
object_()
{}
element_type* get() const
{
if (!object_.is_none())
{
return boost::python::extract<element_type*>(object_)();
}
return ptr_ ? ptr_.get() : NULL;
}
void reset(boost::python::object object)
{
// Verify the object holds the expected element.
boost::python::extract<element_type*> extractor(object_);
if (!extractor.check()) return;
object_ = object;
ptr_.reset();
}
private:
boost::shared_ptr<element_type> ptr_;
boost::python::object object_;
};
/// @brief Helper function used to extract the pointed to object from
/// an object_holder. Boost.Python will use this through ADL.
template <typename T>
T* get_pointer(const object_holder<T>& holder)
{
return holder.get();
}
/// @brief Indexing suite that will resets the element's HeldType to
/// that of the proxy during element insertion.
template <typename Container,
typename HeldType>
class custom_vector_indexing_suite
: public boost::python::def_visitor<
custom_vector_indexing_suite<Container, HeldType>>
{
private:
friend class boost::python::def_visitor_access;
template <typename ClassT>
void visit(ClassT& cls) const
{
// Define vector indexing support.
cls.def(boost::python::vector_indexing_suite<Container>());
// Monkey patch element setters with custom functions that
// delegate to the original implementation then obtain a
// handle to the proxy.
cls
.def("append", make_append_wrapper(cls.attr("append")))
// repeat for __setitem__ (slice and non-slice) and extend
;
}
/// @brief Returned a patched 'append' function.
static boost::python::object make_append_wrapper(
boost::python::object original_fn)
{
namespace python = boost::python;
return python::make_function([original_fn](
python::object self,
HeldType& value)
{
// Copy into the collection.
original_fn(self, value.get());
// Reset handle to delegate to a proxy for the newly copied element.
value.reset(self[-1]);
},
// Call policies.
python::default_call_policies(),
// Describe the signature.
boost::mpl::vector<
void, // return
python::object, // self (collection)
HeldType>() // value
);
}
// .. make_setitem_wrapper
// .. make_extend_wrapper
};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
// Expose spam. Use a custom holder to allow for transparent delegation
// to different instances.
python::class_<spam, object_holder<spam>>("Spam", python::init<int>())
.def_readwrite("val", &spam::val)
;
// Expose a vector of spam.
python::class_<std::vector<spam>>("SpamVector")
.def(custom_vector_indexing_suite<
std::vector<spam>, object_holder<spam>>())
;
python::def("modify_spams", &modify_spams);
}
Interactive usage:
>>> import example
>>> spam = example.Spam(5)
>>> spams = example.SpamVector()
>>> spams.append(spam)
>>> assert(spams[0].val == 5)
>>> spam.val = 21
>>> assert(spams[0].val == 21)
>>> example.modify_spams(spams)
>>> assert(spam.val == 42)
>>> spams.append(spam)
>>> spam.val = 100
>>> assert(spams[1].val == 100)
>>> assert(spams[0].val == 42) # The container does not provide indirection.
As the vector_indexing_suite
is still being used, the underlying C++ container should only be modified using the Python object's API. For instance, invoking push_back
on the container may cause a reallocation of the underlying memory and cause problems with existing Boost.Python proxies. On the other hand, one can safely modify the elements themselves, such as was done via the modify_spams()
function above.
Related Topics
Is Storing an Invalid Pointer Automatically Undefined Behavior
G++ "Calling" a Function Without Parenthesis (Not F() But F; ). Why Does It Always Return 1
Is It a Good Idea to Return " Const Char * " from a Function
Vector of Const Objects Giving Compile Error
Mixing C++11 Atomics and Openmp
Loading 8 Chars from Memory into an _M256 Variable as Packed Single Precision Floats
Variadic Function Template with Pack Expansion Not in Last Parameter
Why Do Compilers Allow String Literals Not to Be Const
Too Many Initializers for 'Int [0]' C++
What am I Not Understanding About Getline+Strings
How Does Modulus and Rand() Work
Error Lnk1104: Cannot Open File 'Debug\Myprojectlib.Lib'
How to Get the Size of a Memory Block Allocated Using Malloc()
How to Create Nvidia Opencl Project
Comparison of Double, Long Double, Float and Float128
Why Is the Sprite Not Rendering in Opengl
Explicit Specialization in Non-Namespace Scope Does Not Compile in Gcc