Mixing Cdef and Regular Python Attributes in Cdef Class

Mixing cdef and regular python attributes in cdef class

Does this mean once you define a python class with cdef all self.* attributes have to be cdef defined?

Yes. This is stated pretty explicitly in the documentation:

Attributes in cdef classes behave differently from attributes in regular classes:

• All attributes must be pre-declared at compile-time
• ...

You can quite happily store a string by defining the attribute to be of type object:

cdef public object msg

---

Internally, the reason for this is that the cdef class does not have a dictionary, which saves space and makes attribute access faster, but it does mean that it cannot have arbitrary attributes added at runtime. This is reasonably similar to using __slots__ in a normal Python class.

Do Cython extension types support class attributes?

The short answer is yes and no.

No, there is not a convenient syntactic idiom for quickly inserting a class attribute in a cdef class. However ....

The whole point of cython is that it gives you lower level access. The usual motive for the extra effort is performance, but you can also do C-like things with the extra freedom. The difficulty is, there are many pitfalls, and in this case, you won't get pure python class attributes without a lot of work. It is nevertheless pretty easy to get what you need for simple use cases.

For example, suppose I'm making some calculating engine as a class and I wish to globally set the precision of the return value for all instances. I want a default at compile time, and from time to time I may want to adjust it lower to quickly process some trials, and then adjust it higher for my final work. A class attribute is made to order, but you can get the functionality you need in cython as follows:

First, define at the module level the following:

cdef int _precision[1]  # storage for my class 'attribute'
_precision[0]=8         # my default value, set during compilation

Using an array permits us to use the cython idiom precision[0] which is equivalent to the C *precision. The cdef name precision is implicitly a pointer since the data item is an array. This permits using cython syntactic idioms to convert from cython storage locations to python references. If all you want is a global constant that may be accessed by cdef code in any of the classes in the module, you are done. If you want to use it strictly as a class attribute, you must enforce that discipline - the compiler doesn't care.

Now if you also want to adjust the value from python code, you will need a pair of cdef functions that python code in the module can call to access the 'attribute':

cdef int*  get_precision(): return _precision
cdef void* set_precision(int i): _precision[0]=i

At this point, the semantics will vary a bit from pure python, unless you really want to sweat. You need a python setter and getter function, and I find the python descriptor protocol implemented by properties is easiest:

cdef class SomeCalculator:
  ...

  property precision:
    def __get__(self):
      """Get or set calculation precision, default == 8.
         This is like a class attribute: setting affects all instances,
         however, it also affects all subclasses."""
      return get_precision()[0]
    def __set__(self,int integer): set_precision(min(30,max(0,integer)))

The first gets a python reference to the 'attribute'. The second sets the 'attribute' with a python integral value, policed to fall within limits. The cython function call and return interface automatically takes care of conversions, which are more complex than they look.

For example, get_precision returns a C-pointer. If you did the dereferencing in get_precision you would get an error trying to return a C-int in __get__ as if it were python. If instead you just omitted the [0] dereference in __get__ you would get an error trying to return a C-pointer as if it were a python int. As written, automatic conversions correctly match types. cython is very finicky about this sort of thing, and can silently return incorrect values, discoverable only at runtime. It can take some experimentation to infer the correct incantation.

The docstring tells you not to expect a pure python class attribute. If you want to sub-class, and have sub-classes use a different global setting, you will need to sweat a bit more. In python, all that is done automatically.

Even so, there are other differences. A real class attribute may be referenced on the class or on an instance. This property may only be referenced on an instance. Setting a real class attribute on the instance creates an instance specific copy, leaving the class attribute untouched, but invisible to the altered instance.

For the given use case, this works. A real class attribute is unnecessary. Since cython code is usually less abstract and compute intensive, this minimal approach is often enough.

Strange behaviour when creating python attributes in cython cdef class

I have found the solution of my problem. When I was trying to solve this problem I have printed only _c_self attribute of the given object to check that the pointer was properly assigned and it was but when I printed entire object it turned out that python object is None instead of proper object declared as attribute.

print(self.obj_a, self.obj_b) # 66f000c0 66f000c0
print(f'{<int>self.obj_a._c_self:x} {<int>self.obj_b._c_self:x}') # None None

The solution is to add Create function to cdef class:

cdef class pC_Obj_A:
    cdef const C_Obj_A * _c_self

    @staticmethod
    cdef Create(C_Obj_A * ptr):
        cdef pC_Obj_A result = pC_Obj_A()
        result._c_self = ptr
        return result

And use it like this:

cdef class Stack:
    cdef public pC_Obj_A obj_a
    cdef public pC_Obj_B obj_b

    def __init__(self, pC_Obj_C obj_c):
        self.obj_a = pC_Obj_A.Create(obj_c._c_a)
        self.obj_b = pC_Obj_B.Create(obj_c._c_b)

Then printout is:

print(self.obj_a, self.obj_b) # <pC_Obj_A object at 0x029FF610> <pC_Obj_B object at 0x029FF620>
print(f'{<int>self.obj_a._c_self:x} {<int>self.obj_b._c_self:x}') # 2134b9c 2134c08

And everything works great!

Cython class AttributeError

By default cdef attributes are only accessible from within Cython. If you make it a public attribute with cdef public in front of the attribute name then Cython will generate suitable properties to be able to access it from Python.

---

Some extra notes about related problems:

If you're getting the same error from within Cython then you have probably forgotten to tell Cython the type of your instance variable (e.g. v1) - Cython itself can happily access cdef attributes but it only knows about them if it knows the type. If it's just a variable then cdef that variable. If you're trying to use the return value from a function, or index a list, or similar, then you can use a cast: <Vectex>(func()).x. Only do this if you're certain of the type.

You can get similar errors with cdef functions, which are again only visible within Cython. In this case cpdef makes the function visible from to both Cython and Python. However, cpdef functions are in some ways the worst of all worlds (they have all the limitations of cdef functions and all the limitations of def functions) - you're usually better choosing an Cython-only (cdef) or Python (def) interface.

cython class with an attributes as python object instance

You can try something like this :

class Obj(object):
    # At first I define a class Obj() to match your example
    def __init__(self):
        self.abc = 32
        self.xyz = "xyzxyz"

# Then I can define the Example class
cdef class Example:
    cdef public:
        object attr1, attr2, attr3, attr4
    def __init__(self):
        self.attr1 = Obj()
        self.attr2 = Obj()
        self.attr3 = Obj()
        self.attr4 = Obj()

It seems to be cythonized/compiled without error and you have access to the attributes related to the Obj object :

In [11]:
a = Example()
a.attr1.abc
​
Out[11]:
32

---

Related Topics