C Function Called from Python Via Ctypes Returns Incorrect Value

C function called from Python via ctypes returns incorrect value

Listing [Python.Docs]: ctypes - A foreign function library for Python.

In order for everything to be properly converted (Python <=> C) when calling the function (residing in a .dll (.so)), 2 things need to be specified (leaving x86 calling convention (Win) aside):

1. Argument types

2. Return type

In CTypes, this is achieved by specifying:

1. argtypes - a list (actually a sequence) containing each argument (CTypes) type (in the order they appear in the function header)

2. restype - a single CTypes type

Side note: an alternative to the above, is prototyping foreign functions (CFUNCTYPE, WINFUNCTYPE, PYFUNCTYPE - check the Function prototypes section (in the URL at the beginning)).

Anyway:

• Failing to specify

• Misspelling (which basically is same thing as previous bullet)

any of them (where required ⁽¹⁾), will result in defaults being applied: all are treated (C89 style) as ints, which (on most systems) are 32 bits long.
This generates Undefined Behavior ⁽²⁾ (also applies when incorrectly specifying them), especially on 64bit CPU / OS, where values of larger types (e.g. pointers) could be truncated.
The displayed errors can be numerous and sometimes misleading.

You misspelled argtype (lacking an s at the end).

Correct that, and you should be fine

Example:

For a function func exported by libdll.dll (libdll.so) with the following header:

double func(uint32_t ui, float f, long long vll[8], void *pv, char *pc);

The Python equivalent would be:

func = libdll.func
func.argtypes = (ctypes.c_uint32, ctypes.c_float, ctypes.c_longlong * 8, ctypes.c_void_p, ctypes.POINTER(ctypes.c_char))
'''
It would be a lot easier (nicer, and in most cases recommended)
  the last element to be ctypes.c_char_p,
  but I chose this form to illustrate pointers in general.
'''
func.restype = ctypes.c_double

Some (more destructive) outcomes of the same (or very similar) scenario (there are many others):

• [SO]: Python ctypes cdll.LoadLibrary, instantiate an object, execute its method, private variable address truncated (@CristiFati's answer)

• [SO]: python ctypes issue on different OSes (@CristiFati's answer)

---

Footnotes

• #1: Technically, there are situations where it's not required to specify them. But even then, it's best to have them specified in order to eliminate any possible confusion:

  • Function with no arguments:

    function_from_dll.argtypes = ()
    

  • Function returning void:

    function_from_dll.restype = None
    

• #2: Undefined Behavior ([Wikipedia]: Undefined behavior) as the name suggests, is a situation when the outcome of a piece of code can't be "predicted" (or guaranteed). The main cases:

  • Works as expected

  • Doesn't work as expected

    • Has some funny outputs / side effects

    • Crashes

  
  The "beauty" of it is that sometimes it seems totally random, sometimes it "only reproduces" under some specific circumstances (different machines, different OSes, different environments, ...). Bottom line is that all of them are purely coincidental! The problem lies in the code (can be the current code (highest chances) or other code it uses (libraries, compilers)).

Getting the wrong type when using a function imported from C in Python - Ctypes

If you don't apply a prototype to function, ctypes assumes that arguments and return are int. See Specifying the required argument types (function prototypes). You can prototype your function so that nothing needs to be recast.

from ctypes import *
c_funcs = CDLL("./myf.so")

c_funcs.map3.argtypes = [c_float, c_float, c_float, c_float]
c_funcs.map3.restype = c_float

def map_num(value, Max, wanted_min, wanted_max):

    x = c_funcs.map3(value, Max, wanted_min, wanted_max)
    print(type(x))
    return x

print(map_num(10,  100, -1, 1))

Now that map3 has been fully prototyped, there isn't a need for the intermediate map_num function if all it does is call the C function and return its result.

Ctypes returns wrong Result

Change this:

lib.add_numbers.restypes = [c_double]

to this:

lib.add_numbers.restype = c_double

Note that it is restype, not restypes.

Wrong return value from c function wrapped in python

Thanks to @Mark Tolonen for the answer:

"At the implementation level, integers are returned in different registers than floating point values. ctypes assumes c_int for return value unless told otherwise. When using ctypes 'explicit is better than implicit'. Always define .restype and .argtypes correctly and you'll have fewer issues."

I had to explicitly define the return type of my function adding the following line to pic_functions.py:

py_cfunctions.dotproduct.restype = c_double

C functions used in python via ctypes slows down when I add if condition

Your original code is only fast because it doesn't actually do anything! The compiler recognizes that you don't actually use dotProductResult, so the function gets optimized down to literally nothing. When you uncomment your conditional, now the value of dotProductResult is used, so it has to actually do the work of figuring out the dot product. If you "add an else condition and write the same statement in if", then it's fast again because the compiler realizes that's equivalent to this:

void dotProduct(double *parameters, double *feature, double *dataset)
    {
        const int DATASET_COUNT = parameters[1];

        if (DATASET_COUNT > 0)
        {
            parameters[3] = DATASET_COUNT - 1;
        }
    }

So it can stop doing the work again.

Python to .cpp and back again with ctypes via an .so

Object files and shared libraries do not contain any information about the types of function parameters or return types, at least not for C linkage functions.

Therefore ctypes has no way of knowing that the function is supposed to take a double argument and return a double.

If you call it with anything else than ctypes.c_double as argument, you will have undefined behavior. But you can set the argument types up for automatic conversion to the correct ctypes type (see below).

ctypes also assumes that the return type is int. If that is not the case, you are supposed to set the return types correctly before calling. In the following I also set up the argument types so that ctypes automatically uses the correct type in calls:

reflect = hdl.reflect
reflect.argtypes = [ctypes.c_double]
reflect.restype = ctypes.c_double
reflect(1.2)

---

Related Topics