Swig C-To-Python Int Array

SWIG C-to-Python Int Array

Use ctypes if you can. It is simpler. However, since you asked for SWIG, what you need is a typemap describing how to handle the int*. SWIG doesn't know how many integers may be pointed to. Below is hacked from an example in the SWIG documentation on multi-argument typemaps:

%typemap(in) (const int memoryCells, int *cellFailure) {
  int i;
  if (!PyList_Check($input)) {
    PyErr_SetString(PyExc_ValueError, "Expecting a list");
    return NULL;
  }
  $1 = PyList_Size($input);
  $2 = (int *) malloc(($1)*sizeof(int));
  for (i = 0; i < $1; i++) {
    PyObject *s = PyList_GetItem($input,i);
    if (!PyInt_Check(s)) {
        free($2);
        PyErr_SetString(PyExc_ValueError, "List items must be integers");
        return NULL;
    }
    $2[i] = PyInt_AsLong(s);
  }
}

%typemap(freearg) (const int memoryCells, int *cellFailure) {
   if ($2) free($2);
}

Note that with this definition, when called from Python leave out the memoryCells parameter and just pass an array such as [1,2,3,4] for cellFailure. The typemap will generate the memoryCells parameter.

P.S. I can post a fully working example (for Windows) if you want it.

SWIG+c+Python: Passing and receiving c arrays

Alright, now I have it. As written in the EDIT above, with numpy.i the arrays can be wrapped quite comfortably. What I did not see was, that the ARGOUT Array does not want an array as Input, as in C. There is just the dimension needed. So, with the Code above, the Script

import bsp
import numpy as np
a = np.array([1, 1], dtype=np.int32)
b = np.array([1, 1], dtype=np.int32)

c = bsp.add(a, b, np.shape(a)[0])

print(c)

Gives the desired Output

[2 2]

Passing numpy array element (int) to c++ int using SWIG

Solved! There were 3 issues:

The numpy.i file I copied over isn't compatible, and the compatible version isn't included in the installation package when you go through anaconda (still not sure why they'd do that).

Answer: Find which version of numpy you're running, then go here (https://github.com/numpy/numpy/releases) and download the numpy-[your_version].zip file, then specifically copy the numpy.i file, found in numpy-[your_version]/tools/swig/. Now paste that numpy.i into your project working directory.

As a default, numpy makes integers of type long. So in tester.py file, I needed to write: a = np.array([1,2,3], dtype=np.intc)
Need to convert numpy int to c++ int in add_vector.i. This can be done by using the %apply directive right above the %include "add_vector.h" line: %apply (int DIM1) {(int x)};

SWIG passing multiple arrays from python to C

When you use a multi-parameter typemap, then multiple C parameters are represented by one Python parameter. In this case, A single numpy array or Python list can be used as a parameter for the pointer and size, since Python knows the length of its objects and the typemap accounts for that.

Also, the error wanted 3 parameters because the second array wasn't matching the types properly. the numpy.i documentation indicated signed char was supported, not char (SWIG can be pedantic), so I used that and it matched the typemap properly and only required two parameters.

Here's a minimal example:

test.i:

%module test

%{
#define SWIG_FILE_WITH_INIT
%}

%include "numpy.i"
%init %{
import_array();
%}

%apply (double* IN_ARRAY1, int DIM1) {(double* dataPtr, int datasize)}
%apply (signed char* IN_ARRAY1, int DIM1) {(char* headerPtr, int headersize)}

%inline %{
    void mainline(double* dataPtr, int datasize, char* headerPtr, int headersize) {
        for(size_t i = 0; i < datasize; ++i)
            printf("[%zu] %lf\n",i,dataPtr[i]);
        for(size_t i = 0; i < headersize; ++i)
            printf("[%zu] %hhd\n",i,headerPtr[i]);
    }
%}

Output:

>>> import test
>>> test.mainline([1.0,1.5,2.0],[1,2,3])
[0] 1.000000
[1] 1.500000
[2] 2.000000
[0] 1
[1] 2
[2] 3
>>> import numpy as np
>>> a = np.array([1.1,2.2,3.3])
>>> b = np.array([1,2,3],dtype=np.int8)
>>> test.mainline(a,b)
[0] 1.100000
[1] 2.200000
[2] 3.300000
[0] 1
[1] 2
[2] 3

Pass str as an int array to a Python C extended function (extended using SWIG)

Regarding my comment, here are some more details about returning arrays from functions: [SO]: Returning an array using C. In short: ways handle this:

Make the returned variable static
Dynamically allocate it (using malloc (family) or new)
Turn it into an additional argument for the function

Getting that piece of C code to run within the Python interpreter is possible in 2 ways:

[Python 3.Docs]: Extending Python with C or C++ - which creates a C written Python module
- A way of doing that is using swig which offers a simple interface for generating the module ([SWIG]: SWIG Basics) saving you the trouble of writing it yourself using [Python 3.Docs]: Python/C API Reference Manual
The other way around, leaving the code in a standard dll which can be accessed via [Python 3.Docs]: ctypes - A foreign function library for Python

Since they both are doing the same thing, mixing them together makes no sense. So, pick the one that best fits your needs.

1. ctypes

This is what you started with
It's one of the ways of doing things using ctypes

ctypes_demo.c:

#include <stdio.h>

#if defined(_WIN32)
#  define CTYPES_DEMO_EXPORT_API __declspec(dllexport)
#else
#  define CTYPES_DEMO_EXPORT_API
#endif

CTYPES_DEMO_EXPORT_API int exposekey(char *bitsIn, char *bitsOut) {
    int ret = 0;
    printf("Message from C code...\n");
    for (int j = 0; j < 1000; j++)
    {
        bitsOut[j] = bitsIn[j + 2000];
        ret++;
    }
    return ret;
}

Notes:

Based on comments, I changed the types in the function from int* to char*, because it's 4 times more compact (although it's still ~700% inefficient since 7 bits of each char are ignored versus only one of them being used; that can be fixed, but requires bitwise processing)
I took a and turned into the 2^nd argument (bitsOut). I think this is best because it's caller responsibility to allocate and deallocate the array (the 3^rd option from the beginning)
I also modified the index range (without changing functionality), because it makes more sense to work with low index values and add something to them in one place, instead of a high index values and subtract (the same) something in another place
The return value is the number of bits set (obviously, 1000 in this case) but it's just an example
printf it's just dummy, to show that the C code gets executed
When dealing with such arrays, it's recommended to pass their dimensions as well, to avoid out of bounds errors. Also, error handling is an important aspect

test_ctypes.py:

from ctypes import CDLL, c_char, c_char_p, c_int, create_string_buffer

bits_string = "010011000110101110101110101010010111011101101010101"

def main():
    dll = CDLL("./ctypes_demo.dll")
    exposekey = dll.exposekey

    exposekey.argtypes = [c_char_p, c_char_p]
    exposekey.restype = c_int

    bits_in = create_string_buffer(b"\0" * 2000 + bits_string.encode())
    bits_out = create_string_buffer(1000)
    print("Before: [{}]".format(bits_out.raw[:len(bits_string)].decode()))
    ret = exposekey(bits_in, bits_out)
    print("After: [{}]".format(bits_out.raw[:len(bits_string)].decode()))
    print("Return code: {}".format(ret))

if __name__ == "__main__":
    main()

Notes:

1^st, I want to mention that running your code didn't raise the error you got
Specifying function's argtypes and restype is mandatory, and also makes things easier (documented in the ctypes tutorial)
I am printing the bits_out array (only the first - and relevant - part, as the rest are 0) in order to prove that the C code did its job
I initialize bits_in array with 2000 dummy 0 at the beginning, as those values are not relevant here. Also, the input string (bits_string) is not 3000 characters long (for obvious reasons). If your bits_string is 3000 characters long you can simply initialize bits_in like: bits_in = create_string_buffer(bits_string.encode())
Do not forget to initialize bits_out to an array with a size large enough (in our example 1000) for its purpose, otherwise segfault might arise when trying to set its content past the size
For this (simple) function, the ctypes variant was easier (at least for me, since I don't use swig frequently), but for more complex functions / projects it will become an overkill and switching to swig would be the right thing to do

Output (running with Python3.5 on Win):

c:\Work\Dev\StackOverflow\q47276327>"c:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" test_ctypes.py
Before: [                                                   ]
Message from C code...
After: [010011000110101110101110101010010111011101101010101]
Return code: 1000

2. swig

Almost everything from the ctypes section, applies here as well

swig_demo.c:

#include <malloc.h>
#include <stdio.h>
#include "swig_demo.h"

char *exposekey(char *bitsIn) {
    char *bitsOut = (char*)malloc(sizeof(char) * 1000);
    printf("Message from C code...\n");
    for (int j = 0; j < 1000; j++) {
        bitsOut[j] = bitsIn[j + 2000];
    }
    return bitsOut;
}

swig_demo.i:

%module swig_demo
%{
#include "swig_demo.h"
%}

%newobject exposekey;
%include "swig_demo.h"

swig_demo.h:

char *exposekey(char *bitsIn);

Notes:

Here I'm allocating the array and return it (the 2^nd option from the beginning)
The .i file is a standard swig interface file
- Defines the module, and its exports (via %include)
- One thing that is worth mentioning is the %newobject directive that deallocates the pointer returned by exposekey to avoid memory leaks
The .h file just contains the function declaration, in order to be included by the .i file (it's not mandatory, but things are more elegant this way)
The rest is pretty much the same

test_swig.py:

from swig_demo import exposekey

bits_in = "010011000110101110101110101010010111011101101010101"

def main():
    bits_out = exposekey("\0" * 2000 + bits_in)
    print("C function returned: [{}]".format(bits_out))

if __name__ == "__main__":
    main()

Notes:

Things make much more sense from Python programmer's PoV
Code is a lot shorter (that is because swig did some "magic" behind the scenes):
- The wrapper .c wrapper file generated from the .i file has ~120K
- The swig_demo.py generated module has ~3K
I used the same technique with 2000 0 at the beginning of the string

Output:

c:\Work\Dev\StackOverflow\q47276327>"c:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" test_swig.py
Message from C code...
C function returned: [010011000110101110101110101010010111011101101010101]

3. Plain Python C API

I added this part as a personal exercise
This is what swig does, but "manually"

capi_demo.c:

#include "Python.h"
#include "swig_demo.h"

#define MOD_NAME "capi_demo"

static PyObject *PyExposekey(PyObject *self, PyObject *args) {
    PyObject *bitsInArg = NULL, *bitsOutArg = NULL;
    char *bitsIn = NULL, *bitsOut = NULL;
    if (!PyArg_ParseTuple(args, "O", &bitsInArg))
        return NULL;
    bitsIn = PyBytes_AS_STRING(PyUnicode_AsEncodedString(bitsInArg, "ascii", "strict"));
    bitsOut = exposekey(bitsIn);
    bitsOutArg = PyUnicode_FromString(bitsOut);
    free(bitsOut);
    return bitsOutArg;
}

static PyMethodDef moduleMethods[] = {
    {"exposekey", (PyCFunction)PyExposekey, METH_VARARGS, NULL},
    {NULL}
};

static struct PyModuleDef moduleDef = {
    PyModuleDef_HEAD_INIT, MOD_NAME, NULL, -1, moduleMethods
};

PyMODINIT_FUNC PyInit_capi_demo(void) {
    return PyModule_Create(&moduleDef);
}

Notes:

It requires swig_demo.h and swig_demo.c (not going to duplicate their contents here)
It only works with Python 3 (actually I got quite some headaches making it work, especially because I was used to PyString_AsString which is no longer present)
Error handling is poor
test_capi.py is similar to test_swig.py with one (obvious) difference: from swig_demo import exposekey should be replaced by from capi_demo import exposekey
The output is also the same to test_swig.py (again, not going to duplicate it here)

How can you wrap a C++ function with SWIG that takes in a Python numpy array as input without explicitly giving a size?

One way I can think of is to suppress the "no size" version of the function and extend the class to have a version with a throw-away dimension variable that uses the actual parameter in the class.

Example:

test.i

%module test

%{
#define SWIG_FILE_WITH_INIT

class Test {
public:
    int _dim; // needs to be public, or have a public accessor.
    Test(int dim) : _dim(dim) {}
    double func(double* array) {
        double sum = 0.0;
        for(int i = 0; i < _dim; ++i)
            sum += array[i];
        return sum;
    }
};
%}

%include "numpy.i"
%init %{
import_array();
%}

%apply (double* IN_ARRAY1, int DIM1) {(double* array, int /*unused*/)};

%ignore Test::func; // so the one-parameter version isn't wrapped

class Test {
public:
    Test(int dim);
    double func(double* array);
};

%rename("%s") Test::func; // unignore so the two-parameter version will be used.

%extend Test {
    double func(double* array, int /*unused*/) {
        return $self->func(array);
    }
}

Demo:

>>> import test
>>> t = test.Test(5)
>>> import numpy as np
>>> a = np.array([1.5,2.0,2.5,3.75,4.25])
>>> t.func(a)
14.0

How to convert C array to Python tuple or list with SWIG?

The following macro did work.

%{
#include "myheader.h"
%}

%define ARRAY_TO_LIST(type, name)
%typemap(varout) type name[ANY] {
  $result = PyList_New($1_dim0);
  for(int i = 0; i < $1_dim0; i++) {
    PyList_SetItem($result, i, PyInt_FromLong($1[i]));
  } // i
}
%enddef

ARRAY_TO_LIST(int, V)

%include "myheader.h"

Swig C-To-Python Int Array