How to Call Python Code from C Code

How do you call Python code from C code?

I recommend the approaches detailed here. It starts by explaining how to execute strings of Python code, then from there details how to set up a Python environment to interact with your C program, call Python functions from your C code, manipulate Python objects from your C code, etc.

EDIT: If you really want to go the route of IPC, then you'll want to use the struct module or better yet, protlib. Most communication between a Python and C process revolves around passing structs back and forth, either over a socket or through shared memory.

I recommend creating a Command struct with fields and codes to represent commands and their arguments. I can't give much more specific advice without knowing more about what you want to accomplish, but in general I recommend the protlib library, since it's what I use to communicate between C and Python programs (disclaimer: I am the author of protlib).

Calling a python method from C/C++, and extracting its return value

As explained before, using PyRun_SimpleString seems to be a bad idea.

You should definitely use the methods provided by the C-API (http://docs.python.org/c-api/).

Reading the introduction is the first thing to do to understand the way it works.

First, you have to learn about PyObject that is the basic object for the C API. It can represent any kind of python basic types (string, float, int,...).

Many functions exist to convert for example python string to char* or PyFloat to double.

First, import your module :

PyObject* myModuleString = PyString_FromString((char*)"mytest");
PyObject* myModule = PyImport_Import(myModuleString);

Then getting a reference to your function :

PyObject* myFunction = PyObject_GetAttrString(myModule,(char*)"myabs");
PyObject* args = PyTuple_Pack(1,PyFloat_FromDouble(2.0));

Then getting your result :

PyObject* myResult = PyObject_CallObject(myFunction, args)

And getting back to a double :

double result = PyFloat_AsDouble(myResult);

You should obviously check the errors (cf. link given by Mark Tolonen).

If you have any question, don't hesitate. Good luck.

Call python code from c via cython

If you rename the quacker.pyx to quacker.py, everything is actually correct. The only problem is that your program won't search for python modules in the current directory, resulting in the output:

Exception NameError: "name 'quack' is not defined" in 'caller.call_quack' ignored

If you add the current directory to the PYTHONPATH environment variable however, the output becomes the one you'd expect:

$ PYTHONPATH=".:$PYTHONPATH" ./main 
Quack!

When running the python shell, according to the documentation the current directory (or the directory containing the script) is added to the sys.path variable automatically, but when creating a simple program using Py_Initialize and Py_Finalize this does not seem to happen. Since the PYTHONPATH variable is also used to populate the sys.path python variable, the workaround above produces the correct result.

Alternatively, below the Py_Intialize line, you could add an empty string to sys.path as follows by just executing some python code, specified as a string:

PyRun_SimpleString("import sys\nsys.path.insert(0,'')");

After recompiling, just running ./main should then work.

Edit

It's actually interesting to see what's going on if you run the code as specified in the question, so without renaming the quacker.pyx file. In that case, the initcaller() function tries to import the quacker module, but since no quacker.py or quacker.pyc exists, the module cannot be found, and the initcaller() function produces an error.

Now, this error is reported the python way, by raising an exception. But the code in the main.c file doesn't check for this. I'm no expert in this, but in my tests adding the following code below initcaller() seemed to work:

if (PyErr_Occurred())
{
    PyErr_Print();
    return -1;
}

The output of the program then becomes the following:

Traceback (most recent call last):
  File "caller.pyx", line 1, in init caller (caller.c:836)
    from quacker import quack
ImportError: No module named quacker

By calling the initquacker() function before initcaller(), the module name quacker already gets registered so the import call that's done inside initcaller() will detect that it's already loaded and the call will succeed.

How to call Python from C++?

As a qualifier, since I don't know from where you obtained cppyy, the main code that was at the time the reason for typing that sentence does not exist in cppyy master, but does exist in its historic home of PyROOT. This so-called "class generator" plugin allows Cling to "see" Python classes as C++ classes, for straightforward callbacks and even inheritance of C++ classes from Python ones. See this publication (page 3) for some examples: https://www.researchgate.net/publication/288021806_Python_in_the_Cling_World

This code was not ported over to cppyy standalone b/c the class generator relies on interactive use (specifically, dynamic scopes), so only works from Cling, not compiled code, and there is no way (yet) to drop into the Cling prompt from Python (vice versa works).

The reason why that sentence is still there even without the class generator, is that cppyy has since grown a multitude of other ways to call Python from C++ (these have been backported into PyROOT). Examples include C-style function pointers, C++ std::function<> objects, lambdas, and cross-language inheritance. Moreover, these can all be used by importing cppyy into embedded Python (and thus be used from compiled C++).

See e.g. these examples in the documentation: callbacks and cross-inheritance. Cross-inheritance is probably the easiest to use: just define an abstract interface, implement it in Python, pass the pointer to C++ and use it like you would with any pointer-to-interface in C++. For callbacks, declare an extern function pointer (or std::function object) in a header, pass that in a cppyy.include in embedded Python, assign a Python function to that pointer, then call it in C++ as desired.

Callbacks can be made quite sophisticated, assuming that the C++ side can handle it. For example, by providing annotations on the Python side, Python functions can instantiate C++ function pointer templates. Take the completely generic callback in C++ below, which accepts any arguments and producing any result:

>>> import cppyy
>>> cppyy.cppdef("""\
... template<typename R, typename... U, typename... A>
... R callT(R(*f)(U...), A&&... a) {
...    return f(a...);
... }""")
True
>>> def f(a: 'int') -> 'double':
...     return 3.1415*a
...
>>> cppyy.gbl.callT(f, 2)
6.283
>>> def f(a: 'int', b: 'int') -> 'int':
...     return 3*a*b
...
>>> cppyy.gbl.callT(f, 6, 7)
126
>>>

The final way of calling from C++ into Python is indeed not documented b/c it is (still) only available for CPython/cppyy, not PyPy/_cppyy and the naming is implementation-specific as well: CPyCppyy/API.h.

This header is meant to be included in C++ code, allowing the boxing and unboxing of cppyy-bound instances from C++, custom converters and executors, memory management, and parameter packing for stub functions. There are also a couple of convenience functions for dealing with one-offs. For example:

import cppyy

def pyfunc():
    return 42

cppyy.cppdef("""\
#include "CPyCppyy/API.h"

int cppfunc() {
    return (int)CPyCppyy::Eval("pyfunc()");
}""")

print(cppyy.gbl.cppfunc())

(although the example here is run from Python for convenience, this can all be called from embedded Python in compiled C++ as well).

How to execute a Python program from C in Linux

To answer the problem as determined in the comments:

The OP was using execlp or the like, in the form:

execlp("python3", "name_of_script.py", (char*)0);

(or if they didn't know about the issue with NULL, they might have passed NULL instead of (char*)0).

Problem is, execlp usually needs the first argument to be passed twice; the second time it's the value to be set as argv[0], while user-supplied arguments are almost always checked for in argv[1] and higher (the value in argv[0] is rarely used, and when it is, it's mostly for usage/debug output). When python3 sees its own argv, it sees that it's been invoked with the "name" of name_of_script.py, but it doesn't see it as a real "argument", so it acts as if it were launched with no arguments, which leads to the interactive interpreter.

The fix is to pass the program name twice, once to find the program, once to set it in argv, so the argument is recognized by python3 as a script to invoke:

execlp("python3", "python3", "name_of_script.py", (char*)0);
//        ^ program to find
//                    ^ name to set in argv[0]
//                                  ^ script name to put in argv[1] so script is run

How to execute Python script from C code using EXECL?

After much playing around and trying to flush everything I realised I needed to CLOSE the STDOUT end of the PIPE going from the parent to the child (after writing to the pipe of course)...

So the code is now:

#include <sys/types.h>  
#include <stdio.h>   
#include <stdlib.h>   
#include <string.h>   
#include <unistd.h>   

#define STDIN_FILENO    0       /* Standard input.  */ 
#define STDOUT_FILENO   1       /* Standard output.  */ 
#define STDERR_FILENO   2       /* Standard error output.  */ 
#define MAXLINE 4096 

int main(void){ 
 int  n, parent_child_pipe[2], child_parent_pipe[2]; 
 pid_t pid; 
 char line[MAXLINE]; 
 int rv;

 if (pipe(parent_child_pipe) < 0 || pipe(child_parent_pipe) < 0) 
  puts("Error creating pipes...\n"); 

 if ( (pid = fork()) < 0) 
  puts("Error forking...\n"); 
 else if (pid > 0) { /* PARENT */ 
  close(parent_child_pipe[0]);  
  close(child_parent_pipe[1]); 
  while (fgets(line, MAXLINE, stdin) != NULL) { 
   n = strlen(line); 
   if (write(parent_child_pipe[1], line, n) != n) 
    puts("write error to pipe...\n"); 
   close(parent_child_pipe[1]);
   wait(&rv); 
   if ( (n = read(child_parent_pipe[0], line, MAXLINE)) < 0) 
    puts("read error from pipe...\n"); 
   if (n == 0) { 
    puts("child closed pipe...\n"); 
    break; 
   } 
   line[n] = 0; /* null terminate */ 
   if (fputs(line, stdout) == EOF) 
    puts("fputs error...\n"); 
  } 
  if (ferror(stdin)) 
   puts("fgets error on stdin...\n"); 
  exit(0); 

 } else {  /* CHILD */ 
  close(parent_child_pipe[1]); 
  close(child_parent_pipe[0]); 
  if (parent_child_pipe[0] != STDIN_FILENO) { 
   if (dup2(parent_child_pipe[0], STDIN_FILENO) != STDIN_FILENO) 
    puts("dup2 error to stdin...\n"); 
   close(parent_child_pipe[0]); 
  } 
  if (child_parent_pipe[1] != STDOUT_FILENO) { 
   if (dup2(child_parent_pipe[1], STDOUT_FILENO) != STDOUT_FILENO) 
    puts("dup2 error to stdout...\n"); 
   close(child_parent_pipe[1]); 
  } 
  if (execl("./NullFilter.py", "./NullFilter.py", (char *) 0) < 0)
   puts("execl error...\n"); 
 } 
} 

You can see the "close(parent_child_pipe[1]);" just after writing to the PIPE above, that is the crucial piece I had to do. That will force the stream to be flushed to the Python script, Lua script , C code etc....

In the above you will see I am executing a Python script "NullFilter.py"....

NOTE: If you run the code above it will work for one iteration of input / output as the Python script closes the pipe after the first test, but the essentials are there for you to build on...

Thanks for all the help though, I have learnt a lot from this exercise ;-)

Lynton

Syntax error calling Python from C with fork-execv

I think this is what you want:

#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
#include <stdio.h>

int main()
{
    int status;
    char * paramsList[] = {
        "/usr/bin/python",
            "-m",
            "NLTK_Python_Libs",
            "-c",
            "NLTK_Python_Libs.Create_Buffer()",
            (char *)NULL };

    if ( fork() == 0 ){
        printf("I am the child\n");
        execv(paramsList[0],paramsList); }
    else {
        printf("I am the parent\n");
        wait( &status ); }

    return 0;
}

The -m option tells it to load the NLTK_Python_Libs module. Then -c tells it to execute a function from that module. You can't combine the script pathname with the function name to execute the way you did.

There's no need to use /bin/bash at all. Just execute python directly.

Calling Python from C using system( ) call in Linux

On a theoretical level, you might want to explore using DBus with python and c.
Have a python daemon that reads your file, then from c, make a dbus call that returns x number of lines (you can specify that in a dbus header).

This way, you can keep the python file open for as long as you'd like AND you wouldn't require the system() call, which would mean your program would be more secure.

(Dbus can be run from user and system privileges, so your program does not have to be run from an admin level as long as you have permissions for it.)

---

Related Topics