Making an executable in Cython
What you want is the --embed
flag for the Cython compiler.
There isn't a ton of documentation on it, but this is what I was able to find. It does link to a simple working example.
To compile the Cython source code to a C file that can then be compiled to an executable you use a command like cython myfile.pyx --embed
and then compile with whichever C compiler you are using.
When you compile the C source code, you will still need to include the directory with the Python headers and link to the corresponding Python shared library on your system (a file named something like libpython27.so
or libpython27.a
if you are using Python 2.7).
Edit: Here are some more instructions on how to get the commands for including the proper headers and linking against the proper libraries.
As I said earlier, you need to run the Cython compiler like this:
cython <cython_file> --embed
To compile using gcc, you will need to find where the python headers are on your system (you can get this location by running distutils.sysconfig.get_python_inc()
(you'll have to import it first).
It is probably just the /include
subdirectory in your Python installation directory.
You will also have to find the python shared library.
For Python 2.7 it would be libpython27.a
on Windows or libpython2.7.so
on Linux.
Your gcc command will then be
gcc <C_file_from_cython> -I<include_directory> -L<directory_containing_libpython> -l<name_of_libpython_without_lib_on_the_front> -o <output_file_name>
It may be wise to include the -fPIC
flag.
On Windows 64 bit machines you will also have to include the flags -D MS_WIN64
that tells mingw to compile for 64 bit windows.
If you are compiling something that depends on NumPy, you will also need to include the directory containing the NumPy headers.
You can find this folder by running numpy.get_include()
(again, after importing numpy).
Your gcc command then becomes
gcc <C_file_from_cython> -I<include_directory> -I<numpy_include_directory> -L<directory_containing_libpython> -l<name_of_libpython_without_lib_on_the_front> -o <output_file_name>
This gcc command option guide may be helpful.
Also, I would recommend you use Cython memory views if possible.
That will make it so that you won't have to include the NumPy headers and include the NumPy pxd file in your Cython file.
It also makes slicing operations easier for the C compiler to optimize.
Can Cython compile to an EXE?
In principal it appears to be possible to do something like what you want, according to the Embedding Pyrex HOWTO. (Pyrex is effectively a previous generation of Cython.)
Hmm... that name suggests a better search than I first tried: "embedding cython" leads to this page which sounds like what you want.
Creating an executable from Cython Code
The following command compiled your cython program
gcc -municode -DMS_WIN64 -O -Wall -I /c/devel/Python34/include -L /c/devel/Python34/libs/ primes.c -lpython34 -o example.exe
This was done in a windows 7 64bit msys2 environment using gcc 4.9.1 and 5.3.0 64bit versions (win32 threads, SEH exception handling) and 64bit python 3.4.
My original thought was that there would be a problem because of missing main() but cython silently adds one.
How do I create executable for Cython project using setuptools on Windows OS?
I figured it.
setup.py has to expose entry_point:
entry_points={
'console_scripts': [
'mypackage-cli=mypackage.command_line:main',
],
},
command_line.py:
from . import cli
def main():
print cli()
__init__.py:
from markdown import markdown
def cli():
return markdown(u'It''s a CLI!')
after python setup.py develop
it creates executable file mypackage-cli.exe
with 'shim' script mypackage-cli-script.py
in %PYTHONPATH%\Scripts
c:\tmp\mypackage>C:\Python27-32\Scripts\mypackage-cli.exe
<p>It's a CLI!</p>
It requires Python distribution to be at C:\Python27-32
I posted demo here: github:mycyexepackage
Cython: Compile a Standalone Static Executable
As you can see, all of the undefined references in your failed linkage
lie in linked object files that are members of libpython3.5m.a
, which
is the static version of the python3 library requested in your linkage
commandline by pkg-config --libs --cflags python3
.
To link a fully static executable (-static
) when the linkage
includes libpython3.5m.a
, the linker must also find static (*.a
) versions of all the
libraries that libpython3.5m.a
depends upon1. The dynamic
(*.so
) versions of all those dependencies are installed on your system.
That is why:
gcc test.c -otest $(pkg-config --libs --cflags python3)
succeeds, without -static
. The static versions of those dependencies are
not all installed on your system. Hence all the undefined reference
linkage errors when you add -static
.
My own system has python3.6, and I see:
$ pkg-config --libs python-3.6
-lpython3.6m
and:
$ locate libpython3.6m.so
/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu/libpython3.6m.so
/usr/lib/x86_64-linux-gnu/libpython3.6m.so.1
/usr/lib/x86_64-linux-gnu/libpython3.6m.so.1.0
The dynamic dependencies of libpython3.6m.so
are:
$ ldd /usr/lib/python3.6/config-3.6m-x86_64-linux-gnu/libpython3.6m.so
linux-vdso.so.1 => (0x00007ffc475af000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fa87cf6e000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fa87cd51000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa87cb32000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa87c92e000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fa87c72a000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa87c3d4000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa87bff4000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa87d85a000)
We can disregard the first and last ones, which don't look like regular libraries
and indeed aren't2. So, I'd conclude that
to satisfy the static dependencies of libpython3.6a
, I need to install the
static versions of:-
libexpat
libz
libpthread
libdl
libutil
libm
libc
which will be provided by the dev packages of those libraries. Since my system is 64 bit Ubuntu, I'd then filter those dev packages
by:
$ dpkg --search libexpat.a libz.a libpthread.a libdl.a libutil.a libm.a libc.a | grep amd64
libexpat1-dev:amd64: /usr/lib/x86_64-linux-gnu/libexpat.a
zlib1g-dev:amd64: /usr/lib/x86_64-linux-gnu/libz.a
libc6-dev:amd64: /usr/lib/x86_64-linux-gnu/libpthread.a
libc6-dev:amd64: /usr/lib/x86_64-linux-gnu/libdl.a
libc6-dev:amd64: /usr/lib/x86_64-linux-gnu/libutil.a
libc6-dev:amd64: /usr/lib/x86_64-linux-gnu/libm.a
libc6-dev:amd64: /usr/lib/x86_64-linux-gnu/libc.a
and install them with:
sudo apt install libexpat1-dev zlib1g-dev libc6-dev
You haven't disclosed what your system is, but no doubt you can adapt this
thinking to discover the static dependencies of libpython3.5m.a
, whatever
your system is.
[1] Strictly, all
libraries that the members of
libpython3.5m.a
that you are linking depend upon,but we won't be that pernickity.
[2] The first one is the library's vDSO,
not a real file at all. The second one is the linux loader.
Build Cython-compiled modules and python code into executable binary using PyInstaller
After getting familiar with PyInstaller package I am able to figure out the issue. I followed the following steps to make it work for me at the end.
Now, posting my answer to help others :)
## Build *.so files from python modules
1. Execute "setup.py" file
> python setup.py build
2. It will generate "*.so" modules inside "build/lib.linux-x86_64-3.6" dir.
## Created binary from cython modules
1. Copy the binaries (i.e. *.so) files into binary folder
2. Get inside the binary folder 'cd binary'
3. Run Pyinstaller command inside binary directory: `python -O -m PyInstaller --clean --onefile idps.spec`
4. Your binary will be inside dist folder 'binary/dist/'
5. Execute the binary in linux using './dist/sample_app'
6. Your app is ready :)
Here is spec file to make it work for me:
# -*- mode: python -*-
block_cipher = None
a = Analysis(['main.py'],
pathex=['cython_pyinstaller_sample/binary'],
binaries=[('program_a.cpython-36m-x86_64-linux-gnu.so', '.'),('program_b.cpython-36m-x86_64-linux-gnu.so', '.')],
datas=[('config_file.txt', '.')],
hiddenimports=['licensing', 'licensing.methods', 'pandas'],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False) pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher) exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[],
name='sample_app',
debug=True,
bootloader_ignore_signals=False,
strip=False,
upx=True,
runtime_tmpdir=None,
console=True )
Related Topics
How to Convert a Python Datetime.Datetime to Excel Serial Date Number
Opencv - Apply Mask to a Color Image
How to Strip the Whitespace from Pandas Dataframe Headers
How to Check If a String Only Contains Letters
How to Check That Multiple Keys Are in a Dict in a Single Pass
Shipping Python Modules in Pyspark to Other Nodes
Deleting Multiple Columns Based on Column Names in Pandas
Fill Between Two Vertical Lines in Matplotlib
Plotting Networkx Graph with Node Labels Defaulting to Node Name
How to Call Function That Takes an Argument in a Django Template
Why Return Notimplemented Instead of Raising Notimplementederror
Django Signals VS. Overriding Save Method
Python: Dynamically Create Function at Runtime
How to Pass an Argument to Event Handler in Tkinter
How to Change Sprite Colours in Pygame
How to Extract Frequency Associated with Fft Values in Python
Read a Small Random Sample from a Big CSV File into a Python Data Frame