Embedding Python in Android

Compile C Cython extensions

April 29, 2014



This article is part of a tutorial series.

  1. Embedding Python in Android
  2. Java Native Interface
  3. Embedding the Python Interpreter
  4. Adding log to the Python Interpreter
  5. Include Python Standard Library
  6. Compile C++ Cython Extensions
  7. Compile C Cython Extensions

In the previous articles of this tutorial series, I've shown how to cross compile and embed Python in Android. In my last article I've also shown how to build and deploy C++ Cython extensions. In this article, I will show how simple is to create a regular C Cython extension and set it up to run on embedded Python for Android.

C Rectangle library

Similar to the C++ Cython example, I will create a C library for simulating and modifying a rectangle. First, let's define rect.h:

/*
 * rect.h
 *
 */

#ifndef RECT_H_
#define RECT_H_

typedef struct Rect {
    int x1, y1, x2, y2;
} Rect;

// Creates a new Rect
Rect* create_rect(int x1, int y1, int x2, int y2);

// Frees a Rect
void free_rect(Rect *rect);

// Returns the area of a Rect object
int rect_area(Rect *rect);

// Moves a Rect object
void move_rect(Rect *rect, int deltax, int deltay);

#endif /* RECT_H_ */

and rect.c:

/*
 * rect.c
 *
 */

#include <stdlib.h>
#include "rect.h"

// Creates a new Rect
Rect* create_rect(int x1, int y1, int x2, int y2) {
    Rect *rect = (Rect *)malloc(sizeof(Rect));
    rect->x1 = x1;
    rect->y1 = y1;
    rect->x2 = x2;
    rect->y2 = y2;
    return rect;
}

// Frees a Rect
void free_rect(Rect *rect) {
    free(rect);
}

// Returns the area of a Rect object
int rect_area(Rect *rect) {
    return (rect->x2 - rect->x1) * (rect->y2 - rect->y1);
}

// Moves a Rect object
void move_rect(Rect *rect, int deltax, int deltay) {
    rect->x1 += deltax;
    rect->x2 += deltax;
    rect->y1 += deltay;
    rect->y2 += deltay;
}

As you can see, we have a Rect structure with the coordinates of the rectangle, and in rect.c we define the functions to create, modify and delete variables of the type of the Rect structure.

Cython wrapper

For wrapping the Rectangle library, we will use the following code in file crect.pyx.

# distutils: language = c
# distutils: sources = rect.c

cdef extern from "rect.h":
    ctypedef struct Rect:
        int x1, y1, x2, y2
    Rect* create_rect(int x1, int y1, int x2, int y2)
    void free_rect(Rect *rect)
    int rect_area(Rect *rect)
    void move_rect(Rect *rect, int deltax, int deltay)

cdef class CRect:
    cdef Rect *thisRect
    def __cinit__(self, int x1, int y1, int x2, int y2):
        self.thisRect = create_rect(x1, y1, x2, y2)
    def __dealloc__(self):
        free_rect(self.thisRect)
    def __str__(self):
        return "(%s, %s, %s, %s)" % (self.thisRect.x1, self.thisRect.y1,
                                     self.thisRect.x2, self.thisRect.y2)
    def area(self):
        return rect_area(self.thisRect)
    def move(self, int dx, int dy):
        move_rect(self.thisRect, dx, dy)

The first cdef extern imports symbols from the rect.h header file. The CRect class is a Python class which works with a pointer to a Rect object to emulate a class.

If you are already familiar with Cython, you know that one of the best ways to build a Cython extension module is using the distutils process. Basically, you create a setup.py file with your distutils recipe and invoke it to build your extension. Here is my setup.py file:

# This can be run as: python setup.py build_ext --inplace

from distutils.core import setup
from distutils.extension import Extension

# Test if Cython is available
try:
    from Cython.Distutils import build_ext
    USE_CYTHON = True
except ImportError:
    USE_CYTHON = False

print "USE_CYTHON =", USE_CYTHON

# If no Cython, we assume a 'crect.c' compiled with: 'cython crect.pyx'
ext = '.pyx' if USE_CYTHON else '.c'
extensions = [Extension('crect', ['crect' + ext], language = "c")]

# Select Extension
if USE_CYTHON:
    from Cython.Build import cythonize
    extensions = cythonize(extensions)
else:
    # No Cython, append 'rect.c' to sources
    extensions[0].sources.append('rect.c');

setup(name = "crect", ext_modules = extensions)

Basically, this script tests for the existence of Cython on the python distribution. If it exists, it uses cythonize to build the extensions from a file named crect.pyx. If there is no Cython, the script assumes that there is a filed named crect.c, which is already Cython-compiled (and adds rect.c as a source file to build the library). This means that, if we do not have Cython available in our Python distribution, we can still compile a Cython extension if we have a file named crect.c. This is useful for distributing Cython extensions for people that do not have Cython in their systems. It is also useful for us to compile to Android, since the Python distribution which we built to deploy to Android does not have Cython installed.

So far, we should have four files (rect.h, rect.c, crect.pyx and setup.py) which are sufficient if we want to test it locally. To build the extension in the same place as the other files are, just execute:

python setup.py build_ext --inplace

This should result with a file named crect.so in the folder. Start the Python interpreter, import CRect, and test.

Python-for-Android recipe

To cross-compile the Rectangle library for Android, we will make use again of Python-for-Android. First, in the python-for-android/ folder (which should already exist as explained in a previous article), open the recipes folder. The recipes folder is a folder where the scripts for building several python libraries are located. For instance, you can find the recipes of Python, Kivy, Numpy, etc.

In recipes, create a new folder named crect. Inside recipes/crect/ create another folder named src/. Copy the source files already created to the recipes/crect/src/ folder (rect.h, rect.c, crect.pyx and setup.py). On recipes/crect/ create a file named recipe.sh. That will be the script to build our Rectangle extension for Android. Set the contents of recipe.sh as follows:

VERSION_crect=
URL_crect=
DEPS_crect=(python)
MD5_crect=
BUILD_crect=$BUILD_PATH/crect/crect
RECIPE_crect=$RECIPES_PATH/crect

function prebuild_crect() {
    cd $BUILD_PATH/crect
    try cp -a $RECIPE_crect/src $BUILD_crect
}

function shouldbuild_crect() {
    true
}

function build_crect() {
    cd $BUILD_crect

    push_arm

    # Cythonize .pyx files
    try find . -iname '*.pyx' -exec $CYTHON {} \;
    try $HOSTPYTHON setup.py build_ext -v
    try $HOSTPYTHON setup.py install -O2

    pop_arm
}

function postbuild_crect() {
    true
}

The interesting bits are:

  • Line 5: We are declaring that our library needs Python. This will force the compilation of Python for Android, since the compilation of native extensions need the Python headers.
  • Lines 7-8: We are setting up the location of the recipe's source and of the build path.
  • Lines 10-13: This function is called before the build happens, and we guarantee that the source files to compile are in the build location. We do that by using cp (copy).
  • Line 19: This is the function which gets called for building the extension.
  • Line 22: This is a function declared in python-for-android/distribute.sh which sets up some environment variables to cross compile. Basically, it sets some flags and the android-armeabi-gcc compiler as default C compiler.
  • Line 25: We manually cythonize the .pyx files to C. We must use Cython manually because the Python distribution of python-for-android does not have Cython installed.
  • Lines 26-27: This is were the extension finally gets built!

To build the extension, cd into the python-for-android root folder, set up the environment variables as explained previously, and run:

./distribute -m "python crect"

If everything worked as it should, you will find crect.so in python-for-android/dist/default/private/lib/python2.7/site-packages/.

Deploy on Android

If you followed the previous articles, you should have a folder in assets/ (assets/lib/python2.7/site-packages/) where you can deploy you extensions since the Python interpreter will look for extensions there (well, not really there, but on /data/data/package_name/files/lib... on the device). You must copy the generated crect.so library to that location on the assets/ folder.

To test the new library, you can add the following to the end of boot.py:

# Import a site-packages compiled module
from crect import CRect
r = CRect(0, 0, 10, 20)
print "CRect =", r
print "CRect area =", r.area()
r.move(20,60)
print "CRect moved to", r
del r

Recompile you Android application, and deploy it on the device. If everything worked as expected, you should see something like the following in your logcat:

I/pyjni   ( 2769): CRect = (0, 0, 10, 20)
I/pyjni   ( 2769): CRect area = 200
I/pyjni   ( 2769): CRect moved to (20, 60, 30, 80)

Congratulations, you have successfully built and deployed a C Cython extension for Python on Android!