Embedding Python in Android

Python Interpreter

April 23, 2014



This article is part of a tutorial series.

  1. Embedding Python in Android
  2. Java Native Interface
  3. Embedding the Python Interpreter
  4. Adding log to the Python Interpreter
  5. Include Python Standard Library
  6. Compile C++ Cython Extensions
  7. Compile C Cython Extensions

In the previous article, I have shown how to create an Android application which uses the Java Native Interface to interface with C/C++ code. Specifically, we have created a Java class which exports a native "square(int)" method which returns the square of a number computed in native code. In this article, I will show how to do something similar, but using an embedded Python interpreter to do the calculations. Since we are going to use CPython, the official Python interpreter in C, we must use JNI again to interface with it.

Python-for-Android

In the opening article, I mentioned Kivy, a cross-platform framework for building touch applications. Kivy is a very good framework if you need a custom interface, and I highly recommend it, but unfortunately, it does not allow you to interface with the Android (or iOS) native GUI APIs. However, there is one Kivy project that will be useful for this tutorial.

Python-for-Android is a Kivy subproject which allows us to create a Python distribution for Android, including the modules we want (yes, C/C++ extensions also), and wrap everything in an apk. We are going to use some tools from Python-for-Android, although this project is too much oriented towards generating Kivy applications.

Before we start, I recommend to read the Python-for-Android docs and install the toolchain for you operating system.

Cross-compiling Python for Android

We are going to reuse the Android application created on the previous article. On my machine, it is located in "~/workspace/pytest/". The first thing to do is to clone the python-for-android git repository into our Android application folder.

cd ~/workspace/pytest/
git clone git://github.com/kivy/python-for-android

In the end of the process, you should have a "python-for-android" folder. In the next step, we must export some environment variables. Execute the following commands in your terminal, and update accordingly to your specific case, i.e., your location of the SDK/NDK, and their respective versions.

export ANDROIDSDK="/path/to/android/android-sdk-linux_86"
export ANDROIDNDK="/path/to/android/android-ndk-r9c"
export ANDROIDNDKVER=r9c
export ANDROIDAPI=19

Now we are going to build a Python distribution, which may include some other modules. In a future article, I may show how you can compile C/C++/Cython extensions for Android, but for now, I recommend you to look at the "recipes" folder inside "python-for-android". That folder contains python extensions which python-for-android knows how to compile for Android. If you want to include a personal extension or other, just create a new recipe (and if it is a popular extension, you should consider sending a submit patch to kivy-for-android).

To cross-compile the python interpreter and build a simple python distribution, do:

cd ~/workspace/pytest/python-for-android/
./distribute.sh -m "pil"

Note that I have used "pil" only because the "distribute" script needs something between the quotes, or else it won't build. However, you could pass another module as argument. This will build a Python 2.7.1 distribution patched for Android.

You may notice that you now have two new folders in your "python-for-android" root. In the "build" folder, you can find the files of build process. In the "dist/default" folder, we have our python distribution. The interesting bits for now are the "dist/default/libs/armeabi/libpython2.7.so" which is the python interpreter compiled as shared library, the standard library in "dist/default/private/lib/python2.7″ (compiled as pyo objects), and the python header files in "dist/default/python-install/include/python2.7″. We will make use of the header files when we try to compile our JNI to python-interpreter wrapper. The rest of the contents in "dist/default" are specific for Kivy applications, and contains Kivy bootstrap code. You can find binaries in "dist/default/python-install/bin" although I did not test them to see if they work on the device.

Java source

First, on the Java side, we are going to create a Java class to wrap our calls to the Python interpreter (via JNI). In your IDE create a new Java class in your Android source, name it "PythonWrapper" and update it to be something like:

package com.example.pytest;

public class PythonWrapper {

    // Declare native method (and make it public to expose it directly)
    public static native int start();
    public static native int end();
    public static native int square(int value);

    // Load library
    static {
        System.loadLibrary("python2.7");
        System.loadLibrary("pyjni");
    }
}

You can see that we are defining three native methods (start, end and square), and we are loading two libraries. The first is the python interpreter (which is libpython2.7 – android implicitly searches for shared libraries named "X" appending a lib before, such as libX). The second is the JNI glue code which we will implement next.

Generate C header file for pyjni (from PythonWrapper.java)

If you are reusing the project from the previous article, you shall have a "jni" and a "tmp" folder already created. Now, we are going to compile the PythonWrapper to java class file, and then generate a JNI header from it. Do it in the terminal:

cd ~/workspace/pytest
javac -d tmp/ src/com/example/pytest/PythonWrapper.java
cd ~/workspace/pytest/tmp
javah -jni com.example.pytest.PythonWrapper
mv com_example_pytest_PythonWrapper.h ../jni/pyjni.h

This compiles the PythonWrapper.java file, generates a JNI for the PythonWrapper and moves it to the "jni" folder as "pyjni.h". We could use another name, but I want it to match the name we gave in PythonWrapper.java.

Create C source file for pyjni

Create a new C source file in "jni" named pyjni.c.

#include <Python.h>
#include "pyjni.h"
#include <android/log.h>

#define APPNAME "pyjni"
#define LOG(x) __android_log_write(ANDROID_LOG_INFO, APPNAME, (x))

JNIEXPORT jint JNICALL Java_com_example_pytest_PythonWrapper_start
  (JNIEnv *env, jclass jc)
{
    LOG("Initializing the Python interpreter");
    Py_Initialize();
    return 0;
}

JNIEXPORT jint JNICALL Java_com_example_pytest_PythonWrapper_end
  (JNIEnv *env, jclass jc)
{
    LOG("Finalizing the Python interpreter");
    Py_Finalize();
    return 0;
}

JNIEXPORT jint JNICALL Java_com_example_pytest_PythonWrapper_square
  (JNIEnv *env, jclass jc, jint value)
{
    int val;
    char buf[100];

    LOG("Squaring number in the Python interpreter");

    sprintf(buf, "value = %d ** 2", value);
    PyRun_SimpleString(buf);

    PyObject *m = PyImport_AddModule("__main__");
    PyObject *v = PyObject_GetAttrString(m, "value");
    val = PyInt_AsLong(v);
    Py_DECREF(v);

    return val;
}

In line 1 we are including the Python header. That is important so we can use the Python interpreter. In lines 3-6 we are just setting up logging facilities to our source code, so we can see prints in the Android logcat. Lines 9-15 and 17-23 are native functions which starts and ends the Python Interpreter.

Lines 26-43 is the square function. Basically, in line 34 we are creating a string corresponding to Python source code ("value = %d ** 2"), and we execute the string in line 35. In lines 37-40 we get the main module and "value" as PyObjects, cast the result to an integer, decrease the reference counter for the value and return it. Simple!

Android.mk and build

Now, we have to update our previous makefile to be able to recognize "Python.h" and include the interpreter in the final apk. Open you Android.mk file inside "jni" and include the following:

LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)
LOCAL_MODULE    := square
LOCAL_SRC_FILES := square.c
include $(BUILD_SHARED_LIBRARY)

# Build libpyjni.so
include $(CLEAR_VARS)
LOCAL_MODULE    := pyjni
LOCAL_SRC_FILES := pyjni.c
LOCAL_CFLAGS := -I $(LOCAL_PATH)/../python-for-android/dist/default/python-install/include/python2.7/
LOCAL_LDFLAGS += -L $(LOCAL_PATH)/../python-for-android/dist/default/libs/armeabi/
LOCAL_SHARED_LIBRARIES += python2.7  # This line links to libpython2.7
LOCAL_LDLIBS += -llog                # This line links to the Android log
include $(BUILD_SHARED_LIBRARY)

# Include libpython2.7.so
include $(CLEAR_VARS)
LOCAL_MODULE := python2.7
LOCAL_SRC_FILES := ../python-for-android/dist/default/libs/armeabi/libpython2.7.so
include $(PREBUILT_SHARED_LIBRARY)

Lines 3-6 are from the previous example. Lines 8-16 are there to build our "libpyjni.so" shared library. In line 12 we are saying to the ndk cross-compiler to use the Python header files generated with "python-for-android". In lines 13-14, we are telling the linker to use the cross-compiled "libpython2.7.so", and finally, line 15 is to add the Android Logging. As for lines 19-22, they are there so that "libpython2.7.so" makes its way to the "/libs/armeabi" folder.

Go to the terminal and build the native libraries.

cd ~/workspace/pytest
ndk-build

It will output something as:

[armeabi] Compile thumb  : pyjni <= pyjni.c
[armeabi] SharedLibrary  : libpyjni.so
[armeabi] Install        : libpyjni.so => libs/armeabi/libpyjni.so
[armeabi] Install        : libpython2.7.so => libs/armeabi/libpython2.7.so
[armeabi] Compile thumb  : square <= square.c
[armeabi] SharedLibrary  : libsquare.so
[armeabi] Install        : libsquare.so => libs/armeabi/libsquare.so

Now you should have the libraries in the "libs/armeabi" folder.

Testing

Open "MainActivity.java" and add the following code to the "onCreate()" method.

public class MainActivity extends ActionBarActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        (...)

        PythonWrapper.start();
        int sq = PythonWrapper.square(3);
        Log.i("PythonWrapper Square", String.format("%d", sq));
        PythonWrapper.end();
    }

    (...)
}

Build and run the application, and if everything is correct, you should see something like this in your logcat:

I/pyjni   ( 9035): Initializing the Python interpreter
I/pyjni   ( 9035): Squaring number in the Python interpreter
I/PythonWrapper Square( 9035): 9
I/pyjni   ( 9035): Finalizing the Python interpreter
I/ActivityManager(  198): Displayed com.example.pytest/.MainActivity: +201ms

Congratulations! You now have an embedded bare-bones Python interpreter and can access it in Android. It does only add about 3 MB to your APK, but it doesn't have much functionality without the standard library. Next, we will add some modules from the standard library, so we can use more of Python on Android.