Android (Command Line)

From Crypto++ Wiki
Jump to: navigation, search

This page will provide instructions for cross-compiling Crypto++ on the command line for Android. The instructions below provide a treatment of building from the command line with a script that sets the environment and a makefile tuned for cross-compiles. If you want to try an Android.mk, then see Android.mk below.

There are four steps to building Crypto++ for Android, and the process will create an Android version of cryptest.exe, the dynamic library, and the static library. After building the library, you can push cryptest.exe and the test vectors to the device, and then execute the tests using a remote shell.

While you can use the Crypto++ shared object directly though JNI, you will likely provide an application specific shared object with C-style external interfaces which depends upon the static Crypto++ library. The C-style interface will make your shared object easy to call using JNI; while the Crypto++ static library will allow you to aggressively strip unused code.

The instructions below are used in the project's release engineering process. Crypto++ is regularly tested with the following devices: HTC Evo 4G, Dual Core Cortex-A9 (ARM), Android 4.2; LG Nexus 5, Quad Core Snapdragon (ARM), CyanogenMod 11; Digiland DL700 tablet, Dual Core A23 (ARM), Android 4.4; LeMaker HiKey Octa-Core Cortex-A53 (ARM64), Debian 8.

Note: the instructions below changed with the release of Crypto++ 5.6.3. At Crypto++ 5.6.3, the patches that used to be required were added to the library, so you won't need to apply them. setenv-android.sh was also added to the library sources in March 2016. It is available in Git and library ZIP files after 5.6.3. Also see Commit 22d6374ce7151ad0, Added Android and embedded environment and test script.

Set the Environment

Cryptopp-android-100.png
Before you begin you must set the cross-compilation environment. Setting the environment will do two things. First, it will ensure the Android toolchain is on-path. Second, it will ensure some environmental variables are set so the makefile picks up specific device flags. For example, the script will set IS_ANDROID, AOSP_SYSROOT, AOSP_STL_INC and AOSP_STL_LIB.

You must set ANDROID_NDK_ROOT prior to running the script. You should also set ANDROID_SDK_ROOT for that matter, though its not used for cross-compiling. Android recommends setting both ANDROID_NDK_ROOT and ANDROID_SDK_ROOT because the NDK and SDK use the variables internally. For details, see Recommended NDK Directory?.

Perform the following to run the script. The source command is required to make changes to the current shell.

$ source setenv-android.sh 
ANDROID_NDK_ROOT: /opt/android-ndk
AOSP_TOOLCHAIN_PATH: /opt/android-ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin
AOSP_ABI: armeabi-v7a
AOSP_API: android-21
AOSP_SYSROOT: /opt/android-ndk/platforms/android-21/arch-arm
AOSP_FLAGS: -march=armv7-a -mfpu=neon -mfloat-abi=softfp -Wl,--fix-cortex-a8 -funwind-tables -fexceptions -frtti
AOSP_STL_INC: /opt/android-ndk/sources/cxx-stl/gnu-libstdc++/4.9/include
AOSP_STL_LIB: /opt/android-ndk/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/libgnustl_shared.so
AOSP_BITS_INC: /opt/android-ndk/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/include
CPU FEATURES: cpu-features.h and cpu-features.c are present

setenv-android.sh takes two optional arguments. The first optional argument is the architecture. The default is armeabi-v7a. Architectures can be one of:

  • armeabi
  • armeabi-v7a
  • armeabi-v7a-hard
  • arm64-v8a
  • mips
  • mips64
  • x86
  • x86_64

The second optional argument is a STL library. The default is the GNU runtime and STL library as a shared library. STL libraries can be one of the following:

  • gnu-static
  • gnu-shared
  • llvm-static
  • llvm-shared
  • stlport-static
  • stlport-shared

There are three caveats when selecting runtime and STL libraries. First, using GNU's runtime and STL library has severe licensing requirements. Its not uncommon to perform contract work with clauses that forbid any GNU licensed components.

Second, all projects must use the same runtime and STL libraries. You cannot use STLport for Crypto++ and then use the resulting binary in a Qt project. Qt uses GNU's runtime and STL library, and mixing and matching of runtime libraries causes obscure memory errors.

Third, if more than one library requires the C++ runtime or STL library, then you must use the dynamic version of the library. For example, if you are using Qt and Crypto++ and they depend on the GNU gear, then you must use libgnustl_shared.so.


By default, the script uses the ARMv7a eabi, GNU runtime in a shared library configuration and the Android 21 API (Android 5.0). If you want to use STLport rather than GNU's C++ runtime library, then you have to specify both an architecture and runtime. Below are some additional ways to invoke the script.

# Build for Android 4.0
AOSP_API="android-14" source setenv-android.sh

# Build for Android 4.4
AOSP_API="android-19" source setenv-android.sh

# Build for ARM with hard floats
$ source setenv-android.sh hard

# Build for ARM with NEON
$ source setenv-android.sh neon

# Build for ARM64 (AARCH64)
$ source setenv-android.sh arm64

# Build for ARM64 (again)
$ source setenv-android.sh armv8a

# Build for MIPS
$ source setenv-android.sh mips

# Build for MIPS64
$ source setenv-android.sh mips64

# Build using GCC 4.8 toolchain
$ $ AOSP_TOOLCHAIN_SUFFIX="4.8" source setenv-android.sh

Build the Library

Building the library consists of running the following command after setting the environment:

$ make -f GNUmakefile-cross static shared
arm-linux-androideabi-g++ -DNDEBUG -g2 -O3 -fPIC -pipe -march=armv7-a -mfpu=neon
 -mfloat-abi=softfp -Wl,--fix-cortex-a8 -funwind-tables -fexceptions -frtti -DAN
DROID --sysroot=/opt/android-ndk/platforms/android-21/arch-arm -Wa,--noexecstack
 -I/opt/android-ndk/sources/cxx-stl/gnu-libstdc++/4.9/include -I/opt/android-ndk
/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/include -c cryptlib.cpp
arm-linux-androideabi-g++ -DNDEBUG -g2 -O3 -fPIC -pipe -march=armv7-a -mfpu=neon
 -mfloat-abi=softfp -Wl,--fix-cortex-a8 -funwind-tables -fexceptions -frtti -DAN
DROID --sysroot=/opt/android-ndk/platforms/android-21/arch-arm -Wa,--noexecstack
 -I/opt/android-ndk/sources/cxx-stl/gnu-libstdc++/4.9/include -I/opt/android-ndk
/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/include -c cpu.cpp
arm-linux-androideabi-g++ -DNDEBUG -g2 -O3 -fPIC -pipe -march=armv7-a -mfpu=neon
 -mfloat-abi=softfp -Wl,--fix-cortex-a8 -funwind-tables -fexceptions -frtti -DAN
DROID --sysroot=/opt/android-ndk/platforms/android-21/arch-arm -Wa,--noexecstack
 -I/opt/android-ndk/sources/cxx-stl/gnu-libstdc++/4.9/include -I/opt/android-ndk
/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/include -c integer.cpp
...

You can verify the library was built for the correct architecture with the following.

$ find . -name cryptest.exe
./cryptest.exe
$ readelf -h ./cryptest.exe | grep -i 'class\|machine'
  Class:                             ELF32
  Machine:                           ARM
$
$ find . -name libcryptopp.so
./libcryptopp.so
$ readelf -h ./libcryptopp.so | grep -i 'class\|machine'
  Class:                             ELF32
  Machine:                           ARM

You can run ndk-depends to verify dependencies:

$ $ANDROID_NDK_ROOT/ndk-depends libcryptopp.so
libcryptopp.so
libstdc++.so
libm.so
libgnustl_shared.so
libdl.so
libc.so

If you want to use SOLIB versioning, then you must include HAS_SOLIB_VERSION=1 in the make command. For example:

$ make -f GNUmakefile-cross lean HAS_SOLIB_VERSION=1

Strip the Library

To perform dead-code stripping, be sure to link your final executable with -Wl,--gc-sections. If you link your final application against libcryptopp.a then you will see significant gains. If you link your final application against libcryptopp.so then you will experience a small gains.

You can also strip debug symbols from your final executable, but it will make back traces useless. To do so, you can strip by running arm-linux-androideabi-strip with --strip-debug, --strip-unneeded or --strip-all. arm-linux-androideabi-strip is available on path after running setenv-android.sh. See arm-eabi-strip and binary size for details.

$ ls -al cryptest.exe
-rwxr-xr-x  1 jwalton  staff 42860268 Jul 29 00:41 cryptest.exe
$
$ arm-linux-androideabi-strip --strip-debug cryptest.exe
$ ls -al cryptest.exe
-rwxr-xr-x  1 jwalton  staff  6112963 Jul 29 00:41 cryptest.exe
$
$ arm-linux-androideabi-strip --strip-all cryptest.exe
$ ls -al cryptest.exe
-rwxr-xr-x  1 jwalton  staff  2903712 Jul 29 00:41 cryptest.exe
$ ls -al libcryptopp.so 
-rwxr-xr-x  1 jwalton  staff  28668308 Jul 29 00:52 libcryptopp.so
$ 
$ arm-linux-androideabi-strip --strip-debug libcryptopp.so 
$ ls -al libcryptopp.so 
-rwxr-xr-x  1 jwalton  staff  5478765 Jul 29 00:52 libcryptopp.so
$ 
$ arm-linux-androideabi-strip --strip-all libcryptopp.so 
$ ls -al libcryptopp.so 
-rwxr-xr-x  1 jwalton  staff  3808428 Jul 29 00:52 libcryptopp.so

Additionally, there are other tricks for modest gains. See Android NDK: How to Reduce Binaries Size for details.

Execute the Program

To test on the device, you will use adb to push cryptest.exe and the test vectors to the device. Once on a device, the program will be executed with the remote shell. To begin, ensure Android's platform-tools are on path (i.e., ANDROID_SDK_ROOT/platform-tools), and your device is recognized:

$ adb devices
List of devices attached 
0380614543dfd297	device
Cryptopp-android-103.png
With a device available, push cryptest.exe, TestData, and TestVectors to /data/local/tmp.
$ adb push cryptest.exe /data/local/tmp
1295 KB/s (43249212 bytes in 32.591s)
$
$ adb push TestData /data/local/tmp/TestData
push: TestData/xtrdh342.dat -> /data/local/tmp/TestData/xtrdh342.dat
push: TestData/xtrdh171.dat -> /data/local/tmp/TestData/xtrdh171.dat
push: TestData/usage.dat -> /data/local/tmp/TestData/usage.dat
...
55 files pushed. 0 files skipped.
471 KB/s (57449 bytes in 0.119s)
$
$ adb push TestVectors /data/local/tmp/TestVectors
push: TestVectors/whrlpool.txt -> /data/local/tmp/TestVectors/whrlpool.txt
push: TestVectors/wake.txt -> /data/local/tmp/TestVectors/wake.txt
push: TestVectors/vmac.txt -> /data/local/tmp/TestVectors/vmac.txt
...
33 files pushed. 0 files skipped.
3643 KB/s (1314149 bytes in 0.352s)

If you linked against libstlport_shared.so or libc++_shared.so, then you will need to push it also:

$ adb push "$AOSP_STL_LIB" /data/local/tmp
3261 KB/s (3547204 bytes in 1.062s)

After pushing, open a remote shell with adb shell and verify all files are present on the device:

$ adb shell
shell@android: $ ls -l /data/local/tmp                                       
drwxrwxr-x shell     shell              2015-12-07 00:37 TestData
drwxrwxr-x shell     shell              2015-12-07 00:38 TestVectors
-rwxrwxrwx shell     shell     26930220 2015-12-07 00:36 cryptest.exe
-rwxrwxrwx shell     shell      5630724 2014-10-17 02:40 libgnustl_shared.so
-rwxrwxrwx shell     shell      3547204 2014-12-10 05:01 libstlport_shared.so
Cryptopp-android-105.png
With the program and test vectors on the device, execute the test program:
shell@android:/ $ cd /data/local/tmp
shell@android:/data/local/tmp $ ./cryptest.exe v
Using seed: 1374978640

Testing Settings...

passed:  Your machine is little endian.
passed:  CRYPTOPP_ALLOW_UNALIGNED_DATA_ACCESS is not defined. Will restrict to aligned data access.
passed:  sizeof(byte) == 1
passed:  sizeof(word16) == 2
passed:  sizeof(word32) == 4
passed:  sizeof(word64) == 8
passed:  sizeof(hword) == 2, sizeof(word) == 4, sizeof(dword) == 8
...

If you linked against libstlport_shared.so or libc++_shared.so, then you will need to modify LD_LIBRARY_PATH when running cryptest.exe:

shell@android:/ $ cd /data/local/tmp
shell@android:/data/local/tmp $ LD_LIBRARY_PATH=./; ./cryptest.exe v
Using seed: 1374994654

Testing Settings...
...

If you link to both libstlport_shared.so and libcryptopp.so, then you will have to preload libstlport_shared.so:

shell@android:/ $ cd /data/local/tmp
shell@android:/data/local/tmp $ LD_LIBRARY_PATH=./; ./cryptest.exe v
link_image[1936]:  5431 could not load needed library './libcryptopp.so' for './cryptest.exe'
(reloc_library[1285]:  5431 cannot locate '_ZNSt12__node_alloc11_M_allocateERj'...)
CANNOT LINK EXECUTABLE
shell@android:/data/local/tmp $
shell@android:/data/local/tmp $ export LD_PRELOAD=`pwd`/libstlport_shared.so
shell@android:/data/local/tmp $ LD_LIBRARY_PATH=./; ./cryptest.exe v
Using seed: 1375005963

Testing Settings...
...

If all goes well, the tail of cryptest.exe v will display "All tests passed!" as shown below.

Cryptopp-android-106.png

Install the Library

Once the library has been verified on a device, copy the header files (*.h), the dynamic library (libcryptopp.so), and the static library (libcryptopp.a) to a directory for use in other projects. You can run sudo make install PREFIX=/usr/local/cryptopp/android-armeabi-v7a since the libraries are for API 21 and ARMv7 architecture.

Under this scheme, headers will be found at /usr/local/cryptopp/android-armeabi-v7a/include, and the libraries will be found at /usr/local/cryptopp/android-armeabi-v7a/lib. You can use the following with ndk-build. ndk-build will set $(TARGET_ARCH_ABI):

  • CRYPTOPP_INCL := /usr/local/cryptopp/android-$(TARGET_ARCH_ABI)/include
  • CRYPTOPP_LIB  := /usr/local/cryptopp/android-$(TARGET_ARCH_ABI)/lib

All Architectures

If you need to build all the architectures and install them, then the following script will accomplish it for you. Place the script in the root of the Crypto++ folder. Prior to running the script, ensure both ANDROID_NDK_ROOT and ANDROID_SDK_ROOT are set.

$ cat build-all-android.sh 
#!/usr/bin/env bash

for arch in armeabi armeabi-v7a armeabi-v7a-hard arm64-v8a mips mips64 x86 x86_64
do
    source setenv-android.sh $arch
    if [ "$?" -eq "0" ]; then
        make -f GNUmakefile-cross distclean
        make -f GNUmakefile-cross static dynamic
        sudo make -f GNUmakefile-cross install PREFIX=/usr/local/cryptopp/android-$arch
    fi
done

To build arm64-v8a and other new additions, you will need the NDK r10e, and you will need to use the Android 21 API. The script uses them by default.

Loading Shared Objects

Crypto++ or your shared object wrapper can be loaded in a number of different ways on Android. Development environments, like QT, can further complicate matters because of the way it calls dlopen. If you are experiencing an UnsatisfiedLinkError: dlopen failed: library "./obj/local/armeabi-v7a/libcryptopp.so", then please see the following:

Android.mk

Android provides a modified makefile based build system. If you are interested in an Android.mk to use with ndk-build, Eclipse, or Android Sudio, then see Pull Request 3, Add Android.mk to build using android NDK. Thanks to Alex Afanasyev for providing it.

An Android.mk has not been added to the Crypto++ sources because there are some open questions about using it to safely build static archives and shared objects. See, for example, No archive symbol table (run ranlib) while building libcryptopp.a through ndk-build and How to run ranlib on an archive built through Android.mk?

Android Activity

Cryptopp-android-prng.png
You can find an example of using prebuilt Crypto++ shared objects on the wiki at Android Activity. Android Activity uses a sample project called Android-PRNG, and it demonstrates two topics. First, it shows you how to integrate a prebuilt Crypto++ library shared object into a Ant-based or NDK-based project. It may work for Android Studio, too. It may even work for Eclipse, depending on how well it imports an Android java project with JNI.

Second, it shows you how to accumulate seed data from sensors that can be used to seed your software based random number generators. A generator should have a fresh seed applied with IncorporateEntropy before each call to GenerateBlock to ensure the fitness of the generator.

If you are using Android Studio, then you may need to add the following. Also see JNI and Gradle in Android Studio on Stack Overflow.

sourceSets.main {
    jni.srcDirs = []
    jniLibs.srcDir 'src/main/libs'
}

Keep in mind you must statically load the libraries in the order that satisfies link dependencies. An example of doing that is:

public class MyClass {
    
    static {
        System.loadLibrary("stlport_shared");
        System.loadLibrary("cryptopp");
        System.loadLibrary("mylib");
    }
    
    ...
}

NDK CXXFLAGS

The following lists Android's default CXXFLAGS used by default in a NDK build. Options like -DNDEBUG, -Os, -g and -fmessage-length=0 were omitted. Some flags, like -fexception, must be used to ensure exceptions pass through C routines. Other flags must not be used, like -fno-exceptions and -fno-rtti.

armeabi

  • -MMD -MP -MF -fpic -ffunction-sections -funwind-tables -fstack-protector -march=armv5te -mtune=xscale -msoft-float -mthumb -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -DANDROID -Wa,--noexecstack -Wformat -Werror=format-security
  • Do not use: -fno-exceptions -fno-rtti

armeabi-v7a

  • -MMD -MP -MF -fpic -ffunction-sections -funwind-tables -fstack-protector -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -mthumb -Wl,--fix-cortex-a8 -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -DANDROID -Wa,--noexecstack -Wformat -Werror=format-security
  • Do not use: -fno-exceptions -fno-rtti

arm64-v8a

  • -MMD -MP -MF -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -fomit-frame-pointer -fstrict-aliasing -funswitch-loops -finline-limit=300 -DANDROID -Wa,--noexecstack -Wformat -Werror=format-security
  • Do not use: -fno-exceptions -fno-rtti

mips

  • -MMD -MP -MF -fpic -fno-strict-aliasing -finline-functions -ffunction-sections -funwind-tables -fno-inline-functions-called-once -fgcse-after-reload -frerun-cse-after-loop -frename-registers -fomit-frame-pointer -funswitch-loops -finline-limit=300 -DANDROID -Wa,--noexecstack -Wformat -Werror=format-security
  • Do not use: -fno-exceptions -fno-rtti

mips64

  • -MMD -MP -MF -fpic -fno-strict-aliasing -finline-functions -ffunction-sections -funwind-tables -fno-inline-functions-called-once -fgcse-after-reload -frerun-cse-after-loop -frename-registers -fomit-frame-pointer -funswitch-loops -finline-limit=300 -DANDROID -Wa,--noexecstack -Wformat -Werror=format-security
  • Do not use: -fno-exceptions -fno-rtti

x86

  • -MMD -MP -MF -ffunction-sections -funwind-tables -fstack-protector -fomit-frame-pointer -fstrict-aliasing -funswitch-loops -finline-limit=300 -DANDROID -Wa,--noexecstack -Wformat -Werror=format-security
  • Do not use: -fno-exceptions -fno-rtti

x86_64

  • -MMD -MP -MF -ffunction-sections -funwind-tables -fstack-protector-strong -fomit-frame-pointer -fstrict-aliasing -funswitch-loops -finline-limit=300 -DANDROID -Wa,--noexecstack -Wformat -Werror=format-security
  • Do not use: -fno-exceptions -fno-rtti

Downloads

setenv-android.sh.zip - Script to set the environment for Android cross-compiles. Added to the library at Crypto++ 5.6.3.

GNUmakefile-cross.zip - Makefile for cross-compiling the library. Added to the library at Crypto++ 5.6.3.