1# Travis Testing 2 3Unbound 1.10 and above leverage Travis CI to increase coverage of compilers and platforms. Compilers include Clang and GCC; while platforms include Android, iOS, Linux, and OS X on AMD64, Aarch64, PowerPC and s390x hardware. 4 5Android is tested on armv7a, aarch64, x86 and x86_64. The Android recipes build and install OpenSSL and Expat, and then builds Unbound. The testing is tailored for Android NDK-r19 and above, and includes NDK-r20 and NDK-r21. Mips and Mips64 are not tested because they are no longer supported under current NDKs. 6 7iOS is tested for iPhoneOS, WatchOS, AppleTVOS, iPhoneSimulator, AppleTVSimulator and WatchSimulator. The testing uses Xcode 10 on OS X 10.13. 8 9The Unbound Travis configuration file `.travis.yml` does not use top-level keys like `os:` and `compiler:` so there is no matrix expansion. Instead Unbound specifies the exact job to run under the `jobs:` and `include:` keys. 10 11## Typical recipe 12 13A typical recipe tests Clang and GCC on various hardware. The hardware includes AMD64, Aarch64, PowerPC and s390x. PowerPC is a little-endian platform, and s390x is a big-endian platform. There are pairs of recipes that are similar to the following. 14 15``` 16- os: linux 17 name: GCC on Linux, Aarch64 18 compiler: gcc 19 arch: arm64 20 dist: bionic 21- os: linux 22 name: Clang on Linux, Aarch64 23 compiler: clang 24 arch: arm64 25 dist: bionic 26``` 27 28OS X provides a single recipe to test Clang. GCC is not tested because GCC is an alias for Clang. 29 30## Sanitizer builds 31 32Two sanitizer builds are tested using Clang and GCC, for a total of four builds. The first sanitizer is Undefined Behavior sanitizer (UBsan), and the second is Address sanitizer (Asan). The sanitizers are only run on AMD64 hardware. Note the environment includes `TEST_UBSAN=yes` or `TEST_ASAN=yes` for the sanitizer builds. 33 34The recipes are similar to the following. 35 36``` 37- os: linux 38 name: UBsan, GCC on Linux, Amd64 39 compiler: gcc 40 arch: amd64 41 dist: bionic 42 env: TEST_UBSAN=yes 43- os: linux 44 name: UBsan, Clang on Linux, Amd64 45 compiler: clang 46 arch: amd64 47 dist: bionic 48 env: TEST_UBSAN=yes 49``` 50 51When the Travis script encounters a sanitizer it uses different `CFLAGS` and configuration string. 52 53``` 54if [ "$TEST_UBSAN" = "yes" ]; then 55 export CFLAGS="-DNDEBUG -g2 -O3 -fsanitize=undefined -fno-sanitize-recover" 56 ./configure 57 make -j 2 58 make test 59elif [ "$TEST_ASAN" = "yes" ]; then 60 export CFLAGS="-DNDEBUG -g2 -O3 -fsanitize=address" 61 ./configure 62 make -j 2 63 make test 64... 65``` 66 67## Android builds 68 69Travis tests Android builds for the armv7a, aarch64, x86 and x86_64 architectures. The builds are trickier than other builds for several reasons. The testing requires installation of the Android NDK and SDK, it requires a cross-compile, and requires OpenSSL and Expat prerequisites. The Android cross-compiles also require care to set the Autotools triplet, the OpenSSL triplet, the toolchain path, the tool variables, and the sysroot. The discussion below detail the steps of the Android recipes. 70 71### Android job 72 73The first step sets environmental variables for the cross-compile using the Travis job. A typical job with variables is shown below. 74 75``` 76- os: linux 77 name: Android armv7a, Linux, Amd64 78 compiler: clang 79 arch: amd64 80 dist: bionic 81 env: 82 - TEST_ANDROID=yes 83 - AUTOTOOLS_HOST=armv7a-linux-androideabi 84 - OPENSSL_HOST=android-arm 85 - ANDROID_CPU=armv7a 86 - ANDROID_API=23 87 - ANDROID_PREFIX="$HOME/android$ANDROID_API-$ANDROID_CPU" 88 - ANDROID_SDK_ROOT="$HOME/android-sdk" 89 - ANDROID_NDK_ROOT="$HOME/android-ndk" 90``` 91 92### ANDROID_NDK_ROOT 93 94The second step for Android is to set the environmental variables `ANDROID_NDK_ROOT` and `ANDROID_SDK_ROOT`. This is an important step because the NDK and SDK use the variables internally to locate their own tools. Also see [Recommended NDK Directory?](https://groups.google.com/forum/#!topic/android-ndk/qZjhOaynHXc) on the android-ndk mailing list. (Many folks miss this step, or use incorrect variables like `ANDROID_NDK_HOME` or `ANDROID_SDK_HOME`). 95 96If you are working from a developer machine you probably already have the necessary tools installed. You should ensure `ANDROID_NDK_ROOT` and `ANDROID_SDK_ROOT` are set properly. 97 98### Tool installation 99 100The second step installs tools needed for OpenSSL, Expat and Unbound. This step is handled in by the script `contrib/android/install_tools.sh`. The tools include curl, tar, zip, unzip and java. 101 102``` 103before_script: 104 - | 105 if [ "$TEST_ANDROID" = "yes" ]; then 106 ./contrib/android/install_tools.sh 107 elif [ "$TEST_IOS" = "yes" ]; then 108 ./contrib/ios/install_tools.sh 109 fi 110``` 111 112### NDK installation 113 114The third step installs the NDK and SDK. This step is handled in by the script `contrib/android/install_ndk.sh`. The script uses `ANDROID_NDK_ROOT` and `ANDROID_SDK_ROOT` to place the NDK and SDK in the `$HOME` directory. 115 116If you are working from a developer machine you probably already have a NDK and SDK installed. 117 118### Android environment 119 120The fourth step sets the Android cross-compile environment using the script `contrib/android/setenv_android.sh`. The script is `sourced` so the variables in the script are available to the calling shell. The script sets variables like `CC`, `CXX`, `AS` and `AR`; sets `CFLAGS` and `CXXFLAGS`; sets a `sysroot` so Android headers and libraries are found; and adds the path to the toolchain to `PATH`. 121 122`contrib/android/setenv_android.sh` knows which toolchain and architecture to select by inspecting environmental variables set by Travis for the job. In particular, the variables `ANDROID_CPU` and `ANDROID_API` tell `contrib/android/setenv_android.sh` which tools and libraries to select. 123 124The `contrib/android/setenv_android.sh` script specifies the tools in a `case` statement like the following. There is a case for each of the architectures armv7a, aarch64, x86 and x86_64. 125 126``` 127armv8a|aarch64|arm64|arm64-v8a) 128 CC="aarch64-linux-android$ANDROID_API-clang" 129 CXX="aarch64-linux-android$ANDROID_API-clang++" 130 LD="aarch64-linux-android-ld" 131 AS="aarch64-linux-android-as" 132 AR="aarch64-linux-android-ar" 133 RANLIB="aarch64-linux-android-ranlib" 134 STRIP="aarch64-linux-android-strip" 135 136 CFLAGS="-funwind-tables -fexceptions" 137 CXXFLAGS="-funwind-tables -fexceptions -frtti" 138``` 139 140### OpenSSL and Expat 141 142The fifth step builds OpenSSL and Expat. OpenSSL and Expat are built for Android using the scripts `contrib/android/install_openssl.sh` and `contrib/android/install_expat.sh`. The scripts download, configure and install the latest release version of the libraries. The libraries are configured with `--prefix="$ANDROID_PREFIX"` so the headers are placed in `$ANDROID_PREFIX/include` directory, and the libraries are placed in the `$ANDROID_PREFIX/lib` directory. 143 144`ANDROID_PREFIX` is the value `$HOME/android$ANDROID_API-$ANDROID_CPU`. The libraries will be installed in `$HOME/android23-armv7a`, `$HOME/android23-aarch64`, etc. For Autotools projects, the appropriate `PKG_CONFIG_PATH` is exported. `PKG_CONFIG_PATH` is the userland equivalent to sysroot, and allows Autotools to find non-system headers and libraries for an architecture. Typical `PKG_CONFIG_PATH` are `$HOME/android23-armv7a/lib/pkgconfig` and `$HOME/android23-aarch64/lib/pkgconfig`. 145 146OpenSSL also uses a custom configuration file called `15-android.conf`. It is a copy of the OpenSSL's project file and located at `contrib/android/15-android.conf`. The Unbound version is copied to the OpenSSL source files after unpacking the OpenSSL distribution. The Unbound version has legacy NDK support removed and some other fixes, like `ANDROID_NDK_ROOT` awareness. The changes mean Unbound's `15-android.conf` will only work with Unbound, with NDK-r19 and above, and a properly set environment. 147 148OpenSSL is configured with `no-engine`. If you want to include OpenSSL engines then edit `contrib/android/install_openssl.sh` and remove the config option. 149 150### Android build 151 152Finally, once OpenSSL and Expat are built, then the Travis script configures and builds Unbound. The recipe looks as follows. 153 154``` 155elif [ "$TEST_ANDROID" = "yes" ]; then 156 export AUTOTOOLS_BUILD="$(./config.guess)" 157 export PKG_CONFIG_PATH="$ANDROID_PREFIX/lib/pkgconfig" 158 ./contrib/android/install_ndk.sh 159 source ./contrib/android/setenv_android.sh 160 ./contrib/android/install_openssl.sh 161 ./contrib/android/install_expat.sh 162 ./configure \ 163 --build="$AUTOTOOLS_BUILD" \ 164 --host="$AUTOTOOLS_HOST" \ 165 --prefix="$ANDROID_PREFIX" \ 166 --with-ssl="$ANDROID_PREFIX" \ 167 --with-libexpat="$ANDROID_PREFIX" \ 168 --disable-gost; 169 make -j 2 170 make install 171``` 172 173Travis only smoke tests an Android build using a compile, link and install. The self tests are not run. TODO: figure out how to fire up an emulator, push the tests to the device and run them. 174 175### Android flags 176 177`contrib/android/setenv_android.sh` uses specific flags for `CFLAGS` and `CXXFLAGS`. They are taken from `ndk-build`, so we consider them the official flag set. It is important to use the same flags across projects to avoid subtle problems due to mixing and matching different flags. 178 179`CXXFLAGS` includes `-fexceptions` and `-frtti` because exceptions and runtime type info are disabled by default. `CFLAGS` include `-funwind-tables` and `-fexceptions` to ensure C++ exceptions pass through C code, if needed. Also see `docs/CPLUSPLUS-SUPPORT.html` in the NDK docs. 180 181To inspect the flags used by `ndk-build` for a platform clone ASOP's [ndk-samples](https://github.com/android/ndk-samples/tree/master/hello-jni) and build the `hello-jni` project. Use the `V=1` flag to see the full compiler output from `ndk-build`. 182 183## iOS builds 184 185Travis tests iOS builds for the armv7a, armv7s and aarch64 architectures for iPhoneOS, AppleTVOS and WatchOS. iPhoneOS is tested using both 32-bit builds (iPhones) and 64-bit builds (iPads). Travis also tests compiles against the simulators. The builds are trickier than other builds for several reasons. The testing requires a cross-compile, and requires OpenSSL and Expat prerequisites. The iOS cross-compiles also require care to set the Autotools triplet, the OpenSSL triplet, the toolchain path, the tool variables, and the sysroot. The discussion below detail the steps of the iOS recipes. 186 187### iOS job 188 189The first step sets environmental variables for the cross-compile using the Travis job. A typical job with variables is shown below. 190 191``` 192- os: osx 193 osx_image: xcode10 194 name: Apple iPhone on iOS, armv7 195 compiler: clang 196 env: 197 - TEST_IOS=yes 198 - AUTOTOOLS_HOST=armv7-apple-ios 199 - OPENSSL_HOST=ios-cross 200 - IOS_SDK=iPhoneOS 201 - IOS_CPU=armv7s 202 - IOS_PREFIX="$HOME/$IOS_SDK-$IOS_CPU" 203``` 204 205### Tool installation 206 207The second step installs tools needed for OpenSSL, Expat and Unbound. This step is handled in by the script `contrib/ios/install_tools.sh`. The tools include autotools, curl and perl. The installation happens at the `before_script:` stage of Travis. 208 209``` 210before_script: 211 - | 212 if [ "$TEST_ANDROID" = "yes" ]; then 213 ./contrib/android/install_tools.sh 214 elif [ "$TEST_IOS" = "yes" ]; then 215 ./contrib/ios/install_tools.sh 216 fi 217``` 218 219### iOS environment 220 221The third step sets the iOS cross-compile environment using the script `contrib/ios/setenv_ios.sh`. The script is `sourced` so the variables in the script are available to the calling shell. The script sets variables like `CC`, `CXX`, `AS` and `AR`; sets `CFLAGS` and `CXXFLAGS`; sets a `sysroot` so iOS headers and libraries are found; and adds the path to the toolchain to `PATH`. 222 223`contrib/ios/setenv_ios.sh` knows which toolchain and architecture to select by inspecting environmental variables set by Travis for the job. In particular, the variables `IOS_SDK` and `IOS_CPU` tell `contrib/ios/setenv_ios.sh` which tools and libraries to select. 224 225The `contrib/ios/setenv_ios.sh` script specifies the tools to use during the cross-compile. For Apple SDKs, the tool names are the same as a desktop. There are no special prefixes for the mobile tools. 226 227``` 228CPP=cpp 229CC=clang 230CXX=clang++ 231LD=ld 232AS=as 233AR=ar 234RANLIB=ranlib 235STRIP=strip 236``` 237 238If you are working from a developer machine you probably already have the necessary tools installed. 239 240### OpenSSL and Expat 241 242The fourth step builds OpenSSL and Expat. OpenSSL and Expat are built for iOS using the scripts `contrib/ios/install_openssl.sh` and `contrib/ios/install_expat.sh`. The scripts download, configure and install the latest release version of the libraries. The libraries are configured with `--prefix="$IOS_PREFIX"` so the headers are placed in `$IOS_PREFIX/include` directory, and the libraries are placed in the `$IOS_PREFIX/lib` directory. 243 244`IOS_PREFIX` is the value `$HOME/$IOS_SDK-$IOS_CPU`. The scheme handles both iOS SDKs and cpu architectures so the pair receives a unique installation directory. The libraries will be installed in `$HOME/iPhoneOS-armv7s`, `$HOME/iPhoneOS-arm64`, `$HOME/iPhoneSimulator-i386`, etc. For Autotools projects, the appropriate `PKG_CONFIG_PATH` is exported. 245 246`PKG_CONFIG_PATH` is an important variable. It is the userland equivalent to sysroot, and allows Autotools to find non-system headers and libraries for an architecture. Typical `PKG_CONFIG_PATH` are `$HOME/iPhoneOS-armv7s/lib/pkgconfig` and `$HOME/iPhoneOS-arm64/lib/pkgconfig`. 247 248OpenSSL also uses a custom configuration file called `15-ios.conf`. It is a copy of the OpenSSL's project file and located at `contrib/ios/15-ios.conf`. The Unbound version is copied to the OpenSSL source files after unpacking the OpenSSL distribution. The changes mean Unbound's `15-ios.conf` will only work with Unbound and a properly set environment. 249 250OpenSSL is configured with `no-engine`. Engines require dynamic loading so engines are disabled permanently in `15-ios.conf`. 251 252### iOS build 253 254Finally, once OpenSSL and Expat are built, then the Travis script configures and builds Unbound. The full recipe looks as follows. 255 256``` 257elif [ "$TEST_IOS" = "yes" ]; then 258 export AUTOTOOLS_BUILD="$(./config.guess)" 259 export PKG_CONFIG_PATH="$IOS_PREFIX/lib/pkgconfig" 260 source ./contrib/ios/setenv_ios.sh 261 ./contrib/ios/install_openssl.sh 262 ./contrib/ios/install_expat.sh 263 ./configure \ 264 --build="$AUTOTOOLS_BUILD" \ 265 --host="$AUTOTOOLS_HOST" \ 266 --prefix="$IOS_PREFIX" \ 267 --with-ssl="$IOS_PREFIX" \ 268 --with-libexpat="$IOS_PREFIX" \ 269 --disable-gost; 270 make -j 2 271 make install 272``` 273 274Travis only smoke tests an iOS build using a compile, link and install. The self tests are not run. TODO: figure out how to fire up an simulator, push the tests to the device and run them. 275 276### iOS flags 277 278`contrib/ios/setenv_ios.sh` uses specific flags for `CFLAGS` and `CXXFLAGS`. They are taken from Xcode, so we consider them the official flag set. It is important to use the same flags across projects to avoid subtle problems due to mixing and matching different flags. 279