KBEA-00130 - Best Practices for Android Builds Using CloudBees Build Acceleration 10.0 and Older Versions

Article ID:360032824992
9 minute readKnowledge base

This KB article applies to CloudBees Build Acceleration versions 9.0 through 10.0. For information about best practices for 10.1 and newer versions, see KBEA-00165 - Best Practices for Android Builds Using CloudBees Build Acceleration 10.1 and Newer Versions.

Overview

This article describes the best practices for running Android builds. It includes descriptions of the CloudBees Build Acceleration template files that you can customize to optimize your builds.
This article contains the following topics:

For the fastest Android builds, you should have at least 16 Accelerator agents.

RAM Requirements for Agent Hosts

The recommended total amount of RAM for an agent host is at least 2 GB per CloudBees Build Acceleration agent plus the amount of RAM normally needed by your build. For example, if you run four agents and your build normally needs 16 GB, you need ((2 * 4) + 16) = 24 GB.

Template Files for Build Optimization

Accelerator provides a set of files that work together to provide optimal settings for Android 6.x (Android M), Android 7.x (Android N), and Android 8.x (Android O). You customize these files for your builds.

For Accelerator 9.0, these files are in the attached unsupported_conf.tgz file. For Accelerator 9.0.1 and higher versions, these files are also in the /opt/ecloud/i686_Linux/unsupported/conf directory.

eadroid.sh

eadroid.sh is a bash script that runs your build. Three versions are provided: one each for Android M, N, and O.

The top few lines of this file specify details such as the locations of your Android source code and your history file. Following are the top few lines in the template file for Android N:

   ...

   # Edit these:
   BUILD_KEY=androidn_1
   ANDROID_SOURCE="/c/src/android/AndroidN"
   HISTORY="$ANDROID_SOURCE/emake.data"
   JAVA_DIR="/c/src/android/jdk1.8.0_72" # Android N
   LOGS_DIR="$ANDROID_SOURCE/logs" # must exist already

   ...
  • BUILD_KEY distinguishes this build from other builds so that log files and annotation files have build-specific names. You must set it to a different value with each build. For example, you can set it to the output from the date command:

       BUILD_KEY ="androidn_$(date +%Y%m%d-%H%M%S)"
  • ANDROID_SOURCE points to the top-level directory of your Android source tree. This directory contains the makefile that starts the build.

  • HISTORY points to your history file. Change this setting only if your build stores history files elsewhere.

  • JAVA_DIR points to your JDK. For Android N, you must use JDK1.8.0_72.

  • LOGS_DIR points to your log files and annotation files. This directory must already exist.

The script also includes settings for addendum makefiles and the setting for a file introduced in Accelerator 9.0 that contains the pragmas for Ninja targets.

noautodep.mk

noautodep.mk contains "hints" that the Accelerator eDepend ("autodep") feature uses to tailor dependency management for better Android build performance. Following is the noautodep.mk template file:

   #pragma noautodep */.git/*
   $(local-intermediates-dir)/libbcc-stamp.c :

   #pragma noautodep */out/target/product/generic/system/bin/cat
   $(linked_module) :

   #pragma nolookup noproguard.classes-with-local.dex
   #pragma nolookup noproguard.classes.dex
   #pragma nolookup javalib.jar
   #pragma nolookup classes-full-debug.jar
   out/target/common/docs/api-stubs-timestamp :

sensitivity.mk

sensitivity.mk contains a pragma to instruct parse avoidance to detect the dependence of a parse result upon path wildcard match results. This pragma makes the parse cache sensitive to the existence (or absence) of specific files in directories that it reads as well as in new subdirectories.

Following is the sensitivity.mk template file:

   #pragma jobcache parse \
   -readdir Android.mk \
   -readdir *.java \
   -readdir *.c \
   -readdir I*.aidl \
   -readdir *.logtags \
   -readdir *.proto \
   -readdir *.rs \
   -readdir *.fs \
   -readdir *.html

cachekati.mk

This file applies only to Android M and N because kati is run as a make job and can be directly cached. In Android O, kati is run as part of another process and therefore cannot be cached.

Because output target names from Android N’s kati tool do not use a well-known pattern, you cannot enable JobCache for kati via the eMake --emake-jobcache command-line option. Instead, you must apply the #pragma jobcache kati pragma inside a makefile such as the cachekati.mk template file.

cachekati.mk contains settings for caching kati target files (.ninja files if you are using the Ninja build system). Following is the cachekati.mk template file:

   #pragma jobcache kati
   $(KATI_BUILD_NINJA) :

emake.pragmas

emake.pragmas is a pragma addendum file. Using pragmas for builds with eMake Ninja emulation enabled (--emake-emulation-ninja) requires an addendum file that contains the pragmas for Ninja targets. (Pragma addendum files are optional with any other emulation mode.)

This file contains #pragma settings similar to those in other addendum makefiles, but it works for Ninja emulation, which has no addendum mechanism. It marks certain targets as "runlocal" for performance. It also disables certain auto-dependencies to match Android’s natural behavior so that a build number change does not automatically cause a rebuild of large parts of the system.

From CloudBees Build Acceleration 10.0.1 onwards this also includes pragmas relating to jobcaching for javadoc targets which was previously in cachejavadoc.mk. This has not been functioning properly for Android N and O hence the change to using emake.pragmas where javadoc caching now works again.

Following are the first few lines from the emake.pragmas template file for Android O:

#pragma runlocal
out/target/product/generic/obj/PACKAGING/systemimage_intermediates/system.img:
#pragma runlocal
out/target/product/generic/system.img:
#pragma runlocal -repopulate ./ -repopulate ./out/
ea_build_ninjafile:
#pragma noautodep ./out/build_number.txt
out/soong/.glob/system/tools/hidl/test/vendor/1_0/Android_bp.glob:
#pragma noautodep ./out/build_number.txt
out/soong/.glob/system/tools/hidl/test/Android_bp.glob:
#pragma noautodep ./out/build_number.txt
out/soong/.glob/frameworks/compile/libbcc/tools/bcc_compat/Android_bp.glob:
#pragma noautodep ./out/build_number.txt
out/target/product/generic/system/framework/oat/arm/bmgr.vdex:
#pragma noautodep ./out/build_number.txt
out/target/product/generic/system/app/PhotoTable/oat/arm/PhotoTable.vdex:
#pragma noautodep ./out/build_number.txt
out/target/product/generic/system/app/CertInstaller/oat/arm/CertInstaller.vdex:
#pragma noautodep ./out/build_number.txt
out/soong/.glob/system/nfc/src/Android_bp.glob:
#pragma noautodep ./out/build_number.txt
out/soong/.glob/test/vts/drivers/Android_bp.glob:
#pragma noautodep ./out/build_number.txt
out/soong/.glob/system/libhidl/transport/manager/1_0/Android_bp.glob:
#pragma noautodep ./out/build_number.txt
out/target/product/generic/system/priv-app/Provision/oat/arm/Provision.vdex:
#pragma noautodep ./out/build_number.txt
out/host/linux-x86/obj/EXECUTABLES/aapt_intermediates/Main.o:
#pragma noautodep ./out/build_number.txt
out/soong/.glob/system/bt/device/Android_bp.glob:

...

main_mk_overrides.mk

# Split up the creation and running of the Ninja file that builds
# Android. We obviously wish to run it via eMake, to generate it
# "runlocal" for reasons of performance and the tools that are
# used to create it must be bootstrapped on an agent so they can
# be patched.
#
## start building with the ninja file
run_soong_ui: ea_build_ninjafile
    $(MAKE) -f out/emake_androido.mk --emake-emulation=ninja
# generate the ninja file (should be runlocal with -repopulate ./ and -repopulate out/
ea_build_ninjafile: ea_prep_soong_ui
  EADROID_ANDROIDO=$$(cat /proc/$$PPID/cmdline | tr '\000' '\n' | head -n 1 | sed 's#/64/bin/.*##')/unsupported/conf/android/8.0.0; $$EADROID_ANDROIDO/soong_ui.bash --generate-only --make-mode $(MAKECMDGOALS)

# Do the bootstrap (build the soong tools) but don't run soong
ea_prep_soong_ui:
 EADROID_ANDROIDO=$$(cat /proc/$$PPID/cmdline | tr '\000' '\n' | head -n 1 | sed 's#/64/bin/.*##')/unsupported/conf/android/8.0.0; $$EADROID_ANDROIDO/soong_ui.bash.partial

Optimizing Your Build

To optimize an Android build, complete the following steps.

  1. Upgrade CloudBees Build Acceleration to at least version 9.0. For instructions, see the CloudBees Build Acceleration Installation and Configuration Guide at https://docs.cloudbees.com/docs/cloudbees-build-acceleration/latest/install-guide/.

  2. If you use CloudBees Build Acceleration 9.0, extract the attached unsupported_conf.tgz tar file to the /i686_Linux/unsupported directory. For example, extract it to /opt/ecloud/i686_Linux/unsupported. Later versions include the scripts in this file.

  3. Clean your build environment. To do so, remove the "out" directory and any residual eMake history files (the default history file name is emake.data).

  4. Set up your environment as normal. For example:

   source build/envsetup.sh
   lunch aosp_arm-eng
  1. Run a "learning" build to generate a history file to record the build dependencies. A learning build also populates the cache.

  2. (Optional) Copy the eadroid.sh build script to your desired location:

    • For Android M, copy the script from
      /opt/ecloud/i686_Linux/unsupported/conf/android/6.0.0/eadroid.sh

  • For Android N, copy the script from
    /opt/ecloud/i686_Linux/unsupported/conf/android/7.0.0/eadroid.sh

  • For Android O, (CloudBees Build Acceleration 9.1.1 and above) copy the script from
    /opt/ecloud/i686_Linux/unsupported/conf/android/8.0.0/eadroid.sh

    1. Customize the following files:

  • eadroid.sh build script to indicate the location of your Android source code, history file, and so on

  • cachejavadoc.mk file as needed to cache and reuse files that Javadoc generates

  • noautodep.mk file as needed to tailor dependency management for better performance

  • sensitivity.mk file as needed to instruct parse avoidance to detect the dependence of a parse result upon path wildcard match results

  • (kati on Android N only) cachekati.mk file as needed to cache kati target files such as Ninja files

  • main_mk_overrides.mk file as needed to cause the Android O build to use Accelerator

  • (Required for Ninja targets) emake.pragmas file as needed to add the noautodep pragmas (to speed up no-touch builds) and runlocal pragmas (to speed up I/O-bound tasks)

    1. Run a make "clean" and then rerun the build with the script from step 5. CloudBees Build Acceleration uses the learning build from that step to optimize performance.

If you want to determine why a cached result was not used, use the Job Cache Misses report in CloudBees Build Acceleration Insight version 5.0 and newer versions. For details, see the "Reports" chapter in the CloudBees Build Acceleration Insight User Guide at https://docs.cloudbees.com/docs/cloudbees-build-acceleration/latest/user-guide/.

File Patching for Android

Some aspects of the Android build process conflict with Accelerator’s ability to virtualize builds; one of these is the Java Android Compiler Kit (Jack), which compiles Java source into Dalvik executable (.dex) bytecode. Jack is a Java program, and the Android project runs it as a server during the build so that its startup time occurs only once.

When a Jack server is started by a job on an agent, it sees the file system according to the needs of the current job that is running on that agent. If a job on Agent X requests actions on a server run by Agent Y, a problem might occur, because the Jack server’s view of the client file system might not match that of the job.

One way to avoid this is to patch the scripts that start and run Jack so that there is in effect one running under each agent. In this scenario, all jobs that want to use Jack can use the local one that is running on the same agent. To enable this, you configure Jack to create its settings file in (and read the file from) the location pointed to by $TMPDIR rather than in $HOME, because $TMPDIR is set uniquely for each agent. Then, it chooses its own unique port and writes it to the settings file.

Accelerator supports a file patching feature to do this automatically. When a file is requested by an agent, the eMake client checks its name, size, and MD5 checksum; if they match, it supplies a "patch" rather than the original content. These patches are in a special format in opt/ecloud/i686_Linux/patches/.

Accelerator contains patches for the jack script and a supporting script named jack-admin, which are suitable for AOSP (the open source version of Android) for Android versions 6.0, 7.0, and 7.1. (If you modified these scripts, then eMake cannot recognize the file to patch with complete certainty and therefore does nothing.)

How to Determine Which Files are Already Patched

The checkpatches.sh script in the unsupported/addpatch directory of your Accelerator installation can be used to list which patches are applied for a particular source file. It also knows which files are to be patched for some common builds such as Android M, N, and O. It is a useful way to check if the source tree that you are compiling is fully patched and to find the corresponding patches if you want to update them.

Example Usage with a List of Source Files

$ checkpatches.sh srcdir/filesomething.MK otherdir/sub/otherthing.c PATCHED:
'srcdir/filesomething.mk' ->
'/opt/ecloud/i686_Linux/patches/filesomething.mk/1573/5709bca85fdc06e70861e1ba93e7504a/patch' PATCHED: 'otherdir/sub/otherthing.c' ->
'/opt/ecloud/i686_Linux/patches/otherthing.c/2583/ce0757b011041df216b653db139cf7da/patch'

Each original file can have one of the following statuses:

  • PATCHED—​A patch exists (the patch path is displayed)

  • NOPATCH—​There is no corresponding patch for this original file

  • MISSING—​The original file cannot be found (make sure that you are running checkpatches.sh from the top directory of the build)

  • UNPATCHED—​A patch exists for this original file but it is not different from the original

Listing the Patches for a Particular OS

You can also use the -l <buildname> command line option to select a built-in list of patches for a particular type of build. For example:

/opt/ecloud/i686_Linux/unsupported/addpatch/checkpatches.sh -l android/8.0.0

The -l <buildname> command line option supports the following builds:

  • android/8.0.0 (Android O)

  • android/7.0.0 (Android N)

  • android/6.0.0 (Android M)

Following is example output for Android O and Accelerator 10.0:

    build@ecdroid3b:/c/src/android/AndroidO$ checkpatches.sh -l android/8.0.0
    PATCHED: 'build/soong/cmd/soong_ui/main.go' -> '/opt/ecloud/i686_Linux/patches/main.go/2458/ce0757a011041df216b653db139cf7da/patch'
    PATCHED: 'build/soong/ui/build/build.go' -> '/opt/ecloud/i686_Linux/patches/build.go/3133/b21317cb2eebd25187791f9b22df7cb5/patch'
    PATCHED: 'build/soong/ui/build/config.go' -> '/opt/ecloud/i686_Linux/patches/config.go/7611/9999260cd690731eceed340f851579eb/patch'
    PATCHED: 'prebuilts/sdk/tools/jack-admin' -> '/opt/ecloud/i686_Linux/patches/jack-admin/19553/d55e68ef0db1dd2ac8af53200ed20a73/patch'
    PATCHED: 'prebuilts/sdk/tools/jack' -> '/opt/ecloud/i686_Linux/patches/jack/6253/500d02d5e19366c52f87d703829d859e/patch'

How to Create Patches

You can use the checkpatches.sh script as described above to find out which files are patched already and use the results to plan what work you might need to do.

  1. Find your original copies of the Jack and Jack startup scripts. These are in:

  prebuilts/sdk/tools/jack
  prebuilts/sdk/tools/jack-admin
  1. Add these files to Accelerator’s patch database with the addpatch script, which is in the unsupported/addpatch directory of the Accelerator installation.

This script displays a path to the database entry. For example:

  /opt/ecloud/i686_Linux/patches/jack/4860/17f21ce31a981ae587e9e189da2edc9a/patch
  1. Modify this file (the patch file) to ensure that there is one Jack server per agent. The following .diff files are provided as examples for AOSP:

  unsupported/addpatch/android/android-6.0.1_r1/jack.diff
  unsupported/addpatch/android/android-6.0.1_r1/jack-admin.diff
  unsupported/addpatch/android/android-7.0.0_r1/jack.diff
  unsupported/addpatch/android/android-7.0.0_r1/jack-admin.diff
  1. Apply the.diff files by using the patch tool from Linux:

   cd /opt/ecloud/i686_Linux &&
   patch patches/jack/4860/17f21ce31a981ae587e9e189da2edc9a/patch < \
   unsupported/addpatch/android/android-6.0.1_r1/jack.diff
The first parameter to patch is the output from the addpatch script.

If your versions of the jack script and jack-admin differ greatly from the AOSP versions, then these patches might fail, so you must apply them manually.

Checking if a Patch Works

Without doing a full Android build, you can check that the patches are working by writing a small makefile similar to the following and running it from the top-level Android directory:

all:
  cat prebuilts/sdk/tools/jack
  cat prebuilts/sdk/tools/jack-admin

Then, you visually check that the output shows your modified script rather than the original.

Before Building with a Patch

The Android build system makes copies of the jack and jack-admin scripts under the out directory. File patching does not affect up-to-dateness checks, so these copies are not updated just because a patch was created for their "source." Thus, you should do a "from-clean" build after creating patches.

Supported Versions

  • Accelerator 9.0 through 10.0 for Android M and N support

  • Accelerator 9.1.1 through 10.0 for Android O support

See Also

Attachments