This KB article applies to CloudBees Build Acceleration versions 9.0 through 10.0. For information about best practices for 10.1 and newer versions, see KBEA-00165 - Best Practices for Android Builds Using CloudBees Build Acceleration 10.1 and Newer Versions.
This article describes the best practices for running Android builds. It includes descriptions of the CloudBees Build Acceleration template files that you can customize to optimize your builds.
This article contains the following topics:
Recommended Agent Deployment and Sizing
For the fastest Android builds, you should have at least 16 Accelerator agents.
RAM Requirements for Agent Hosts
The recommended total amount of RAM for an agent host is at least 2 GB per CloudBees Build Acceleration agent plus the amount of RAM normally needed by your build. For example, if you run four agents and your build normally needs 16 GB, you need ((2 * 4) + 16) = 24 GB.
Template Files for Build Optimization
Accelerator provides a set of files that work together to provide optimal settings for Android 6.x (Android M), Android 7.x (Android N), and Android 8.x (Android O). You customize these files for your builds.
For Accelerator 9.0, these files are in the attached
unsupported_conf.tgz file. For Accelerator 9.0.1 and higher versions, these files are also in the
eadroid.sh is a bash script that runs your build. Three versions are provided: one each for Android M, N, and O.
The top few lines of this file specify details such as the locations of your Android source code and your history file. Following are the top few lines in the template file for Android N:
... # Edit these: BUILD_KEY=androidn_1 ANDROID_SOURCE="/c/src/android/AndroidN" HISTORY="$ANDROID_SOURCE/emake.data" JAVA_DIR="/c/src/android/jdk1.8.0_72" # Android N LOGS_DIR="$ANDROID_SOURCE/logs" # must exist already ...
BUILD_KEYdistinguishes this build from other builds so that log files and annotation files have build-specific names. You must set it to a different value with each build. For example, you can set it to the output from the
BUILD_KEY ="androidn_$(date +%Y%m%d-%H%M%S)"
ANDROID_SOURCEpoints to the top-level directory of your Android source tree. This directory contains the makefile that starts the build.
HISTORYpoints to your history file. Change this setting only if your build stores history files elsewhere.
JAVA_DIRpoints to your JDK. For Android N, you must use
LOGS_DIRpoints to your log files and annotation files. This directory must already exist.
The script also includes settings for addendum makefiles and the setting for a file introduced in Accelerator 9.0 that contains the pragmas for Ninja targets.
noautodep.mk contains "hints" that the Accelerator eDepend ("autodep") feature uses to tailor dependency management for better Android build performance. Following is the
noautodep.mk template file:
#pragma noautodep */.git/* $(local-intermediates-dir)/libbcc-stamp.c : #pragma noautodep */out/target/product/generic/system/bin/cat $(linked_module) : #pragma nolookup noproguard.classes-with-local.dex #pragma nolookup noproguard.classes.dex #pragma nolookup javalib.jar #pragma nolookup classes-full-debug.jar out/target/common/docs/api-stubs-timestamp :
sensitivity.mk contains a pragma to instruct parse avoidance to detect the dependence of a parse result upon path wildcard match results. This pragma makes the parse cache sensitive to the existence (or absence) of specific files in directories that it reads as well as in new subdirectories.
Following is the
sensitivity.mk template file:
#pragma jobcache parse \ -readdir Android.mk \ -readdir *.java \ -readdir *.c \ -readdir I*.aidl \ -readdir *.logtags \ -readdir *.proto \ -readdir *.rs \ -readdir *.fs \ -readdir *.html
This file applies only to Android M and N because kati is run as a make job and can be directly cached. In Android O, kati is run as part of another process and therefore cannot be cached.
Because output target names from Android N’s kati tool do not use a well-known pattern, you cannot enable JobCache for kati via the eMake
--emake-jobcache command-line option. Instead, you must apply the
#pragma jobcache kati pragma inside a makefile such as the
cachekati.mk template file.
cachekati.mk contains settings for caching kati target files (
.ninja files if you are using the Ninja build system). Following is the
cachekati.mk template file:
#pragma jobcache kati $(KATI_BUILD_NINJA) :
emake.pragmas is a pragma addendum file. Using pragmas for builds with eMake Ninja emulation enabled (--
emake-emulation-ninja) requires an addendum file that contains the pragmas for Ninja targets. (Pragma addendum files are optional with any other emulation mode.)
This file contains
#pragma settings similar to those in other addendum makefiles, but it works for Ninja emulation, which has no addendum mechanism. It marks certain targets as "runlocal" for performance. It also disables certain auto-dependencies to match Android’s natural behavior so that a build number change does not automatically cause a rebuild of large parts of the system.
From CloudBees Build Acceleration 10.0.1 onwards this also includes pragmas relating to jobcaching for javadoc targets which was previously in cachejavadoc.mk. This has not been functioning properly for Android N and O hence the change to using emake.pragmas where javadoc caching now works again.
Following are the first few lines from the
emake.pragmas template file for Android O:
#pragma runlocal out/target/product/generic/obj/PACKAGING/systemimage_intermediates/system.img: #pragma runlocal out/target/product/generic/system.img: #pragma runlocal -repopulate ./ -repopulate ./out/ ea_build_ninjafile: #pragma noautodep ./out/build_number.txt out/soong/.glob/system/tools/hidl/test/vendor/1_0/Android_bp.glob: #pragma noautodep ./out/build_number.txt out/soong/.glob/system/tools/hidl/test/Android_bp.glob: #pragma noautodep ./out/build_number.txt out/soong/.glob/frameworks/compile/libbcc/tools/bcc_compat/Android_bp.glob: #pragma noautodep ./out/build_number.txt out/target/product/generic/system/framework/oat/arm/bmgr.vdex: #pragma noautodep ./out/build_number.txt out/target/product/generic/system/app/PhotoTable/oat/arm/PhotoTable.vdex: #pragma noautodep ./out/build_number.txt out/target/product/generic/system/app/CertInstaller/oat/arm/CertInstaller.vdex: #pragma noautodep ./out/build_number.txt out/soong/.glob/system/nfc/src/Android_bp.glob: #pragma noautodep ./out/build_number.txt out/soong/.glob/test/vts/drivers/Android_bp.glob: #pragma noautodep ./out/build_number.txt out/soong/.glob/system/libhidl/transport/manager/1_0/Android_bp.glob: #pragma noautodep ./out/build_number.txt out/target/product/generic/system/priv-app/Provision/oat/arm/Provision.vdex: #pragma noautodep ./out/build_number.txt out/host/linux-x86/obj/EXECUTABLES/aapt_intermediates/Main.o: #pragma noautodep ./out/build_number.txt out/soong/.glob/system/bt/device/Android_bp.glob: ...
# Split up the creation and running of the Ninja file that builds # Android. We obviously wish to run it via eMake, to generate it # "runlocal" for reasons of performance and the tools that are # used to create it must be bootstrapped on an agent so they can # be patched. # ## start building with the ninja file run_soong_ui: ea_build_ninjafile $(MAKE) -f out/emake_androido.mk --emake-emulation=ninja # generate the ninja file (should be runlocal with -repopulate ./ and -repopulate out/ ea_build_ninjafile: ea_prep_soong_ui EADROID_ANDROIDO=$$(cat /proc/$$PPID/cmdline | tr '\000' '\n' | head -n 1 | sed 's#/64/bin/.*##')/unsupported/conf/android/8.0.0; $$EADROID_ANDROIDO/soong_ui.bash --generate-only --make-mode $(MAKECMDGOALS) # Do the bootstrap (build the soong tools) but don't run soong ea_prep_soong_ui: EADROID_ANDROIDO=$$(cat /proc/$$PPID/cmdline | tr '\000' '\n' | head -n 1 | sed 's#/64/bin/.*##')/unsupported/conf/android/8.0.0; $$EADROID_ANDROIDO/soong_ui.bash.partial
Optimizing Your Build
To optimize an Android build, complete the following steps.
Upgrade CloudBees Build Acceleration to at least version 9.0. For instructions, see the CloudBees Build Acceleration Installation and Configuration Guide at https://docs.cloudbees.com/docs/cloudbees-build-acceleration/latest/install-guide/.
If you use CloudBees Build Acceleration 9.0, extract the attached
unsupported_conf.tgztar file to the
/i686_Linux/unsupporteddirectory. For example, extract it to
/opt/ecloud/i686_Linux/unsupported. Later versions include the scripts in this file.
Clean your build environment. To do so, remove the "out" directory and any residual eMake history files (the default history file name is
Set up your environment as normal. For example:
source build/envsetup.sh lunch aosp_arm-eng
Run a "learning" build to generate a history file to record the build dependencies. A learning build also populates the cache.
(Optional) Copy the
eadroid.shbuild script to your desired location:
For Android M, copy the script from
For Android N, copy the script from
For Android O, (CloudBees Build Acceleration 9.1.1 and above) copy the script from
Customize the following files:
eadroid.shbuild script to indicate the location of your Android source code, history file, and so on
cachejavadoc.mkfile as needed to cache and reuse files that Javadoc generates
noautodep.mkfile as needed to tailor dependency management for better performance
sensitivity.mkfile as needed to instruct parse avoidance to detect the dependence of a parse result upon path wildcard match results
(kati on Android N only)
cachekati.mkfile as needed to cache kati target files such as Ninja files
main_mk_overrides.mkfile as needed to cause the Android O build to use Accelerator
(Required for Ninja targets)
emake.pragmasfile as needed to add the noautodep pragmas (to speed up no-touch builds) and runlocal pragmas (to speed up I/O-bound tasks)
Run a make "clean" and then rerun the build with the script from step 5. CloudBees Build Acceleration uses the learning build from that step to optimize performance.
|If you want to determine why a cached result was not used, use the Job Cache Misses report in CloudBees Build Acceleration Insight version 5.0 and newer versions. For details, see the "Reports" chapter in the CloudBees Build Acceleration Insight User Guide at https://docs.cloudbees.com/docs/cloudbees-build-acceleration/latest/user-guide/.|
File Patching for Android
Some aspects of the Android build process conflict with Accelerator’s ability to virtualize builds; one of these is the Java Android Compiler Kit (Jack), which compiles Java source into Dalvik executable (.dex) bytecode. Jack is a Java program, and the Android project runs it as a server during the build so that its startup time occurs only once.
When a Jack server is started by a job on an agent, it sees the file system according to the needs of the current job that is running on that agent. If a job on Agent X requests actions on a server run by Agent Y, a problem might occur, because the Jack server’s view of the client file system might not match that of the job.
One way to avoid this is to patch the scripts that start and run Jack so that there is in effect one running under each agent. In this scenario, all jobs that want to use Jack can use the local one that is running on the same agent. To enable this, you configure Jack to create its settings file in (and read the file from) the location pointed to by $TMPDIR rather than in $HOME, because $TMPDIR is set uniquely for each agent. Then, it chooses its own unique port and writes it to the settings file.
Accelerator supports a file patching feature to do this automatically. When a file is requested by an agent, the eMake client checks its name, size, and MD5 checksum; if they match, it supplies a "patch" rather than the original content. These patches are in a special format in opt/ecloud/i686_Linux/patches/.
Accelerator contains patches for the jack script and a supporting script named jack-admin, which are suitable for AOSP (the open source version of Android) for Android versions 6.0, 7.0, and 7.1. (If you modified these scripts, then eMake cannot recognize the file to patch with complete certainty and therefore does nothing.)
How to Determine Which Files are Already Patched
The checkpatches.sh script in the unsupported/addpatch directory of your Accelerator installation can be used to list which patches are applied for a particular source file. It also knows which files are to be patched for some common builds such as Android M, N, and O. It is a useful way to check if the source tree that you are compiling is fully patched and to find the corresponding patches if you want to update them.
Example Usage with a List of Source Files
$ checkpatches.sh srcdir/filesomething.MK otherdir/sub/otherthing.c PATCHED: 'srcdir/filesomething.mk' -> '/opt/ecloud/i686_Linux/patches/filesomething.mk/1573/5709bca85fdc06e70861e1ba93e7504a/patch' PATCHED: 'otherdir/sub/otherthing.c' -> '/opt/ecloud/i686_Linux/patches/otherthing.c/2583/ce0757b011041df216b653db139cf7da/patch'
Each original file can have one of the following statuses:
PATCHED—A patch exists (the patch path is displayed)
NOPATCH—There is no corresponding patch for this original file
MISSING—The original file cannot be found (make sure that you are running checkpatches.sh from the top directory of the build)
UNPATCHED—A patch exists for this original file but it is not different from the original
Listing the Patches for a Particular OS
You can also use the -l <buildname> command line option to select a built-in list of patches for a particular type of build. For example:
/opt/ecloud/i686_Linux/unsupported/addpatch/checkpatches.sh -l android/8.0.0
The -l <buildname> command line option supports the following builds:
android/8.0.0 (Android O)
android/7.0.0 (Android N)
android/6.0.0 (Android M)
Following is example output for Android O and Accelerator 10.0:
build@ecdroid3b:/c/src/android/AndroidO$ checkpatches.sh -l android/8.0.0 PATCHED: 'build/soong/cmd/soong_ui/main.go' -> '/opt/ecloud/i686_Linux/patches/main.go/2458/ce0757a011041df216b653db139cf7da/patch' PATCHED: 'build/soong/ui/build/build.go' -> '/opt/ecloud/i686_Linux/patches/build.go/3133/b21317cb2eebd25187791f9b22df7cb5/patch' PATCHED: 'build/soong/ui/build/config.go' -> '/opt/ecloud/i686_Linux/patches/config.go/7611/9999260cd690731eceed340f851579eb/patch' PATCHED: 'prebuilts/sdk/tools/jack-admin' -> '/opt/ecloud/i686_Linux/patches/jack-admin/19553/d55e68ef0db1dd2ac8af53200ed20a73/patch' PATCHED: 'prebuilts/sdk/tools/jack' -> '/opt/ecloud/i686_Linux/patches/jack/6253/500d02d5e19366c52f87d703829d859e/patch'
How to Create Patches
You can use the checkpatches.sh script as described above to find out which files are patched already and use the results to plan what work you might need to do.
Find your original copies of the Jack and Jack startup scripts. These are in:
Add these files to Accelerator’s patch database with the addpatch script, which is in the unsupported/addpatch directory of the Accelerator installation.
This script displays a path to the database entry. For example:
Modify this file (the patch file) to ensure that there is one Jack server per agent. The following .diff files are provided as examples for AOSP:
unsupported/addpatch/android/android-6.0.1_r1/jack.diff unsupported/addpatch/android/android-6.0.1_r1/jack-admin.diff unsupported/addpatch/android/android-7.0.0_r1/jack.diff unsupported/addpatch/android/android-7.0.0_r1/jack-admin.diff
Apply the.diff files by using the patch tool from Linux:
cd /opt/ecloud/i686_Linux && patch patches/jack/4860/17f21ce31a981ae587e9e189da2edc9a/patch < \ unsupported/addpatch/android/android-6.0.1_r1/jack.diff
|The first parameter to patch is the output from the addpatch script.|
If your versions of the jack script and jack-admin differ greatly from the AOSP versions, then these patches might fail, so you must apply them manually.
Checking if a Patch Works
Without doing a full Android build, you can check that the patches are working by writing a small makefile similar to the following and running it from the top-level Android directory:
all: cat prebuilts/sdk/tools/jack cat prebuilts/sdk/tools/jack-admin
Then, you visually check that the output shows your modified script rather than the original.
Before Building with a Patch
The Android build system makes copies of the jack and jack-admin scripts under the out directory. File patching does not affect up-to-dateness checks, so these copies are not updated just because a patch was created for their "source." Thus, you should do a "from-clean" build after creating patches.
Accelerator 9.0 through 10.0 for Android M and N support
Accelerator 9.1.1 through 10.0 for Android O support
For information about Javadoc caching, eDepend, parse avoidance, job caching for kati, and pragma addendum files, see the CloudBees Build Acceleration Electric Make User Guide at https://docs.cloudbees.com/docs/cloudbees-build-acceleration/latest/emake-user-guide/.
For information about the parse avoidance feature, see KBEA-00129 - Using the parse avoidance feature.
For information about the dependency optimization feature, see KBEA-00131 - Using the dependency optimization feature.
For information about setting up your local work environment to build Android source files, see Establishing a Build Environment.