CloudBees Build Acceleration v12.0.0

In this release the Cluster Manager is updated to use Apache 2.4.46, PHP 7.4.12, and OpenSSL 1.1.1i.

The Build Details page in the Cluster Manager has been redesigned and expanded to include performance analysis and visualizations derived from CloudBees Build Acceleration Insight. This enhancement enables users and administrators to easily access the most critical performance data for builds from any system that can access the Cluster Manager web interface, even if the annotation file for the build is not accessible. The Build Details page now includes several new sub-tabs:

In addition to the updates described above, the Cluster Manager includes a number of other improvements:

This release extends Android support to include builds based on AOSP 11. In addition, this release features several performance improvements for both full and incremental Android builds. For best results, users should regenerate their history files by running a full build with the --emake-history=create option; subsequent builds can be run without that option or with the value set to merge. The changes made include:

This release also includes enhancements to Bitbake and Yocto support to expand compatibility and improve preformance. The changes include:

This release adds support for configuring agent security settings during installation and after installation with ecconfig. Security settings configured using these methods will be preserved during future upgrades. Prior to this change, configuring agent security settings required modification to the agent startup scripts, and those settings were not retained during upgrades (EC-13407).

This release introduces a new command-line option for emake, --emake-create-output-dirs. When enabled, this will cause emake to automatically and efficiently create output directories for targets when running in GMake emulation modes. In addition, the details of the implementation are such that this will avoid provoking conflicts even in builds that have no history file or have an incomplete history file. This feature is a superior alternative to mechanisms such as mkdir -p $(dirname $@), directory creation sentry files, and others that have commonly been employed with make-based build systems to solve the problem of creating output directories for build targets (EC-13680).

This release adds "build signatures", a hash computed from the serial ordered list of output targets in the build. This value can be used to identify builds that are substantively "the same" to facilitate comparisons and identification of trends across sets of like builds. The hash uses emake-root-relative target names, so the signature will be the same even if the absolute location of the build has changed (for example, as in continuous integration builds, or between developers working on the same project from their own copies of the source tree). The build signature for a build can be found in the annotation properties.

This release includes an updated release of CloudBees Build Acceleration Insight with several enhancements to support the new Build Details page in the Cluster Manager:

This release includes an enhancement to the way symbolic links are handled, such that timestamps on symlinks created during the build are in the correct relative order when compared to timestamps on regular files created during the build. This avoids spurious rebuilds of up-to-date outputs during incremental builds. (EC-13444)

With this fix, instead of failing with an assertion, emake will treat a job as in conflict if previous job activity indicated that the job should be in conflict, even if the file usage at the end of the job would not otherwise indicate a conflict.

With this fix, electrify treats the values specified in --electrify-remote as literal characters as expected, rather than as regex characters. This means that the user can specify --electrify-remote=g`, for example, and it will match the literal string `g rather than "g repeated one or more times". Use --electrify-allow-regex to specify process names using regex syntax.

With this fix, the installer correctly handles upgrades of agent hosts that are configured to automatically select the number of agents at startup, as is typical for cloudburst agent machine images.

With this fix, emake correctly updates autodep information based on usage from jobs pulled from the jobcache.

Prior to this change there was no way to completely expunge the autodep information for a single job, even if the build was changed such that autodep no longer applied to the job. With this fix, emake will clear the autodep entry for the job if it is no longer applicable.

A race condition sometimes caused the Cluster Manager to fail to remove the database entries for agents corresponding to cloudburst instances that had been terminated. With this fix, the race condition is eliminated so the database is updated reliably.

If the Cluster Manager was restarted while cloudburst agents were active, the Cluster Manager would lose track of the agents after the restart. The Cluster Manager would subsequently fail to terminate those agent instances, and would fail to reuse the agents for new builds. With this fix, the Cluster Manager correctly resumes management of cloudburst agents after restarting, ensuring that the agents are reused when appropriate and that the agents are terminated as expected.

If the annotation file for a task did not include at least waiting annotation detail and the annotation included recursive make invocations, bb2anno would erroneously truncate the task annotation data at the end of the first recursive make when incorporating it into the combined output annotation. With this fix, bb2anno instead detects task annotation files that do not have waiting annotation detail, prints a warning, and excludes those annotation files from the combined annotation.

A defect in the Cluster Manager where deleting the logs for builds that were deleted via the Cluster Manager web interface or with cmtool resulted in the deletion of build logs for unrelated builds. With this fix, only the logs for the deleted builds are deleted.

When using Kubernetes cloudbursting, the Cluster Manager failed to specify a CPU count when making agent container requests. This could result in the Kubernetes cluster trying to run more containers on a host than that host could reasonably support. With this fix, the Cluster Manager specifies a CPU count when starting new agent containers, enabling Kubernetes to correctly limit total deployment and balancing loads across hosts in the Kubernetes cluster.

Because the agent to Cluster Manager communication uses TLS by default, the agent image used for AWS EC2, GCP or Azure cloud bursting must also be upgraded to the 11.3 release or later in order for cloud bursting to work.

Starting in August 2019, the release strategy for CloudBees Build Acceleration is updated to add a “preview” release in addition to the standard long-term support (LTS), maintenance (patch), and hotfix releases.

As of version 11.1, perpetual license files include a Version field and are therefore no longer transferable to succeeding product versions. At run time, to be considered valid by the Cluster Manager, the license file must have a Version that equals or exceeds the version of the Cluster Manager itself. Version checks consider only the major.minor version (for example, a license for 11.1 is valid for 11.1.0, 11.1.1, 11.1.2, and so on).

The recommended total amount of RAM for an agent host is 2 GB per agent plus the amount of RAM normally needed to execute your build. For example, if you are running four agents, and your build normally needs 16 GB, you will need ((2 * 4) + 16) = 24 GB.

The upgrade process does not preserve the existing files. Back up the /opt/ecloud/<arch>/cloud directory for Linux or the C:\ECloud\<arch> folder for Windows to a safe location.

Affected kernel versions:

You should consider upgrading to 2.6.32-131 (RHEL 6.1) or downgrading to 2.6.18-194.32 (RHEL 5.5).