CloudBees Accelerator 11.1 Release Notes

Product Description

CloudBees Accelerator® is a software build accelerator that dramatically reduces build times by distributing the build over a large cluster of inexpensive servers. CloudBees Accelerator (“Accelerator”) uses a patented dependency-management system to identify and fix problems in real time that break traditional parallel builds. Accelerator plugs seamlessly into existing software development environments and includes web-based management and reporting tools.

Accelerator includes the following components:

  • Electric Make® (“eMake”)

  • Electric File System (EFS)

  • CloudBees Accelerator Agents (“Agents”)

  • Cluster Manager

  • Electrify

What’s New or Modified

New Features and Functionality

Version Enforcement in Cluster Manager Licensing

As of version 11.1, perpetual license files include a Version field and are therefore no longer transferable to succeeding product versions. At run time, to be considered valid by the Cluster Manager, the license file must have a Version that equals or exceeds the version of the Cluster Manager itself. Version checks consider only the major.minor version (for example, a license for 11.1 is valid for 11.1.0, 11.1.1, 11.1.2, and so on).

eMake authenticates with the Cluster Manager via the eMake/Cluster Manager protocol version to ensure that the eMake and Cluster Manager licenses match. If the license has no Version field, it will be considered invalid by the 11.1 Cluster Manager.

You can upgrade to new patch releases (but not new feature releases) without acquiring a new license. For example, if your current license specifies 11.1 , then you can use a Cluster Manager with version 11.1 , 11.1.1 , 11.1.2 , and so on. Also, for example, if your license specifies 12.0 , then you can use a Cluster Manager with version 12.0 , 12.0.1 , 12.0.2 , and so on, as well as 11.x .

The Administration > Licenses page in the Cluster Manager web UI now has a Version column, which provides the product version of each license in the list. This information is blank for a license with no explicit version (such as an evaluation license).

Make sure that you import a compliant license into the Cluster Manager before upgrading. For assistance, see https://support.cloudbees.com/ . For details about importing a license, see the “Importing Your License” section in the “Configuring CloudBees Accelerator” chapter of the CloudBees Accelerator Installation and Configuration Guide , available in the CloudBees Accelerator documentation . (EC-13170 and EC-13154)

Google Cloud Platform Support (Linux only)

Accelerator now supports Google Cloud Platform (GCP) on Linux platforms. This integration lets you configure the Cluster Manager to use GCP to spin up agents on demand. (EC-13242)

Azure Cloud Provider Support (Linux only)

Accelerator now supports Azure Cloud Provider (Azure) on Linux platforms. This integration lets you configure the Cluster Manager to use Azure to spin up agents on demand. (EC-13240)

Amazon EC2 Cloud Provider Support (Linux only)

For Amazon EC2, the ability to choose to use the IAM role of the server as credentials is introduced on Linux platforms. (EC-12710). For details about this enhancement, see the CloudBees Accelerator {accelerator-product-version-11-1} Help , available in the CloudBees Accelerator documentation .

TLS for eMake-to-Agent Communication (Linux only)

TLS-based authentication and encryption is now available for communication between eMake and agents on Linux platforms. For details about configuring TLS, see the “Configuring TLS” section in the “Configuring CloudBees Accelerator” chapter of the CloudBees Accelerator {accelerator-product-version-11-1} Installation and Configuration Guide , available in the CloudBees Accelerator documentation . For a list of the eMake command-line options for using TLS for builds, see the “List of eMake Command-Line Options” section in the “Electric Make Basics” chapter of the CloudBees Accelerator {accelerator-product-version-11-1} Electric Make User Guide . (EC-13267)

Automatic Scaling of the Number of Agents in an Agent AMI

You can now configure an agent AMI that you use for cloud bursting to automatically scale the number of agents according to the number of CPUs on the host. This facilitates using this single image as a template across many different instance types, regardless of the CPU count on each type, to get the ideal number of agents. For details, see the “Enabling Automatic Scaling of the Number of Agents in the Agent AMI” section in the “Configuring CloudBees Accelerator” chapter of the CloudBees Accelerator {accelerator-product-version-11-1} Installation and Configuration Guide , available in the CloudBees Accelerator documentation . (EC-13270)

eMake Job Hard Timeout

The --emake-job-timeout eMake option is introduced. This option is especially useful for adding resiliency to the eMake “test case” mode. It specifies the number of seconds to allow a job to run before timing out, after which it terminates the job and discards all job output. For details, see the “List of eMake Command-Line Options” section in the “eMake Basics” chapter of the CloudBees Accelerator Electric Make User Guide , available in the CloudBees Accelerator documentation . (EC-13043)

Android Build Improvements

This release provides better performance and greater compatibility for Android builds as follows.

  • The metalava keyword is added to the --emake-jobcache eMake option. This option lets you enable job caching for Metalava jobs. (EC-13313)

  • Accelerator is updated to work with Android 10 (”Q”) and is verified with “preview 1.” (EC-13250)

  • Kati job caching is now supported on Android P and Q builds. (EC-13311)

Visual Studio Integration Update

The Visual Studio integration (which is bundled with Accelerator) is upgraded to version 5.3.0. This version contains a number of updates.

Among these updates, the VS IDE extension uses CloudBees Accelerator Electrify to build rather than ecdevenv, so there are extensive changes to the settings dialog and the behavior of the build. Differences include the following:

  • Electrify has replaced eMake.

    Electrify does not use a history file and cannot detect conflicts like eMake. Your build must be able to be run in parallel with Visual Studio.

  • Schedule optimization is no longer available.

  • Electrify does not convert Visual Studio projects into NMAKE format, so there is no “conversion” time.

  • Electrify does not use the Visual Studio converter extension, so its options are removed from the Solution Settings dialog.

For a detailed list of new features and resolved issues in version 5.3.0, see the “What’s New” section in the CloudBees Accelerator 5.3.0 Visual Studio Integration Guide , available in the CloudBees Accelerator documentation .

ElectricInsight Update

ElectricInsight (which is bundled with Accelerator) is upgraded to version 5.5.1 and is rebranded to CloudBees Accelerator Insight (”Insight”). The 32- and 64-bit executables for Insight are installed.

Insight is licensed separately from Accelerator. An Insight license file is required to run reports. All other functionality (view-only mode) is available without a license.

For the list of new features and resolved issues, see the CloudBees Accelerator Insight 5.5.1 Release Notes , available in the CloudBees Accelerator documentation . For information about the 32- and 64-bit executables for Insight and about using Insight, see the CloudBees Accelerator Insight 5.5 User Guide , available in the CloudBees Accelerator documentation .

Other New Features and Functionality

  • The cloud burst tag used for Accelerator agents in Amazon EC2 instances is rebranded from EA-Name:ElectricAccelerator-Agent to CB-Name:Accelerator-Agent . (EC-13396)

  • The all keyword is added to the --emake-jobcache eMake option. This keyword is a shortcut for adding all available job caching command-line keywords ( cl , clang , clang-cl , gcc , jack , javac , and metalava ). (Because kati output target names do not use a well-known pattern, you cannot use this keyword to enable JobCache for kati. Instead, you must add #pragma jobcache kati to the makefile.) (EC-13312)

  • The Cluster Manager web UI has had a limited rebranding to CloudBees Accelerator branding. (EC-13383)

  • An Install Visual Studio Plugin checkbox is added to the Base Setup dialog box in the CloudBees Accelerator installer wizard. This lets you opt out of the installation of this bundled component. For details, see the “Installing CloudBees Accelerator on Windows” section in the “Installing CloudBees Accelerator” chapter of the CloudBees Accelerator {accelerator-product-version-11-1} Installation and Configuration Guide , available in the CloudBees Accelerator documentation . (EC-13077)

  • The certificate in the keystore for the Cluster Manager web server is updated. (However, CloudBees recommends that you use your own certificate. For details about using your own certificate, see the “Installing an Apache Server Certificate” section in the “Configuring CloudBees Accelerator” chapter of the CloudBees Accelerator {accelerator-product-version-11-1} Installation and Configuration Guide , available in the CloudBees Accelerator documentation .) (EC-12945)

Behavior Changes

None.

Resolved Issues

The following Cluster Manager web server security-related issues are fixed:

  • (Linux platforms only) OpenSSL is upgraded from version 1.0.2d to version 1.1.1b. (EC-13316)

  • (Linux platforms only) PHP is upgraded from version 5.6.37 to version 5.6.40. (EC-13316)

  • (Linux platforms only) Apache is upgraded from version 2.4.23 to version 2.4.39. (EC-13309)

  • Certain Cross-Site Request Forgery (CSRF) vulnerabilities are fixed. (EC-13181)

Other Issues

  • (Windows platforms only) A kernel memory leak was fixed in the Electric File System (EFS) that occurred when directory scans were restarted. (EC-13345)

  • The "Setting Up the Cluster Manager for AWS EC2" section in the CloudBees Accelerator 11.1 Installation and Configuration Guide is rewritten to explain how to set up a new credential and resource to use EC2. For details, see the guide, located in the CloudBees Accelerator documentation . (EC-13295)

  • When eMake is running in test case mode, if the build is interrupted, completed jobs that are later in “serial order” than the aborted job are now included in the annotation file. (EC-13229)

  • (Windows platforms only) In eMake test case mode, an issue in registry mirroring that caused test failures is fixed. (EC-13166)

  • eMake no longer caches parse results if the parse job does not read any makefiles. (EC-12977)

Platform Support

Support is added for the following platforms.

  • SUSE Linux Enterprise (SLES) 12 Service Pack 4 (kernel 4.12.4) (64-bit). (EC-13271)

  • SLES 15 (kernel 4.12.14) (64-bit). (EC-13254)

  • Ubuntu Linux 18.04 (kernel 4.18). (Kernel 4.15 is still supported under this Linux version.)

  • CentOS 7.4-xxxx (kernel 3.10.0-xxx) and 7.5-xxxx (kernel 3.10.0-xxx) (64-bit) (EC-xxxxx)

  • Debian 9 (kernel 4.x) (64-bit) (EC-xxxxx)

  • RHEL x.xx (kernel 2.6.32-xxx) (64-bit for the Cluster Manager; 32- and 64-bit for eMake and agents) (EC-xxxxx)

Installation and Upgrade Notes

Hardware Requirements

  • Accelerator 7.2 and newer versions require a Pentium 4 or newer processor when running in a 32-bit Solaris x 86 environment.

  • The recommended total amount of RAM for an agent host is 2 GB per agent plus the amount of RAM normally needed to execute your build. For example, if you are running four agents, and your build normally needs 16 GB, you will need ((2 * 4) + 16) = 24 GB.

Backing Up Before You Upgrade

  • The upgrade process does not preserve the existing files. Back up the /opt/ecloud/<arch>/cloud directory for Linux and Solaris or the C:\ECloud\<arch> folder for Windows to a safe location.

  • For additional security, back up the database by following the recommended procedure from your database vendor.

Installing JDBC Drivers for MySQL or Oracle Databases

CloudBees no longer distributes the JDBC drivers for MySQL or Oracle databases. To use one of these databases, you must download its driver directly from the Oracle website, then copy it to the appropriate directory on the Cluster Manager server, and then restart the Cluster Manager service. For details, see the “Installing JDBC Database Drivers” section in the “Installing CloudBees Accelerator” chapter of the CloudBees Accelerator {accelerator-product-version-11-1} Installation and Configuration Guide , available in the CloudBees Accelerator documentation .

Copying the execserver Executable to a New Location if You Relocate eMake

If you copy the emake executable to a new location, you must also copy the execserver executable to that location. By default, the path to the execserver executable is /opt/ecloud/i686_Linux/32/bin/execserver (for 32-bit eMake) or /opt/ecloud/i686_Linux/64/bin/execserver (for 64-bit eMake).

Regenerating History Files After an Upgrade

The identifier that is used to find certain types of jobs in the eMake history file changed in version 8.0. After an upgrade from version 7.2.2 or older versions to version 8.0 or newer versions, users should regenerate their history files by running their first build with the --emake-history=create option to avoid unnecessary serializations. This build might have more conflicts than normal (but subsequent builds should return to normal).

Concurrent Build Licensing

As of version 9.1, for new Accelerator subscription licenses, the number of builds that you can run concurrently is license-limited. The noLicenseWaitTime performance metric indicates the amount of time that a build spent waiting for a concurrent build license because the number of concurrent builds reached the license limit. Also, as of version 9.1, JobCache is not separately licensed and is now included with the concurrent build license.

Customers using pre-9.1 Accelerator licenses may continue to use those licenses, including the licenses for the JobCache add-on.

For details about licensing for concurrent builds, see the CloudBees Accelerator {accelerator-product-version-11-1} Installation and Configuration Guide , available in the CloudBees Accelerator documentation . (EC-12095)

Known Issues

Linux Kernel Issue That Affects Accelerator Performance

Affected Kernel Versions

  • RHEL kernel versions later than 2.6.18-194.32 and earlier than 2.6.32-131

  • Ubuntu Linux kernel versions 2.6.31, 2.6.32, 2.6.33, and 2.6.34

Symptoms

Affected systems might encounter reduced performance on both ext3 and ext4 file systems. Symptoms might include

  • hung_task_timeout_secs messages in system dmesg logs

  • Widely variable agent availability (entering and exiting agent “penalty” status frequently)

  • Contention over the ecagent.state file

  • Slower builds (with unexplained variances)

To help determine if this issue exists, run the dmesg | grep hung_task_timeout command. hung_task_timeout errors show that this issue is present. Contact your kernel provider for another version of the precompiled kernel.

Fixes for Systems Running RHEL 5.6, 5.7, 5.8, and 6.0

You should consider upgrading to 2.6.32-131 (RHEL 6.1) or downgrading to 2.6.18-194.32 (RHEL 5.5).

Other Known Issues

  • For GCP cloud burst instances, the static host and agents are not shown as matching on the Resources page in the Cluster Manager web UI. (EC-13317)

  • Android M is not compatible with Accelerator 11.1. You must use Android M with Accelerator 10.0.

  • If you kill a build manually, and the agents running on an Amazon EC2 instance fail to connect, the instance will continue to run. You must kill the instance manually.

  • If the Cluster Manager goes down while agents are running on an Amazon EC2 instance, they will still connect but will not be associated with a resource. You must kill the corresponding instance manually.

  • The cmtool importData command does not import license properties (such as maxAgents ). To work around this issue, re-import the license after using importData . (EC-12371)

  • You cannot control breakpoints from the Cluster Manager. (EC-12322)

  • If Apache fails to start properly after a new Cluster Manager installation, reboot the system.

Documentation

Product Documentation

Accelerator documentation is available in the CloudBees Accelerator documentation as follows:

  • CloudBees Accelerator Installation and Configuration Guide

  • CloudBees Accelerator Electric Make User Guide

  • CloudBees Accelerator cmtool Reference Guide

  • CloudBees Accelerator Error Messages Guide

  • CloudBees Accelerator Visual Studio Integration Guide

  • CloudBees Accelerator Release Notes (this document)

  • PDF, HTML, and mobile-optimized HTML versions of the online help that is also accessible from the Cluster Manager Help button.

Documentation on the website is updated periodically.

Cluster Manager Online Help and Tooltips

Built into the Cluster Manager are

  • A complete, robust, context-sensitive online help system (click the Help button in any page of the Cluster Manager web UI). See the documentation website as described above for the latest updates to this information.

  • Tooltips with information to help fill in form fields.