CloudBees Accelerator 2019.10.00 Release Notes

10 minute read

Product Description

CloudBees Accelerator® is a software build accelerator that dramatically reduces build times by distributing the build over a large cluster of inexpensive servers. CloudBees Accelerator (“Accelerator”) uses a patented dependency-management system to identify and fix problems in real time that break traditional parallel builds. Accelerator plugs seamlessly into existing software development environments and includes web-based management and reporting tools.

CloudBees Accelerator includes the following components:

  • Electric Make® (“eMake”)

  • Electric File System (EFS)

  • Accelerator agents (“Agents”)

  • Cluster Manager

  • Electrify

What’s New or Modified

New Features and Functionality

2019.10.00

Deployment on Google Cloud Platform Marketplace

You can now deploy CloudBees Accelerator on Google Cloud Platform (GCP) Marketplace. GCP Marketplace lets you start up software packages easily without having to manually configure the software, virtual machine instances, storage, or network settings.

For details about GCP Marketplace, see What is Google Cloud Platform Marketplace? . To get started, go to the GCP Marketplace page of the GCP Console and search for “CloudBees Accelerator” or go directly to the CloudBees Accelerator solution page at https://console.cloud.google.com/marketplace/details/cloudbees/cloudbees-accelerator .

Amazon EC2 Spot Instance Support for Agent Cloud Bursting

A Prefer Spot Instances checkbox is added to the New Resource page in the Resources tab of the Cluster Manager web UI. When this feature is enabled, Amazon EC2 Spot Instances will be launched for agent cloud bursting.

Spot Instances let you use spare Amazon EC2 computing capacity at a discount versus on-demand pricing. For more information about Spot Instances and how to enable them, see Resources - Create or Edit a Resource.

Agent cloud bursting is supported only on Linux platforms.

Installation and Configuration Improvements

This release contains several improvements to make it easier to install and configure Accelerator for simple one-system installations for first-time or trial-basis users.

  • In the interactive command line and GUI installation modes, when you choose the Express installation type and select either a Full installation (which includes agents) or an Agent installation, the installer now displays how many agents that it will configure. (EC-13554)

  • When you install all Accelerator components (eMake, agents, and the Cluster Manager) on the same machine in Express installer mode, the installer no longer prompts you for the hostname or IP address of your Cluster Manager. Instead, the installer automatically sets the Cluster Manager hostname to localhost . This means that you no longer must specify the --emake-cm=host[:port] eMake option when invoking a build on a system that is running all components (such as for testing or other nonproduction or trial-basis situations).

    The installer places the localhost hostname setting in the emake.conf configuration file for eMake to use. For upgrades, the installer moves your existing emake.conf configuration file to a backup file and creates a new emake.conf file to include this setting along with your other settings. (EC-13572 and EC-13552)

  • (Linux platforms only) Installation of prerequisite libraries is simplified: The installer now installs missing prerequisite libraries after prompting you to confirm. If installation of these libraries fails, the installer displays the commands that you enter to install them afterward.

    For a list of the prerequisite libraries for your Linux platform (as well as the commands to install them), see Supported Linux Platforms and Prerequisites. (EC-13550)

2019.09.00

Android 10 Support

On Linux platforms, support for Android 10 (“Q”) R2 builds is added. For Android first-time (“learning”) builds, eMake “kick starts” these builds by determining if you lack a history file and then providing a “kickstart” history file, which helps to reduce the build duration by mitigating the effect of a large number of conflicts. This release adds this capability as part of Android 10 support. (EC-13542)

Build Performance for kati on Android 9.x and Android 10

For kati job caching on Android 9. x (”P”), the corresponding noautodep pragmas that enable autodependency prevention are now created (in /opt/ecloud/i686_Linux/conf/types/android/version/emake.pragmas ) for common files for the ea_run_kati target. kati job caching was re-introduced in CloudBees Accelerator 11.1; the 2019.09.00 update also ensures that kati job caching works for Android 10 R2. (EC-13522)

CloudBees Accelerator Insight Update

CloudBees Accelerator Insight (”Insight”), which is bundled with Accelerator, is upgraded to version 2019.09.00. The 32- and 64-bit executables for Insight are installed. This upgrade adds the ability to run the Root Conflicts report from the command line. Insight is licensed separately from Accelerator, and an Insight license file is required to run reports.

For details about this feature, see the CloudBees Accelerator Insight 2019.09.00 Release Notes , available on the CloudBees Accelerator release notes page. For information about the 32- and 64-bit executables for Insight and about using Insight, see the CloudBees Accelerator Insight User Guide.

Installation and Configuration Improvements
  • The Accelerator installer now displays the command that you must run to add the location of the emake executable to your PATH environment variable. (EC-13533)

    • For Linux GUI installation mode, the installer’s final dialog box displays To add emake to the PATH - execute command source /opt/ecloud/i686_Linux/conf/ecloud.bash.profile . For Linux interactive command line installation mode, the installer displays this message as well.

    • For both Windows installation modes (GUI and silent), the installer automatically adds the full Accelerator installation directory path to the PATH variable. Note that if you start the installer in silent mode from PowerShell Core or a cmd.exe command prompt, you must run a new independent session of pwsh.exe or cmd.exe as applicable to add the full Accelerator installation directory path to the PATH variable.

  • (Linux platforms only) On systems that have an /etc/profile.d directory, the installer now creates an /etc/profile.d/accelerator.sh bash shell script, which automatically adds the Accelerator /opt/ecloud/i686_Linux/64/bin directory to the PATH environment variable. This functionality makes it easier to invoke eMake and CloudBees Accelerator Insight because you do not need to enter the full paths to these executables. Note that the accelerator.sh script is the same script as the /opt/ecloud/i686_Linux/conf/ecloud.bash.profile script. (EC-12098)

2019.08.00

Support for JobCache in Electrify (Linux only)

Support for JobCache in Electrify on Linux platforms is introduced. For details about this enhancement, see Electrify. (EC-13150)

ecmsbuild.exe Wrapper

The ecmsbuild.exe wrapper, which provides a simpler command syntax for using Electrify to build Visual Studio solutions and MSBuild projects from the command line, is introduced.

Cluster Manager Web UI Reskin

The Cluster Manager web UI is reskinned to match that of other CloudBees product web interfaces. (EC-13393)

Monitoring Build Progress on the Console for Command-Line Builds

The --emake-show-progress eMake option is introduced. This option outputs the build progress, the number of agents running jobs, and other statistics as well as error messages to the console. For details, see eMake Command-Line Options, Environment Variables, and Configuration File. (EC-13390)

Support for using the Link Time Optimization (LTO) build option with GCC is added.

Android Build Improvements

The Accelerator Android integration now supports Android builds that are invoked as components of another build. (EC-13402)

Visual Studio Integration Update

The Visual Studio Integration (which is bundled with Accelerator) is upgraded to preview version 2019.08.00. For a list of new features and resolved issues in this release, see Visual Studio Integration Guide - What’s New.

Rebranding
  • The ElectricAccelerator Error Messages Guide is rebranded with CloudBees Accelerator branding and renamed to the CloudBees Accelerator Error Messages Guide . (EC-13436)

  • The installation process is rebranded with CloudBees Accelerator branding. This includes all installer file names and all installer modes (GUI, console, and silent modes). (EC-13430)

  • The Cluster Manager login web page is redesigned and rebranded with CloudBees Accelerator branding. For details, see Cluster Manager Guide - Signing In To the Cluster Manager Web UI. (EC-13413)

Behavior Changes

None.

2019.09.00

None.

2019.08.00

None.

Resolved Issues

None.

Other Issues

  • On Android 8. x , 9. x , and 10 builds, an issue is fixed in which $OUT_DIR/ was not used in the override rules in the patch to the build/core/main.mk file (but was hardcoded as out/ instead). (EC-13580)

  • On Android 10 builds, an issue is fixed in which the file name pattern filename.zip in the out/target/product/generic/obj/CONFIG/kati_packaging/dist.mk packaging dependencies file that Accelerator generates should have been name.$USER.zip , where $USER is from BUILD_USER , which must be set in the environment when kati is executed. This issue triggered cache misses in the next eMake build. (EC-13577)

  • (RHEL 8.0 platform only) The installer no longer reports when the ncurses-libs.x86_64 library is missing even though this library is not required. (EC-13545)

  • An issue where the output that was sent to the console when you used the --emake-show-progress eMake option sometimes stopped when warnings were encountered during a build is fixed. Now, only the latest warning encountered during the build appears on the console, and all warnings are redirected to the emake.out file. (EC-13507)

2019.09.00

None.

Other Issues
  • (Linux platforms only) An issue where Android builds failed with an unexpected EOF while looking for matching `)' error is fixed. (EC-13544)

2019.08.00

None.

Other Issues
  • (Windows platforms only) Corrected the error code returned when attempting to use certain invalid characters in file names. (EC-13450)

  • In Ninja emulation mode, eMake now considers all outputs (not just the first output) of a job to determine if the job is up-to-date. (EC-13431)

  • (Linux platforms only) A NoSuchMethodError message that occurred when the first build with on-demand agents started on Google Cloud Platform (GCP) is fixed. (EC-13425)

  • An issue with GCP cloud burst instances in which the static host and agents were not shown as matching on the Resources page in the Cluster Manager web UI is fixed. (EC-13317)

  • (SLES 15 platform only) The error, The upgrade of MariaDB has failed , that sometimes occurred at the end of Cluster Manager installation is fixed. (EC-13254)

Platform Support

No changes.

2019.09.00

Support is added for the following platforms.

  • RHEL 8.0 (kernel 4.18.0-80) (64-bit) (EC-13433)

2019.08.00

No changes.

Database Support

No changes.

2019.09.00

No changes.

2019.08.00

No changes.

Browser Support

No changes.

2019.09.00

No changes.

2019.08.00

No changes.

Installation and Upgrade Notes

New Accelerator Release Strategy

Starting in August 2019, the release strategy for CloudBees Accelerator is updated to add a “preview” release in addition to the standard long-term support (LTS), maintenance (patch), and hotfix releases.

The release numbering for the preview releases uses a new <year> . <month> .00 numbering scheme. For example, {accelerator-product-version-accelerator-next} .

CloudBees recommends that you upgrade to the preview release and test these features in a controlled environment before rolling them out to production.

For details about the new release strategy, see the CloudBees maintenance lifecycle policies web page.

Version Enforcement in Cluster Manager Licensing

As of version 11.1, perpetual license files include a Version field and are therefore no longer transferable to succeeding product versions. At run time, to be considered valid by the Cluster Manager, the license file must have a Version that equals or exceeds the version of the Cluster Manager itself. Version checks consider only the major.minor version (for example, a license for 11.1 is valid for 11.1.0, 11.1.1, 11.1.2, and so on).

eMake authenticates with the Cluster Manager via the eMake/Cluster Manager protocol version to ensure that the eMake and Cluster Manager licenses match. If the license has no Version field, it will be considered invalid by the 11.1 Cluster Manager.

You can upgrade just the Cluster Manager, but starting with version 11.1, if you upgrade eMake or agents, you must also upgrade the Cluster Manager. When you upgrade the Cluster Manager, you must also acquire a new license if you are currently using a perpetual license.

You can upgrade to new patch releases (but not new feature releases) without acquiring a new license. For example, if your current license specifies 11.1 , then you can use a Cluster Manager with version 11.1 , 11.1.1 , 11.1.2 , and so on. Also, for example, if your license specifies 12.0 , then you can use a Cluster Manager with version 12.0 , 12.0.1 , 12.0.2 , and so on, as well as 11.x .

The Administration > Licenses page in the Cluster Manager web UI has a Version column, which provides the product version of each license in the list. This information is blank for a license with no explicit version (such as an evaluation license).

Make sure that you import a compliant license into the Cluster Manager before upgrading. For assistance, see https://support.cloudbees.com/ . For details about importing a license, see Logging In and Enabling Licensing - Importing Your License. (EC-13170 and EC-13154)

Hardware Requirements

  • Accelerator 7.2 and newer versions require a Pentium 4 or newer processor when running in a 32-bit Solaris x 86 environment.

  • The recommended total amount of RAM for an agent host is 2 GB per agent plus the amount of RAM normally needed to execute your build. For example, if you are running four agents, and your build normally needs 16 GB, you will need ((2 * 4) + 16) = 24 GB.

Backing Up Before You Upgrade

  • The upgrade process does not preserve the existing files. Back up the /opt/ecloud/<arch>/cloud directory for Linux and Solaris or the C:\ECloud\<arch> folder for Windows to a safe location.

  • For additional security, back up the database by following the recommended procedure from your database vendor.

Installing JDBC Drivers for MySQL or Oracle Databases

CloudBees no longer distributes the JDBC drivers for MySQL or Oracle databases. To use one of these databases, you must download its driver directly from the Oracle website, then copy it to the appropriate directory on the Cluster Manager server, and then restart the Cluster Manager service. For details, see Installing JDBC Drivers for MySQL or Oracle Databases.

Copying the execserver Executable to a New Location if You Relocate eMake

If you copy the emake executable to a new location, you must also copy the execserver executable to that location. By default, the path to the execserver executable is /opt/ecloud/i686_Linux/32/bin/execserver (for 32-bit eMake) or /opt/ecloud/i686_Linux/64/bin/execserver (for 64-bit eMake).

Regenerating History Files After an Upgrade

The identifier that is used to find certain types of jobs in the eMake history file changed in version 8.0. After an upgrade from version 7.2.2 or older versions to version 8.0or newer versions, users should regenerate their history files by running their first build with the --emake-history=create option to avoid unnecessary serializations. This build might have more conflicts than normal (but subsequent builds should return to normal).

Concurrent Build Licensing

As of version 9.1, for new Accelerator subscription licenses, the number of builds that you can run concurrently is license-limited. The noLicenseWaitTime performance metric indicates the amount of time that a build spent waiting for a concurrent build license because the number of concurrent builds reached the license limit. Also, as of version 9.1, JobCache is not separately licensed and is now included with the concurrent build license.

Customers using pre-9.1 Accelerator licenses may continue to use those licenses, including the licenses for the JobCache add-on.

For details about licensing for concurrent builds, see the Logging In and Enabling Licensing. (EC-12095)

Known Issues

Linux Kernel Issue That Affects Accelerator Performance

Affected Kernel Versions

  • RHEL kernel versions later than 2.6.18-194.32 and earlier than 2.6.32-131

  • Ubuntu Linux kernel versions 2.6.31, 2.6.32, 2.6.33, and 2.6.34

Symptoms

Affected systems might encounter reduced performance on both ext3 and ext4 file systems. Symptoms might include

  • hung_task_timeout_secs messages in system dmesg logs

  • Widely variable agent availability (entering and exiting agent “penalty” status frequently)

  • Contention over the ecagent.state file

  • Slower builds (with unexplained variances)

To help determine if this issue exists, run the dmesg | grep hung_task_timeout command. hung_task_timeout errors show that this issue is present. Contact your kernel provider for another version of the precompiled kernel.

Fixes for Systems Running RHEL 5.6, 5.7, 5.8, and 6.0

You should consider upgrading to 2.6.32-131 (RHEL 6.1) or downgrading to 2.6.18-194.32 (RHEL 5.5).

Other Known Issues

  • Android M is not compatible with Accelerator 11.1. You must use Android M with Accelerator 10.0.

  • If you kill a build manually, and the agents running on an Amazon EC2 instance fail to connect, the instance will continue to run. You must kill the instance manually.

  • If the Cluster Manager goes down while agents are running on an Amazon EC2 instance, they will still connect but will not be associated with a resource. You must kill the corresponding instance manually.

  • The cmtool importData command does not import license properties (such as maxAgents ). To work around this issue, re-import the license after using importData . (EC-12371)

  • You cannot control breakpoints from the Cluster Manager. (EC-12322)

  • If Apache fails to start properly after a new Cluster Manager installation, reboot the system.

Documentation

Product Documentation

Accelerator documentation for the latest Technical Preview release is available at CloudBees Accelerator:

  • CloudBees Accelerator Installation and Configuration Guide

  • CloudBees Accelerator Electric Make User Guide

  • CloudBees Accelerator cmtool Reference Guide

  • CloudBees Accelerator Error Messages Guide

  • CloudBees Accelerator Visual Studio Integration Guide

  • CloudBees Accelerator Release Notes (this document)

  • PDF, HTML, and mobile-optimized HTML versions of the online help that is also accessible from the Cluster Manager Help button.

Documentation on the website is updated periodically.

Cluster Manager Online Help and Tooltips

Built into the Cluster Manager are

  • A complete, robust, context-sensitive online help system (click the Help button in any page of the Cluster Manager web UI). See the documentation website as described above for the latest updates to this information.

  • Tooltips with information to help fill in form fields.