CloudBees CD/RO v2023.03.0
CloudBees is pleased to announce the v2023.03.0 long-term support (LTS) release of CloudBees CD/RO. With this release, CloudBees added several new features and system improvements, including:
-
Advanced microservices deployment strategies with Argo Rollouts
-
YAML DSL enhancements
-
Jenkins interactive input support
-
API
changeOwner
enhancement -
Common tools archive for Linux distributions
-
Agent images with preinstalled tools
-
Proxy support for email configurations
-
EKS v1.25 support
-
Improved Kubernetes upgrade instructions
Refer to New features and Feature enhancements below for more information.
Security fixes
- This release includes the following security updates to address potential vulnerabilities:
-
-
Apache web server is upgraded from 2.4.54 to 2.4.55. For details, refer to the Apache HTTP Server Project. [BEE-29816]
-
The Apache ZooKeeper server included in CloudBees CD/RO Docker images has been upgraded to v3.8.1. [BEE-17920]
-
Elasticsearch is upgraded to 7.17.9. [BEE-29817]
-
OpenSSL is upgraded from 1.1.1s to 1.1.1t.[BEE-29816]
-
PHP is upgraded from 8.1.14 to 8.1.16. For details, refer to the PHP 8.1.16 Release Announcement. [BEE-29816]
-
New features
- Advanced microservices deployment strategies with Argo Rollouts
-
CloudBees has added the ability to use Argo Rollouts to provide advanced deployment capabilities, such as blue-green, canary, canary analysis, experimentation, and progressive delivery features to Kubernetes. To support this feature, the following changes were made:
-
CloudBees now supplies
cbflow-agent
images withkubectl
v1.16.1 and the kubectl-argo-rollout plugin v1.4.0 preinstalled for Argo Rollouts.For more information, refer to Install and update CloudBees CD/RO agents.
-
Default rollout options allow you to set the default assigned approvers and turn on/off email notifications when rollout deployment is configured with manual approval. If approvers are not specified, the user who launched the deployment is auto-assigned as the approver. These default values can be overwritten later during process runs, in pipeline tasks, or in the release deployer configuration.
-
This enhancement includes the following UI updates:
-
- Jenkins interactive input support
-
Allows for manual approval of a CloudBees CI build job in CloudBees CI from CloudBees CD/RO. If a CloudBees CI build job triggered by CloudBees CD/RO pauses waiting for user input, the CloudBees CI Input icon displays on the task. If you select this icon, the system takes you directly to the approval screen in CloudBees CI for you to approve. Once approved, the system returns to the build data window and displays the continuing pipeline run.
For more information, refer to Native CI integration.
- Common tools archive for Linux distributions
-
As of release 2023.03.0, CloudBees CD/RO now includes an option to manually download and install a common tools archive for Linux distributions. This archive includes the following tools:
-
ectool
-
ecclientpreflight
-
ec-groovy
-
ec-perl
-
postp
-
This option may be helpful if you do not want to over-assign ROOT privileges to installers on your platform. For more information, refer to Install CloudBees Tools from archive.
Feature enhancements
- CloudBees CD/RO agent images with preinstalled tools
-
Starting with CloudBees CD/RO 2023.03.0, agent images are available with the following preinstalled tools:
-
Helm v3.11.1
-
kubectl v1.26.1
-
kubectl-argo-rollouts v1.4.0
-
For more information, refer to CloudBees CD/RO agent images with preinstalled third-party tools.
- Enhanced
changeOwner
API -
The
--recursive
argument of thechangeOwner
method has been enhanced to apply the ownership changes of pipelines, releases, andflowRuntime
objects to associated child objects. When set to the default value offalse
, ownership changes are applied only to the selected object. If set totrue
, ownership modifications are applied as follows:-
Pipeline and releases: Changes are applied to stages, gates, tasks, associated flow and flow state objects, and common objects. Common objects are properties, email notifiers, event subscriptions, and formal parameters.
-
flowRuntime
: Changes are applied toflowRuntimeState
, nestedflowRuntime
objects, and common objects. Common options are properties, email notifiers, event subscriptions.
-
- YAML DSL enhancement
-
The DSL IDE was enhanced to display object hints. When you place your cursor on an object, such as Object ID, and enter the keyboard shortcut, a help window displays the object details.
For more information, refer to DSL editors.
- Proxy support for email configurations
-
CloudBees CD/RO now supports proxies in email configurations. To add proxy configuration for SMTP traffic:
-
On traditional platforms, update your CloudBees CD/RO server
wrapper.conf
file with the following entries:-
wrapper.java.additional.2000=-Dmail.smtp.proxy.host=<yourhost>
-
wrapper.java.additional.2010=-Dmail.smtp.proxy.password=<yourpassword>
-
wrapper.java.additional.2020=-Dmail.smtp.proxy.user=<youruser>
-
wrapper.java.additional.2030=-Dmail.smtp.proxy.port=<yourport>
-
-
For Kubernetes installations, the following lines must be passed in the
flow-server
Helm chart asecconfigure
commands:server: ecconfigure: "--wrapperJavaAdditional=10001=-Dmail.smtp.proxy.host=<yourhost> \ --wrapperJavaAdditional=10002=-Dmail.smtp.proxy.password=<yourpassword> \ --wrapperJavaAdditional=10003=-Dmail.smtp.proxy.user=<youruser> \ --wrapperJavaAdditional=10004=-Dmail.smtp.proxy.port=<yourport>"
-
- Improved Kubernetes upgrade instructions
-
Improved instructions for upgrading from Kubernetes v1.21 and earlier to v1.22 or later. The instructions now include the required steps for disabling
nginx-ingress
and enablingingress-nginx
, as required by Kubernetes. For more information, refer to Upgrade CloudBees CD/RO on Kubernetes.
Plugin enhancements
- CloudBees CD/RO plugins catalog
-
The CloudBees CD/RO plugins catalog is available on the CloudBees CD/RO documentation site.
For more information about plugin support and versioning, refer to Plugin Concepts.
- Plugin updates
EC-DslDeploy 4.1.9 |
Fixed regression with multiline property values generation. |
EC-FeatureFlags 1.2.5 |
Updated third-party library dependencies. |
EC-GCP-ComputeEngine 2.5.11 |
Fixed the null value in the ListInstances JSON output. |
EC-Helm 1.7.0 |
Internal improvements for CloudBees CD/RO microservices deployment. |
EC-Kubectl 1.3.0 |
Updated the logger. |
EC-Selenium 2.0.10 |
Released to the community. |
EC-SonarQube 2.1.3 |
Fixed an issue with the Ignore SSL errors option for the Get Last SonarQube Metrics and CollectReportingData REST-based procedures. |
EC-SQLServer 3.0.0 |
The plugin has been migrated to the PDK framework, and now supports PDK model plugin configurations. Upgraded from Perl 5.8 to Perl 5.32. The plugin is not backward-compatible with releases prior to CloudBees CD/RO 10.3. Starting with 3.0.0, a new agent is required to run the plugin procedures. |
Trigger-Property 2.0.0 |
Upgraded from Perl 5.8 to Perl 5.32. The plugin is not backward-compatible with releases prior to CloudBees CD/RO 10.3. Starting with this release, a new agent is required to run the plugin procedures. |
- Plugin Development Kit enhancements
-
None.
New platform support
This section lists new platform support.
- EKS v1.25 support
-
As of v2023.03.0, CloudBees CD/RO has been tested on EKS v1.25 and is supported.
Refer to the following topics for a list of officially supported platforms for CloudBees CD/RO:
Resolved issues
BEE-14396 |
When an artifact name and version are specified in a component with a property reference, the |
BEE-28130 |
Fixed an issue that caused an error when a pipeline manual task was saved without an approver. |
BEE-28351 |
Increased the CloudBees CD/RO web server memory limit from 100M to 250M to resolve an issue with DSL exports. To set the CloudBees CD/RO web server memory limit, use the |
BEE-28889 |
Fixed an issue that caused empty values in the Top 10 SCM Repositories with the most file changes report. |
BEE-28938 |
Updated the |
BEE-29185 |
Unique constraints work on MySQL, MariaDB, and PostgreSQL databases when a release and a deployed application are located in the same project. |
BEE-29863 |
The |
BEE-29233 |
The SeedInventory procedure no longer updates multiple resources for an environment when only one resource is specified. |
BEE-29932 |
Fixed a NPE in the archiving service that occurred when the |
BEE-30031 |
Fixed an issue with restarting failed pipelines that caused unselected stages to restart. |
BEE-30093 |
Fixed an issue in v10.11 that prevented importing projects with custom properties using EC-DslDeploy. |
BEE-30138 |
Fixed an issue with the See more link on the pipeline Properties page in the UI. |
BEE-30182 |
Updated the default value of the Track changes to value property option to be |
BEE-30376 |
Fixed an issue with CloudBees Analytics dashboard imports with the |
BEE-30466 |
Fixed a StackOverflowError issue with masking secrets in logs. |
BEE-30468 |
Applied By default, the legacy mode is enabled in the CloudBees CD/RO server If your platform contains ill-formed Groovy scripting and does not use URL encoding, it could be possible to exploit in unexpected ways. To disable this option and configure your CloudBees CD/RO server to be more secure, use the following |
BEE-30579 |
seedEnvironmentInventory returns the last successful version for one particular resource if it was seeded only for one resource. |
BEE-30642 |
Fixed an issue in the |
BEE-30939 |
The |
Installation notes
For complete installation and upgrade information, refer to CloudBees CD/RO on Kubernetes and Install CloudBees CD/RO on traditional platforms.
CloudBees deprecated the CloudBees CD/RO ec-jruby and ec-jython wrapper programs with v10.11. The wrapper programs are no longer installed as part of CloudBees CD/RO tools.
|
CloudBees CD/RO server installation binaries are signed for traditional installations so that you can verify their origin and authenticity. Verifying binaries is an optional step in the installation process than can help ensure you are not the victim of a man-in-the-middle attack. For more information, refer to Verify installation binaries.
- CloudBees CD/RO on Kubernetes
-
Sample CloudBees CD/RO server and agent Helm chart values provide the CloudBees default installation values. The CloudBees CD/RO
images.tag
value associated with v2023.03.0 is2023.03.0.161439_3.2.38_20230307
.
Starting with 2023.03.0, CloudBees introduced the ability to add additional environmental variables for Kubernetes CloudBees CD/RO agents. You can add environmental variables in the cloudbees-flow-agent
Helm chart using the entry for extraEnvs
.
CloudBees CD/RO Docker images and Helm charts are signed so that you can verify their origin and authenticity. Verifying Docker tags and Helm charts is an optional step in the installation process than can help ensure you are not the victim of a man-in-the-middle attack. For more information, refer to Verify Docker images and Verify Helm charts.
CloudBees CD/RO v2023.03.0 now supports Kubernetes v1.25. As a result, Kubernetes PSP support is now deprecated in CloudBees CD/RO 2023.03.0 Helm charts.
- CloudBees CD/RO Universal Base Image (UBI)
-
The actual UBI associated with v2023.03.0 is
9.1.0-1760.1675784957
. - Upgrading gateway agents
-
All gateway agents that meet the following criteria must be updated to CloudBees CD/RO v10.2+:
-
Your enterprise implements a multi-zone environment.
-
Agent versions are a combination of pre-v10.2 and v10.2+.
-
The access route to a v10.2+ agent is configured through a pre-v10.2 gateway agent.
-
- Configuring autostart services for Linux installations
-
Linux installations that you perform as a non-root user or without
sudo
permissions cannot automatically start the CloudBees CD/RO server, web server, repository server, or agents. Instead, you must set up service autostart after installation is complete. Refer to Configure autostart for non-root/non-sudo Linux installations to learn more.
- Upgrading your CloudBees CD/RO environment
-
Before starting an upgrade, make sure to back up your existing CloudBees CD/RO data. - Upgradable versions
-
Upgrades to CloudBees CD/RO 10.x are supported only from ElectricCommander 5.0. For upgrade instructions, refer to the Upgrade on traditional platforms.
- Updating the MySQL configuration before upgrading
-
Since release 8.0.1, CloudBees has instructed customers using a MySQL database to add the following two lines to their MySQL configuration:
init_connect='SET collation_connection = utf8_unicode_ci, NAMES utf8' skip-character-set-client-handshake
Before upgrading CloudBees CD/RO, you must remove these lines or comment them out. Otherwise, jobs will not start.
- Ensuring the correct default MySQL default collation
-
Make sure that the default collation for the MySQL database schema is set to
utf8_unicode_ci
orutf8_general_ci
and that no table in the schema overrides this. The CloudBees CD/RO server checks this configuration on startup and logs errors in the server log if it is not set correctly.If the collation is not configured correctly, entering non-ASCII text into CloudBees CD/RO can cause errors. For example, setting a release name to a non-ASCII value, and attempting a search, causes an exception.
If your MySQL database schema, or any tables within, are set to a non-UTF-8 collation order, refer to the Knowledge Base article KBEC-00385 - Converting a MySQL Database From Latin-1 to UTF-8 for detailed instructions about safely converting your schema to UTF-8. [NMB-26521, NMB-27459]
- Upgrading agents that run the
ec-groovy
job step in multizone deployments -
In multizone CloudBees CD/RO deployments, CloudBees CD/RO agents that are in a different zone than the CloudBees CD/RO server must be upgraded to version 9.0 or later for the
ec-groovy
job step to run successfully on those agents. You must also upgrade the gateway agents that lead back to the server’s zone, including those in any zones in between the agent’s zone and the server’s zone. [NMB-27490]For details about multiple zones and gateway agents, refer to Zones and gateways.
- Removing the
SSL 2.0 Client Hello
orSSLv2Hello
protocol from your security configurations -
CloudBees recommends removing the
SSL 2.0 Client Hello
orSSLv2Hello
protocol from your security configurations for all components. [NMB-27934, NMB-29326]-
Upgrade agents to the latest operating system version for security reasons.
-
If this warning appears on the Automation Platform UI:
Note: We recommend removing `SSL 2.0 Client Hello` format from server configuration and upgrade older agents as indicated on the Cloud/Resources Page to avoid security risk.
then enter the following command on the CloudBees CD/RO server:
$ ecconfigure --serverTLSEnabledProtocol=TLSv1.2
-
- Upgrading the CloudBees Analytics server
-
This section provides information about upgrading the CloudBees Analytics server.
-
It is not possible to upgrade CloudBees Analytics v9.0.1 and below to CloudBees Analytics v10.2.0 and above. The installer exits with an error and an appropriate message when such an update is attempted. If you need to upgrade CloudBees Analytics v9.0.1 and below, you must first upgrade to a version between 9.1.0 and 10.1.0, or 9.0.2 and above. After that, you can upgrade CloudBees Analytics to v10.3.0 or higher. [NMB-31030]
-
For previous CloudBees Analytics upgrades from v9.0.1 and below: CloudBees Analytics data may contain obsolete indexes that are incompatible with CloudBees Analytics v10.2.0 and above. To work correctly, it is necessary to re-index these indexes before an upgrade. The installer prompts you to do this before upgrading.
-
In console mode and UI mode, the installer displays the following prompt if outdated indexes are detected:
One or more Elasticsearch indexes were created in an obsolete version of Elasticsearch. These indexes must be re-indexed for the upgrade to be successful. Do you want to start the reindexation? [n/Y]
After an affirmative answer, the installer automatically re-indexes and continues the upgrade.
-
In silent mode, the installer reindexes automatically.
-
-
Backing up and restoring custom settings
The CloudBees Analytics installer overwrites the
elasticsearch.yml
configuration file with a new file. This file includes aCustom Settings
section, which lets you add Elasticsearch settings not managed by the CloudBees Analytics server without being lost during an upgrade. The installer preserves the settings in theCustom Settings
section. [NMB-25850] -
Upgrading CloudBees Analytics clusters
The principle of forming a cluster in CloudBees Analytics has changed in v10.2 due to the update of Elasticsearch v7.10.2. In this regard, an additional action is required to upgrade to CloudBees Analytics v10.2 or later:
When updating the first master node, you must explicitly specify that it is the first node to be updated. If this action is not performed, any cluster that is being updated is placed out of service.
All installers have been instrumented to accommodate this change. Refer to Upgrade the CloudBees Analytics server for more details. [BEE-2717]
-
CloudBees Analytics server configuration notes
For a production environment, CloudBees recommends that you install the CloudBees Analytics server on a system separate from systems running other CloudBees CD/RO components (such as the CloudBees CD/RO server, web server, repository server, or agent). If you must install it on the same system (such as for testing or other non-production or trial-basis situations), refer to CloudBees Analytics server with other components for details.
If your CloudBees Analytics server is configured with multiple nodes in a Kubernetes environment, you must pre-generate your certificates. For more information, refer to Install CloudBees CD/RO within Kubernetes.
Configuration notes
- Performing a full import
-
During a full import, the import operation might hang in the following scenarios. To import successfully into CloudBees CD/RO 8.0 and newer versions, perform the appropriate workarounds [CEV-15447, CEV-11873]:
-
A manual process step in a process has formal parameters. The workaround is to remove the entry related to the property sheet for the job step that is associated with the manual process step.
-
In the exported XML file from an earlier release, two pipelines are in different projects, and both pipelines have no gate tasks. The flow associated with the pipeline is duplicated under both projects. The workaround is to remove the flow element under the projects.
-
- Limitations
-
When an application is cloned from one project (the original project) to another (the destination project), the tier maps for the application point to the environments with the same names in the destination project. To deploy the application to the environments in the original project, you must first create tier maps connecting the application to those environments.
Known issues
BEE-7512 |
With CloudBees CD/RO v10.2.1 and earlier, the DSL Import service catalog fails for grouped tasks. |
||
BEE-14581 |
The MeanLeadTime report does not work correctly when Elasticsearch has pipeline runs but no release runs. |
||
BEE-14933 |
The UI does not allow the transfer of artifacts across zones. |
||
BEE-17259 |
When a custom data retention policy schedule is set to run once, the data is not purged after archiving. To purge data after archiving, use a repeat schedule or the global data retention setting. |
||
BEE-19742 |
When you save DSL for a dropdown menu, the code is evaluated to catch syntax errors. This evaluation is not the same as when the parameter is used. This can result in a property reference error because the properties may only be available when the parameter values are set. A workaround is to use a try-catch statement where the
|
||
BEE-20205 |
CloudBees CI job tasks with a |
||
BEE-20536 |
When using Postgres with change tracking enabled, EcAuditStrategy errors may appear in the server log. This is a known issue, but is not expected to affect system performance. |
||
BEE-27713 |
Events that originate from the default CloudBees CI create default configurations. URLs for these new controllers are not Jenkins configured URLs and cause 401 errors. |
||
BEE-28886 |
You may experience SSO sign-in issues when using Kerberos due to a Microsoft known issue. |
||
BEE-29494 |
When a process step that is not manual is modified to be manual after the process runs but before the associated job step evaluated, the step hangs and adds a |
||
BEE-28318 |
When upgrading to CloudBees CD/RO 10.6 (Helm chart version 2.16.0) from previous versions, the As a workaround, in your
|
||
BEE-31027 |
When you restart a running pipeline with an active CloudBees CI job task, the task is aborted with |
||
NMB-30095 |
Browser redirects to port 2080 during first navigation to CloudBees CD/RO deployed from CloudBees Software Delivery Automation and Flow Helm charts. |
||
NMB-24734 |
|
||
NMB-24949 |
When you use the Automation Platform UI to upload and publish artifact files with non-English characters in their file names, the operation fails with the following error: |
||
NMB-26021 |
Modifications of LDAP user data (such as email addresses) on an Active Directory server after registration in CloudBees CD/RO do not appear properly in user details (in the Automation Platform UI, the Deploy UI, or |
||
NMB-26962 |
(Microsoft Windows platforms only) If the Elasticsearch cluster used by CloudBees Analytics is in the red state (meaning that it only partly functions and some data is unavailable), then upgrade, reconfigure, and uninstall operations will not work. Since the Elasticsearch service cannot be stopped when a cluster is in a red state, you must stop the Elasticsearch service process from the task manager before running the installer for these actions. |
||
NMB-28135 |
The Microsoft Edge browser does not work with SAML 2.0 and is missing a self-signed certificate during redirection from the identity provider to the service provider. Edge is not recommended for sign-in via SAML 2.0. |
||
NMB-29486 |
The LANG environment variable must be set to |
||
CEV-11106 |
When an application with snapshots created in CloudBees CD/RO 6.1 or earlier is cloned and a project containing this application is imported to CloudBees CD/RO 6.3 or higher, the import operation fails. |
||
CEV-12363 |
Error prompts for runtimes started by a schedule are not visible if the schedule was created with a missing configuration. |
||
CEV-12429 |
The stage inclusion status in the Release Dashboard changes color after a stage is renamed. |
||
CEV-14689 |
No error prompt appears for failed tasks and retry tasks during a pipeline runtime. |
||
CEV-15122 |
If an application process step cannot expand to its child steps (because of an invalid run condition or an invalid formal parameter), then the step is not retried even if it uses "retry on error" error handling. The job eventually completes with an error. |
||
CEV-15829 |
The retry count for group tasks or rules using "automated retry on error" is missing from the Pipeline runtime page. |
||
CEV-16245 |
Multiple mapped environments with the same name from different projects are not supported in email notifications. |
||
CEV-16250 |
A project import might not include the path-to-production view. |
||
CEV-16930 |
Jobs might not appear upon drill-down into the "Clusters With Most Deployments" widget in the CloudBees Analytics Microservices Dashboard if the service does not contain a deploy step in the process. |
||
CEV-18531 |
All subreleases of a release must appear before the release in the DSL for the release-to-subrelease link to be created. |
||
CEV-19239 CEV-19259 |
The ability to search by assignee in a Deployment Report is not available in the CloudBees Analytics report editor. |
||
CEV-21426 |
If Release Command Center was set up for JIRA for user stories and defects, and the JIRA project name was mapped to the release project name using the field mapping |
||
CEV-23624 |
Approval by email on manual tasks should not expect parameters. |
||
CEV-25150 |
If you use the |
||
CEV-26700 |
SSO does not work unless PHP configuration is changed due to a security-related request. Workaround: Change |
||
CEV-28704 |
CloudBees CD/RO v10.1 introduced new triggers and an updated UI for them. Pre-v10.1 triggers will continue to work but there is no UI to review or run them. |
||
CEV-28779 |
Before using the export command to perform a full data export from the CloudBees CD/RO database, delete any legacy definitions and references to |
||
N/A |
You can revert changes only for high-level design objects such as applications procedures, procedure steps, workflow definitions, and state definitions.
|
||
N/A |
Enabling Recursively Traverse Group Hierarchy might impact system performance when the LDAP group hierarchy is traversed. The amount of impact varies with the configurations of the CloudBees CD/RO and LDAP servers, the depth of group hierarchy in the LDAP server and the network latency between the servers. Make sure that your directory provider can handle the additional load for supporting nested group hierarchy traversal. |
||
N/A |
System performance might decrease if you disable change tracking at the server level and then re-enable it. Change tracking is enabled by default. For details about using change tracking, refer to change tracking. |