CloudBees CI release highlights
What’s new in CloudBees CI 2.414.3.7
- Upgraded multiple dependencies to address security concerns
Apache Ivy has been upgraded from 2.5.1 to 2.5.2 in the
Guava has been upgraded from 31.0.1 to 32.1.2 and had j2objc-annotations:2.8 added in the
Guava has been upgraded from 31.1-jre to 32.1.2-jre in the
Jetty IO has been upgraded from 10.0.13 to 10.0.16 where applicable.
Jetty XML has been upgraded from 10.0.13 to 10.0.16 where applicable.
commons-httpclientlibrary used in WebDAV and Azure stores (transitively through
com.sun.jersey:jersey.client) is vulnerable to man in the middle (MITM) attacks. The fix removes the
commons-httpclientlibrary and uses the native Java HttpClient.
- Add GitLab support in the CloudBees Assurance Program (CAP)
The GitLab Branch Source plugin and the GitLab API plugin were added to the CloudBees Assurance Program (CAP). These plugins enable users to have a supported way of creating multibranch and organization folder jobs backed by a GitLab server. These plugins will now receive necessary updates and security fixes under the CloudBees Assurance Program (CAP).
- Bundle Update Timing is now available
The new CloudBees CI feature Bundle Update Timing is now available. With this new feature, instances using Configuration as Code (CasC) for Controllers can be configured to:
Automatically reload a new bundle version (If the bundle can be hot reloaded, the current restrictions remain.).
Automatically schedule a safe restart after the new bundle is detected.
Skip any new version detected. It is also possible to skip the versions when the instance is up and running and later apply the bundle during a restart.
Skip a concrete bundle version so it is never applied.
Reject or accept bundle versions with warnings, leaving the acceptance threshold to administrator users (administrator users can configure the instance).
Bundle Update Timing is enabled by default. However, if desired, you can disable Bundle Update Timing by setting the system property
-Dcore.casc.bundle.update.timing.enabled=falsein the start script.
- Adding and removing Maven-related plugins from CAP
The Maven Integration (
maven-plugin) plugin was removed from the CloudBees Assurance Program (CAP). As an alternative to this plugin, CloudBees recommends the use of the Pipeline Maven Integration plugin (
The Pipeline Maven plugin (
pipeline-maven) and Pipeline Maven API (
pipeline-maven-api) were added to CAP. See the upgrade notes for more details.
The CloudBees Maven Migration Assistant plugin (
cloudbees-maven-migration-assistant), a plugin to help you migrate your Maven jobs to Pipeline jobs, is available from the CloudBees Update Center.
Bulk migration Groovy scripts will soon be available to ease the migration of Maven jobs.
- Configuration as Code Operations center Bundle Retriever for SCM
The CasC Operations center Bundle Retriever is a service that ensures Operations center CasC bundles are always available and updated before the Operations center starts. The CasC Bundle Retriever is deployed alongside the Operations center container and allows you to manage it’s configuration via source control management (SCM) while ensuring that the latest bundle version is always locally available during both initialization and while running.
The Bundle Retriever uses a polling service to check the SCM by default. If GitHub is configured as the SCM then the Bundle Retriever can be configured to use webhooks for push notifications.
The Bundle Retriever can also be used in combination with Bundle Update Timing to optimize your Operations center CasC bundle management.
For more information refer to Retrieving bundles using SCM.
- Configuration as Code (CasC) for Controllers Bundle Report in the Operations center CasC Management Page
New ability to view a report of all of your controllers and their associated bundles. The report is now available on the Operations center CasC management page. The user can search for controllers as well as search and filter the bundles.
- CasC Items support for GitLab organization folders and multibranch projects
CasC Items plugin now supports GitLab organization folders and multibranch projects.
- Display queue size in the High Availability (HA) summary
Now, the queue size on each cluster member is displayed in the High Availability (HA) summary.
- Display the load per replica summary in the High Availability (HA) diagnostic page
The High Availability (HA) diagnostic page now displays a list of replicas and some load information, such as running builds and online agents.
- Include GitHub-related header values in the hibernation monitor logs
The pod logs now report the values of selected headers, without personal information, that can be correlated with webhook delivery logs in GitHub.
- Windows agent container images for CloudBees CI
Agent container images for CloudBees CI based on Windows Server Core and Nano Server LTSC2019 and LTSC2022 are now available. These images must be used if you’re using the new High Availability mode for CloudBees CI. For more information, refer to High Availability mode.
- When running a controller in High Availability (HA) mode, provision a pod disruption budget
When running a controller in replicated mode, set up a PodDisruptionBudget to ensure that the service is always running, even in maintenance events (node replacements). Use a pod disruption budget to ensure at least one replica is always up when scaling down node pools.
- Add network policy support
Network policy support for the Hazelcast port has been added when a controller uses High Availability (HA) mode.
- Improve texts on the configuration page for controller items
Adapted the label and inline help texts for the Configuration as Code (CasC) for Controllers property on the configuration page of controller items.
cluster-autoscaler.kubernetes.io/safe-to-evict=falseon controllers in High Availability (HA) mode
When using HA mode, you can evict replicas when you scale down a node pool as it does not cause the loss of availability.
- Load-aware build adoption in High Availability (HA)
Previously, when a High Availability (HA) controller terminated, its builds were adopted at random by other replicas. Now, the adoption is preferentially directed toward replicas that currently have the least load according to a simple metric.
- Base64 support for secret masking
Base64 support for secret masking was added in this release. Now when users configuring jobs use any type of credentials, the Base64 version of their secret is added to the list of patterns that the application masks. This should reduce the number of leaks when the verbose mode of curl (or an equivalent mechanism) is used. However, there are limits to credentials masking. Refer to Limitations of Credentials Masking for more information.
- When CloudBees Pipeline Explorer was enabled, some pages were very slow for complex Pipelines, leading to controller-wide performance issues in some cases
When CloudBees Pipeline Explorer was enabled, some pages, such as Blue Ocean and the Pipeline Steps view, were very slow for complex Pipelines. This can lead to high CPU usage on the controller, causing controller-wide performance issues. CloudBees Pipeline Explorer plugin now avoids performance-intensive computation in APIs that are used to access all of the Pipeline steps.
- Older versions of controllers cannot start after receiving new bundle versions
All operations center supporting Branch and Tag for controller CasC bundles cannot send new bundle versions to the controllers older than 2.401.1.3. When those controllers receive a new bundle version, they cannot start.
Now, bundles sent to controller versions older than 2.401.1.3 are accepted.
- Some invalid fields were not found during Configuration as Code (CasC) for Controllers bundle validation
Now a warning message is generated if a nonexistent or invalid field is found during validation.
- Credentials cache not working when controller disconnected from operations center
When a controller disconnected from the operations center, yet the operations center was still running, attempts to look up credentials from the operations center would fail rather than falling back to a cached list as expected.
This issue is resolved.
- Cluster Ops with backup controller on operations center is broken
Cluster Ops backup on operations center is failing because certain classes were not found during the backup cluster operation.
Cluster Ops with backup controller now functions correctly, but it requires the CloudBees Backup plugin (plugin ID:
infradna-backup) to be installed on the controller.
- Send a warning if the
managed-master-hibernationplugin is installed on a High Availability (HA) controller
Added an administrative monitor warning to administrators who run an High Availability (HA) controller that the
managed-master-hibernationplugin should be uninstalled as it is incompatible with it.
- Custom Update Center signature check uses an outdated certificate
The Update Center certificate was updated recently, and that modification needs to be taken care of for custom Update Center signature checks. The Custom Update Center signature now uses the updated Update Center certificate.
- New health check endpoint
Previously, the health check endpoint for the operations center pods used the
/loginURL, that creates a new Jetty session each time. Now, a new endpoint has been selected that uses an anonymous REST API that avoids session creation. (The analogous change was made for managed controllers in a previous release.)
- Updated expired embedded certificate of the Update Center
An expired certificate was previously embedded in the product, causing the download of some tool installers and plugins to fail. The certificate has been updated.
- Operations center startup blocked by Update Center item loading
The loading of Update Center items is slowed down by the sha256 and sha512 calculation of the stored plugin and core binaries. This can delay the startup of the operations center for several minutes for Update Center items with a large plugins base.
Checksums are now saved during download and manual uploads, and during loading only if the checksums file does not exist.
- Managed controller YAML validation section remains empty
prototype.jsusage was removed, the YAML validation section remains empty when creating managed controllers.
The YAML validation section now properly contains the necessary data.
- External groups not propagated from the Reverse Proxy Auth plugin used with the operations center SSO
When the operations center SSO is used and the operations center is configured with an SSO-type security realm that only defines a user’s external groups within the context of a browser request, a connected controller is authenticated with the correct user ID, but none of the groups. This issue has been resolved.
- Rename the CasC browser label
The CasC browser label
gitLabis used by both the
gitlab-branch-sourceplugins. The label used by
gitlab-branch-sourcehas now been changed to
- Optimized handling of job next build number in HA controllers
Synchronizing the next build number of jobs can be a bottleneck in HA controllers on slow network filesystems. This process is now improved and the issue has been resolved.
- More reliable cross-replica build loading for High Availability (HA) controllers
Previously, High Availability (HA) controller replicas could partially load build records even when the builds were in progress on other controllers, leading to errors. Now, a replica only loads a build record when the build is complete or when it is adopting a build in progress.
- Fix a logging thread contention when the client controller calls the Operations center.
In some cases, logging call errors to the Operations center could cause thread contention. Rate limiting has now been applied to these use cases to prevent logging from causing thread contention.
- UncheckedIOException related to bundle update can stop startup
File corruption in the bundle log directory can stop a controller startup, instead of a smooth recovery. This issue has been resolved.
- Excessive HTTP client threads created
Certain operations, such as contacting the operations center for user lookup, could create excessive Java threads. This issue has now been resolved.
- Retain queue ID when adopting queue items in High Availability (HA)
If you schedule a build using a High Availability (HA) controller, but it does not start before the replica shuts down, the corresponding adopted queue item in another replica now uses the same ID. This allows external tools to track the build when it does start (or the item is cancelled).
- Incompatible HA agent administrative monitor is displayed even if empty
In some cases, the administrative monitor that displays incompatible High Availability (HA) agents can be displayed, even when an incompatible agent is not found. This issue has been resolved and the administrative monitor is no longer displayed.
- Log recorders synchronization does not work for creation and deletion
The log recorders are now correctly synchronized between replicas of a controller in High Availability (HA) mode.
- CasC operations center RBAC "Granted at" offset is not honored when a group is created or updated
"Granted at" is now properly honored.
- YAML file is not validated in the Default YAML field of the Configure Controller Provisioning page
On the Configure Controller Provisioning page, the YAML file is not validated when it is configured in the Default YAML field.
- Auto-update bundle version may incorrectly update the wrong bundle version if multiple
bundle.yamlfiles exist in the repository
When using "Auto-update bundle version", the wrong bundle may be updated if multiple
bundle.yamlfiles exist in the repository.
- Wrapped token file can not be deleted by the operations center
.wrappedTokenfile is inadvertently being included in the bundle by the operations center bundle retriever and giving a structural warning,
[STRUCTURE] - The following files have been detected in the bundle folder but they are not yaml files: .wrappedToken.
- Add CloudBees CI communications to the CasC Bundle Retriever
When a bundle is retrieved from the SCM there are benign warnings in the logs about communication with the operations center.
- Error when renaming an existing EC2 cloud
When the name of an existing cloud node is updated, the user receives a 404 error after selecting save. This is because the cloud page uses the cloud name as part of its URL. When the user saves the name, Jenkins sends the user to the URL with the old cloud name. Please note that all node changes are saved successfully.
- Duplicate Pipeline Template Catalogs in the Configuration as Code (CasC) for Controllers
jenkins.yamlfile on each instance restart
If a Pipeline Template Catalog is configured in the CasC
jenkins.yamlfile and the
idproperty is not defined, the catalog is duplicated on each instance restart and in the exported CasC configuration.