Security fixes
- Upgraded multiple dependencies to address security concerns
-
Apache Ivy has been upgraded from 2.5.1 to 2.5.2 in the
pipeline-groovy-lib
plugin.
Guava has been upgraded from 31.0.1 to 32.1.2 and had j2objc-annotations:2.8 added in the google-storage-plugin
.
Guava has been upgraded from 31.1-jre to 32.1.2-jre in the google-oauth-plugin
.
Jetty IO has been upgraded from 10.0.13 to 10.0.16 where applicable.
Jetty XML has been upgraded from 10.0.13 to 10.0.16 where applicable.
commons-httpclient
library vulnerability-
The
commons-httpclient
library used in WebDAV and Azure stores (transitively throughcom.sun.jersey:jersey.client
) is vulnerable to man in the middle (MITM) attacks. The fix removes thecommons-httpclient
library and uses the native Java HttpClient.
Upgrade Notes
- Operations center CloudBees Assurance Program plugin changes since 2.414.2.2
-
The following plugins have been removed from the Operations center CloudBees Assurance Program since 2.414.2.2:
-
Maven Integration plugin (
maven-plugin
)
-
- Controller CloudBees Assurance Program plugin changes since 2.414.2.2
-
The following plugins have been added to the Controller CloudBees Assurance Program since 2.414.2.2:
-
GitLab API Plugin (
gitlab-api
) -
GitLab Branch Source Plugin (
gitlab-branch-source
) -
Jersey 2 API (
jersey2-api
) -
Pipeline Maven Integration Plugin (
pipeline-maven
) -
Pipeline Maven Plugin API (
pipeline-maven-api
)
-
The following plugins have been removed from the Controller CloudBees Assurance Program since 2.414.2.2:
-
Maven Integration plugin (
maven-plugin
)
New Features
- Add GitLab support in the CloudBees Assurance Program (CAP)
-
The GitLab Branch Source plugin and the GitLab API plugin were added to the CloudBees Assurance Program (CAP). These plugins enable users to have a supported way of creating multibranch and organization folder jobs backed by a GitLab server. These plugins will now receive necessary updates and security fixes under the CloudBees Assurance Program (CAP).
- Bundle Update Timing is now available
-
The new CloudBees CI feature Bundle Update Timing is now available. With this new feature, instances using Configuration as Code can be configured to:
-
Automatically reload a new bundle version (If the bundle can be hot reloaded, the current restrictions remain.).
-
Automatically schedule a safe restart after the new bundle is detected.
-
Skip any new version detected. It is also possible to skip the versions when the instance is up and running and later apply the bundle during a restart.
-
Skip a concrete bundle version so it is never applied.
-
Reject or accept bundle versions with warnings, leaving the acceptance threshold to administrator users (administrator users can configure the instance).
Bundle Update Timing is enabled by default. However, if desired, you can disable Bundle Update Timing by setting the system property
-Dcore.casc.bundle.update.timing.enabled=false
in the start script. -
For more information refer to Bundle update timing for the operations center or Bundle update timing for controllers.
- Adding and removing Maven-related plugins from CAP
-
The Maven Integration (
maven-plugin
) plugin was removed from the CloudBees Assurance Program (CAP). As an alternative to this plugin, CloudBees recommends the use of the Pipeline Maven Integration plugin (pipeline-maven
).
The Pipeline Maven plugin (pipeline-maven
) and Pipeline Maven API (pipeline-maven-api
) were added to CAP. See the upgrade notes for more details.
The CloudBees Maven Migration Assistant plugin (cloudbees-maven-migration-assistant
), a plugin to help you migrate your Maven jobs to Pipeline jobs, is available from the CloudBees Update Center.
Bulk migration Groovy scripts will soon be available to ease the migration of Maven jobs.
- Configuration as Code Operations center Bundle Retriever for SCM
-
The CasC Operations center Bundle Retriever is a service that ensures Operations center CasC bundles are always available and updated before the Operations center starts. The CasC Bundle Retriever is deployed alongside the Operations center container and allows you to manage it’s configuration via source control management (SCM) while ensuring that the latest bundle version is always locally available during both initialization and while running.
The Bundle Retriever uses a polling service to check the SCM by default. If GitHub is configured as the SCM then the Bundle Retriever can be configured to use webhooks for push notifications.
The Bundle Retriever can also be used in combination with Bundle Update Timing to optimize your Operations center CasC bundle management.
For more information refer to Retrieving bundles using SCM.
- Configuration as Code Bundle Report in the Operations center CasC Management Page
-
New ability to view a report of all of your controllers and their associated bundles. The report is now available on the Operations center CasC management page. The user can search for controllers as well as search and filter the bundles.
- CasC Items support for GitLab organization folders and multibranch projects
-
CasC Items plugin now supports GitLab organization folders and multibranch projects.
Feature Enhancements
- Display queue size in the High Availability (HA) summary
-
Now, the queue size on each cluster member is displayed in the High Availability (HA) summary.
- Display the load per replica summary in the High Availability (HA) diagnostic page
-
The High Availability (HA) diagnostic page now displays a list of replicas and some load information, such as running builds and online agents.
- Include GitHub-related header values in the hibernation monitor logs
-
The pod logs now report the values of selected headers, without personal information, that can be correlated with webhook delivery logs in GitHub.
- Windows agent container images for CloudBees CI
-
Agent container images for CloudBees CI based on Windows Server Core and Nano Server LTSC2019 and LTSC2022 are now available. These images must be used if you’re using the new High Availability mode for CloudBees CI. For more information, refer to High Availability mode.
- When running a controller in High Availability (HA) mode, provision a pod disruption budget
-
When running a controller in replicated mode, set up a PodDisruptionBudget to ensure that the service is always running, even in maintenance events (node replacements). Use a pod disruption budget to ensure at least one replica is always up when scaling down node pools.
- Add network policy support
-
Network policy support for the Hazelcast port has been added when a controller uses High Availability (HA) mode.
- Improve texts on the configuration page for controller items
-
Adapted the label and inline help texts for the Configuration as Code property on the configuration page of controller items.
- Omit
cluster-autoscaler.kubernetes.io/safe-to-evict=false
on controllers in High Availability (HA) mode -
When using HA mode, you can evict replicas when you scale down a node pool as it does not cause the loss of availability.
- Load-aware build adoption in High Availability (HA)
-
Previously, when a High Availability (HA) controller terminated, its builds were adopted at random by other replicas. Now, the adoption is preferentially directed toward replicas that currently have the least load according to a simple metric.
- Base64 support for secret masking
-
Base64 support for secret masking was added in this release. Now when users configuring jobs use any type of credentials, the Base64 version of their secret is added to the list of patterns that the application masks. This should reduce the number of leaks when the verbose mode of curl (or an equivalent mechanism) is used. However, there are limits to credentials masking. Refer to Limitations of Credentials Masking for more information.
Resolved Issues
- When CloudBees Pipeline Explorer was enabled, some pages were very slow for complex Pipelines, leading to controller-wide performance issues in some cases
-
When CloudBees Pipeline Explorer was enabled, some pages, such as Blue Ocean and the Pipeline Steps view, were very slow for complex Pipelines. This can lead to high CPU usage on the controller, causing controller-wide performance issues. CloudBees Pipeline Explorer plugin now avoids performance-intensive computation in APIs that are used to access all of the Pipeline steps.
- Older versions of controllers cannot start after receiving new bundle versions
-
All operations center supporting Branch and Tag for controller CasC bundles cannot send new bundle versions to the controllers older than 2.401.1.3. When those controllers receive a new bundle version, they cannot start.
Now, bundles sent to controller versions older than 2.401.1.3 are accepted.
- Some invalid fields were not found during Configuration as Code bundle validation
-
Now a warning message is generated if a nonexistent or invalid field is found during validation.
- Credentials cache not working when controller disconnected from operations center
-
When a controller disconnected from the operations center, yet the operations center was still running, attempts to look up credentials from the operations center would fail rather than falling back to a cached list as expected.
This issue is resolved.
- Cluster Ops with backup controller on operations center is broken
-
Cluster Ops backup on operations center is failing because certain classes were not found during the backup cluster operation.
Cluster Ops with backup controller now functions correctly, but it requires the CloudBees Backup plugin (plugin ID:
infradna-backup
) to be installed on the controller.
- Send a warning if the
managed-master-hibernation
plugin is installed on a High Availability (HA) controller -
Added an administrative monitor warning to administrators who run an High Availability (HA) controller that the
managed-master-hibernation
plugin should be uninstalled as it is incompatible with it.
- Custom Update Center signature check uses an outdated certificate
-
The Update Center certificate was updated recently, and that modification needs to be taken care of for custom Update Center signature checks. The Custom Update Center signature now uses the updated Update Center certificate.
- New health check endpoint
-
Previously, the health check endpoint for the operations center pods used the
/login
URL, that creates a new Jetty session each time. Now, a new endpoint has been selected that uses an anonymous REST API that avoids session creation. (The analogous change was made for managed controllers in a previous release.)
- Updated expired embedded certificate of the Update Center
-
An expired certificate was previously embedded in the product, causing the download of some tool installers and plugins to fail. The certificate has been updated.
- Operations center startup blocked by Update Center item loading
-
The loading of Update Center items is slowed down by the sha256 and sha512 calculation of the stored plugin and core binaries. This can delay the startup of the operations center for several minutes for Update Center items with a large plugins base.
Checksums are now saved during download and manual uploads, and during loading only if the checksums file does not exist.
- Managed controller YAML validation section remains empty
-
Since
prototype.js
usage was removed, the YAML validation section remains empty when creating managed controllers.The YAML validation section now properly contains the necessary data.
- External groups not propagated from the Reverse Proxy Auth plugin used with the operations center SSO
-
When the operations center SSO is used and the operations center is configured with an SSO-type security realm that only defines a user’s external groups within the context of a browser request, a connected controller is authenticated with the correct user ID, but none of the groups. This issue has been resolved.
- Rename the CasC browser label
gitLab
togitLabBrowser
-
The CasC browser label
gitLab
is used by both thegit-plugin
andgitlab-branch-source
plugins. The label used bygitlab-branch-source
has now been changed togitLabBrowser
.
- Optimized handling of job next build number in HA controllers
-
Synchronizing the next build number of jobs can be a bottleneck in HA controllers on slow network filesystems. This process is now improved and the issue has been resolved.
- More reliable cross-replica build loading for High Availability (HA) controllers
-
Previously, High Availability (HA) controller replicas could partially load build records even when the builds were in progress on other controllers, leading to errors. Now, a replica only loads a build record when the build is complete or when it is adopting a build in progress.
- Fix a logging thread contention when the client controller calls the Operations center.
-
In some cases, logging call errors to the Operations center could cause thread contention. Rate limiting has now been applied to these use cases to prevent logging from causing thread contention.
- UncheckedIOException related to bundle update can stop startup
-
File corruption in the bundle log directory can stop a controller startup, instead of a smooth recovery. This issue has been resolved.
- Excessive HTTP client threads created
-
Certain operations, such as contacting the operations center for user lookup, could create excessive Java threads. This issue has now been resolved.
- Retain queue ID when adopting queue items in High Availability (HA)
-
If you schedule a build using a High Availability (HA) controller, but it does not start before the replica shuts down, the corresponding adopted queue item in another replica now uses the same ID. This allows external tools to track the build when it does start (or the item is cancelled).
- Incompatible HA agent administrative monitor is displayed even if empty
-
In some cases, the administrative monitor that displays incompatible High Availability (HA) agents can be displayed, even when an incompatible agent is not found. This issue has been resolved and the administrative monitor is no longer displayed.
- Log recorders synchronization does not work for creation and deletion
-
The log recorders are now correctly synchronized between replicas of a controller in High Availability (HA) mode.
- CasC operations center RBAC "Granted at" offset is not honored when a group is created or updated
-
"Granted at" is now properly honored.
Known Issues
- Failed parsing of data in the User Activity Monitoring plugin leads to incomplete data
-
Failed parsing of data from the User Activity Monitoring plugin will overwrite the user activity database. All user activity data that is logged up to that point in time is lost, in order to avoid this, refer to this knowledge base article Why is my user activity missing?.
- HTTP Client used for Operations Center to Controllers connection leads to performance issues
-
Because of known issues in the Java HTTP Client, there could be performance issues in Operations Center to Controllers interactions in heavily loaded environments.
More details about this issue and workarounds are documented in Operations Center Client leaks HTTP Clients since version 2.401.1.3.
- YAML file is not validated in the Default YAML field of the Configure Controller Provisioning page
-
On the Configure Controller Provisioning page, the YAML file is not validated when it is configured in the Default YAML field.
- Auto-update bundle version may incorrectly update the wrong bundle version if multiple
bundle.yaml
files exist in the repository -
When using "Auto-update bundle version", the wrong bundle may be updated if multiple
bundle.yaml
files exist in the repository.
- Wrapped token file can not be deleted by the operations center
-
The
.wrappedToken
file is inadvertently being included in the bundle by the operations center bundle retriever and giving a structural warning,[STRUCTURE] - The following files have been detected in the bundle folder but they are not yaml files: .wrappedToken.
- Add CloudBees CI communications to the CasC Bundle Retriever
-
When a bundle is retrieved from the SCM there are benign warnings in the logs about communication with the operations center.
- Error when renaming an existing EC2 cloud
-
When the name of an existing cloud node is updated, the user receives a 404 error after selecting save. This is because the cloud page uses the cloud name as part of its URL. When the user saves the name, Jenkins sends the user to the URL with the old cloud name. Please note that all node changes are saved successfully.
- Duplicate Pipeline Template Catalogs in the Configuration as Code (CasC) for Controllers
jenkins.yaml
file on each instance restart -
If a Pipeline Template Catalog is configured in the CasC
jenkins.yaml
file and theid
property is not defined, the catalog is duplicated on each instance restart and in the exported CasC configuration.