Upgrade Notes
- Administrative monitor for Tomcat end of support in October 2025
-
If you run CloudBees CI in Tomcat, it has been deprecated and CloudBees CI’s support will end after October 2025. An Administrative Monitor has been added to advise customers how to migrate from Tomcat to a CloudBees Linux package. For more information about the migration, refer to Migrate CloudBees CI on traditional platforms from Tomcat to a CloudBees Linux package.
- Operations center CloudBees Assurance Program plugin changes since 2.452.3.2
-
The following plugins have been added to the Operations center CloudBees Assurance Program since 2.452.3.2:
-
CloudBees CI Catalog Plugin (
cloudbees-ci-catalog
)
-
- Controller CloudBees Assurance Program plugin changes since 2.452.3.2
-
The following plugins have been added to the Controller CloudBees Assurance Program since 2.452.3.2:
-
CloudBees CI Catalog Plugin (
cloudbees-ci-catalog
)
-
Feature Enhancements
- In the CloudBees Pipeline Explorer, step badges displayed in the log view when Show steps option and related filters are active in tree view
-
The step badges are displayed in the log view when the Show steps option and related filters are active in the tree view. The related builds (
build
andtriggerRemoteJob
) and input steps are always displayed in the tree view, but the Show steps option needs to be activated to be displayed in the log view. The nodes are never displayed as badges in the log view.
- Configuration as Code Addition of Dismiss button to disable specific Administrative monitors
-
A number of Configuration as Code Administrative monitors were checked and endowed with a Dismiss button that disables the monitor. Disabled monitors can subsequently be enabled via Dashboard > Manage Jenkins > Administration Monitors > Enable | Disable.
- Clarifying display name of the High Availability (HA)-specific agent retention strategy
-
High Availability (HA) controllers offer a special permanent agent retention strategy. This only applies to outbound agents (for example, SSH) and not inbound (TCP/WebSocket) agents. The display label and associated wording has been improved.
- Change in the admin monitor references to pipeline durability
-
The pipeline durability admin monitor display has been updated to
Maximum survivability/durability
instead ofMAX_SURVIVABILITY
.
- Use of waitForQualityGate step on High Availability (HA) controller
-
The
waitForQualityGate
Pipeline step from the SonarQube Scanner plugin on an High Availability (HA) controller was unreliable, as the webhook could be delivered to a different replica than the one running the build and was lost, causing the build to hang. Now, any webhooks for this plugin are mirrored to all replicas. This allows the step to complete, regardless of whether or not the webhook happened to be processed on the same replica.
- Use of waitForWebhook step on High Availability (HA) controller
-
The
waitForWebhook
Pipeline step from the Webhook Step plugin on an High Availability (HA) controller was unreliable, as the webhook could be delivered to a different replica than the one running the build and was lost, causing the build to hang. Now, any webhooks for this plugin are mirrored to all replicas and allows the step to complete, regardless of whether or not the webhook was processed on the same replica.
- Collect statistics about High Availability (HA) synchronization in support bundles
-
The support bundles for a High Availability (HA) controller already included statistics about the size of the distributed Hazelcast collections and similar metadata. Now, they will also list the number of synchronization events (such as for completed builds or settings changes) sent or received by the replica that renders the bundle, and are broken out by message type. This can be used to diagnose performance issues caused by certain types of excessive messages.
- Improvements made to the administrative monitor for migration of libraries for High Availability (HA)
-
The improvements made to the administrative monitor include:
-
Add support for untrusted global libraries to the administrative monitor for High Availability (HA).
-
Allow users with
Overall/Manage
permission to migrate untrusted global libraries and folder libraries.
-
- CloudBees CI Plugin Usage plugin only analyzed usage of Pipeline steps in builds that were already loaded
-
Previously, the CloudBees CI Plugin Usage plugin only analyzed the usage of Pipeline steps in builds that were already loaded. If the build was not fully loaded; for example, if the controller restarted after it completed, the Pipeline step usage was not analyzed and was not part of the plugin usage report. The CloudBees CI Plugin Usage plugin now loads the Pipeline steps in builds and analyzes them even if they are not already loaded.
Resolved Issues
- Fixing
NullPointerException
when storage class name is validated -
Validating the storage class in the global configuration screen leads to a
NullPointerException
.The field has been moved inside the Endpoint screen to better support multiple endpoint scenarios as well as to fix the
NullPointerException
.
- Some RBAC permissions would not load when using the FINE logger
-
When you configure a FINE logger for RBAC and restart a controller, a silent
NullPointerException
would block the proper loading of RBAC groups permissions.
- Fix sharing of
VaultAWSCredentials
between the operations center, controllers, and agents -
The SSH Credentials defined in the HashiCorp Vault store of the Operations center were not usable in controllers and agents.
Now, the SSH Credentials defined in the Operations center are shared and used in controllers and agents.
- CloudBees HashiCorp Vault GitHub App Credentials cannot be used in an organization item
-
When using a CloudBees HashiCorp Vault GitHub App Credentials in an organization item the organization scan would fail.
- CloudBees HashiCorp Vault GitHub App Credentials cannot be used through remoting
-
The CloudBees HashiCorp Vault GitHub App credentials retrieval would fail from remote agents and generate a
Failed to deserialize the Callable object
error message.
- Folder Configuration as Code (CasC) export is showing CloudBees HashiCorp Vault SSH private keys in plain text
-
In the exported YAML file, the key
privateKeys
was showing the SSH private keys in plain text for CloudBees HashiCorp Vault SSH credentials. This key was removed from the export.
- Operations center SSO Login fails with a
404
error -
When using the operations center SSO, a login from a controller could have been failing with a
404
error on/cjoc/operations-center-id/
. This occurred after a failed logout.
- Improve user experience for error handling in license manager
-
Previously, the full error stack trace used to be shown to the user and an invalid license was saved into the product.
The issue is now fixed by showing a user-friendly message in the UI and by logging the full stack trace in the backend for analysis. Also, before persisting the license data to disk, it is now validated and does not allow invalid licenses to be saved. In addition, the form validation was improved so that the user is immediately aware when there is a problem with the license.
- In the CloudBees Pipeline Explorer, contextual menus close unexpectedly when moving the mouse in the map display
-
When opening CloudBees Pipeline Explorer contextual menus, they unexpectedly closed when the mouse was moved in the map display.
- In the CloudBees Pipeline Explorer, the map node menu flickers with the map and log display
-
In the CloudBees Pipeline Explorer when the map and log were displayed at the same time, the menu of a node closed to the bottom of the map leading to a slight display jump.
- In the CloudBees Pipeline Explorer, the "Scroll to line" feature is broken in unsuccessful steps view
-
CloudBees Pipeline Explorer 1.17 broke the behavior of the “scroll to line” feature in the CloudBees Pipeline Explorer. It originally cleared the active filter and scrolled down to the last line of the step that produced that error.
- CloudBees Pipeline Explorer poor performance for builds with a large number of recorded test failures
-
Builds that had a large number of test failures recorded by the JUnit plugin performed poorly in the CloudBees Pipeline Explorer. The initial page load times were slow, and this can cause the web browser tab to crash. Performance has been improved by loading the test failures only when the test insights drawer is open, and truncating the failed tests shown in the CloudBees Pipeline Explorer to only the first 1000 failed tests. To see all of the failed tests, you can access the "Test Result" page that is linked from the CloudBees Pipeline Explorer test insights drawer.
- The CloudBees Pipeline Explorer has a performance issue with a very large number of steps
-
The CloudBees Pipeline Explorer has some performance issues when the tree displays a very large number of steps. The performance has been improved for the tree and the map.
- High Availability (HA) controller should exit if the Hazelcast cluster is broken
-
Under some circumstances, such as an
OutOfMemoryError
caught when processing a message, the Hazelcast system used by an High Availability (HA) controller could shut itself down, but the controller remained running, maybe indefinitely depending on other liveness criteria. Now, when Hazelcast exits unexpectedly, the controller also exits to allow a fresh replica to take its place.
- Inbound agents do not transfer among High Availability (HA) controller replicas without a label
-
A High Availability (HA) controller will detect when a replica requests an inbound permanent agent that is currently attached to another replica but is idle, and asks the current owner to disconnect so that it can reconnect to the replica that now needs it. However, this logic presumed that the request was via a label expression, and silently did nothing when a build was waiting for an available agent. Now, the absence of a label is handled uniformly. Any idle inbound permanent agent in the cluster will be considered eligible for transfer.
- Redundant Hazelcast events when saving proxy configuration
-
When saving the system outbound proxy configuration in an High Availability (HA) controller, a replica that receives the modification event does not load the modified configuration or re-fire the same event. These actions waste resources by forcing other replicas to verify that the configuration had not changed again. Now, recipients of the event just load the new configuration, as it already did for other global configurations.
- Crumb errors and repeated build log output in High Availability (HA) controllers
-
When the Atlassian Bitbucket Server Integration plugin was installed on an High Availability (HA) controller, several functions broke for builds that run on a different replica than the one with the sticky session; input steps gave a crumb error when they were submitted, the classic console log would repeat blocks of text, and so on.
- Blocking call to Hazelcast while holding queue lock
-
The code that manages inbound permanent agents in an High Availability (HA) controller might block a call that is waiting for a response from Hazelcast while holding the lock on the build queue.
This call is now made asynchronously.
- Possible infinite loop in ReplicasComponent
-
There was a possible infinite loop in the support component that collects data from replicas when it generates a support bundle.
- Failure to resume build using High Availability (HA) multi-executor agents in parallel
-
When a single Pipeline build on an High Availability (HA) controller was using multiple executors of an agent simultaneously, it could abort after a failover with a message that states that one of the clone agents failed to reattach to the controller when in fact it did. This was due to a confusion between the node self-labels and extra labels that is now corrected.
- Boot failures caused the controller pod to hang
-
If the Jenkins startup process was terminated by a boot failure (a fatal error, such as incorrect CasC bundle contents) in a managed controller, the pod would not become ready or attempt to restart the container; it would just hang until manually deleted, even if the underlying issue was corrected. In the case of an High Availability (HA) controller, this could mean that a rolling restart would not make any progress. Now, the container will promptly exit in this case, and the pod will be shown as a
CrashLoopBackOff
status and periodically try again to start the controller process. If the condition that caused the error is corrected, the next container restart should allow the pod to become ready, and the controller restart will complete.
- Builds that run in other replicas must be aborted before job is deleted
-
When you delete a project (or a folder that contains projects, maybe in subfolders) the controller first attempts to abort all of the running builds of the project(s) to avoid data corruption. This did not work in High Availability (HA) controllers when some builds that were running on other replicas than the one with the sticky session, or otherwise processing the deletion event (for example, due to branch indexing). Now, the builds on all of the replicas are aborted before deletion can proceed.
- NullPointerException occurred when checking if a rolling restart is in progress
-
A
NullPointerException
occurred when attempting to check if a rolling restart was in progress when a deployment was not yet ready for inspection. This issue typically occurred during the initialization phase of the deployment.
- Permission check updated for Move/Copy/Promote loading from URL
-
When a user does not have permission to perform any Move/Copy/Promote actions, the option should not be visible anywhere on the dashboard if the user is trying to navigate directly from the URL.
A check was added to the main jelly file level before it loads the page.
- Resolve "detected dubious ownership in repository" error when using an AWS EFS volume
-
When running a controller on an AWS EFS volume, and cloning a Git repository on the controller (for example, when using a heavyweight checkout), then based on how EFS sets up filesystem ownership, the Git clone can fail with a fatal error
detected dubious ownership in repository
.
This condition is now detected and automatically sets up Git appropriately to avoid reporting this error.
- Graceful recovery from incompatible changes to objects serialized in session during rolling upgrade
-
When there is a rolling upgrade, if a serialized object cannot be deserialized due to incompatible class changes, this object is now dropped instead of breaking the controller startup.
- X-Jenkins-Replica-Host & X-Jenkins-Replica-Address headers should follow developer mode
-
Previously, the X-Jenkins-Replica-Host & X-Jenkins-Replica-Address headers were added to all HTTP responses in a High Availability (HA) controller. Since these are diagnostic tools and internal host names or IP addresses can be considered sensitive in some environments, these are now included only when developer mode is enabled, and only for requests authenticated as a user with Overall/SystemRead permission, just like the corresponding notations in the GUI.
Known Issues
- Duplicate Pipeline Template Catalogs in the Configuration as Code for controllers
jenkins.yaml
file on each instance restart -
If a Pipeline Template Catalog is configured in the Configuration as Code
jenkins.yaml
file and theid
property is not defined, the catalog is duplicated on each instance restart and in the exported Configuration as Code configuration.
- Pod templates page is read-only for
NonConfigurableKubernetesCloud
-
The
NonConfigurableKubernetesCloud
setting on the pod template page appeared to be editable. However, it is read-only.
- Duplicate Plugins in Operations center Plugin Manager UI
-
When searching for a specific plugin under 'Available' tab in operations center Plugin Manager, the search results shows duplicate entries of the searched plugin.