Critical issues

Critical regression fixes in version 2.249.3.2

Version 2.249.3.2 release fixes some critical regressions found in version 2.249.3.1.

CloudBees CI on modern cloud platforms, CloudBees CI on traditional platforms Installing version 2.249.3.2 is required to avoid these issues.

CloudBees Jenkins Enterprise CloudBees Jenkins Enterprise 1.x customers need to download and install using the 1.11.39 installer, which will bundle the corresponding operations center/Managed Master versions.

CloudBees Jenkins Platform CloudBees Jenkins Platform operations center and CloudBees Jenkins Platform Client Master (2.x.y.z) must be upgraded to version 2.249.3.2. CloudBees Jenkins Platform operations center and CloudBees Jenkins Platform Client Master (2.222.x.0.z) must be upgraded to version 2.222.42.0.1-rev6. CloudBees Jenkins Platform operations center and CloudBees Jenkins Platform Client Master (2.190.x.0.z) must be upgraded to version 2.190.33.0.2-rev6.

CloudBees Jenkins Distribution No updates necessary.

New features

Add support level on Plugin Manager table alongside the plugin name in the Jenkins LTS Update Manager UI (FNDJEN-3095)

To help you to quickly identify the support tier for plugins, CloudBees now adds badges to the Plugin Manager to identify the plugins that are proprietary, verified, and compatible.

Feature enhancements

Create/Update folders using CasC (CTR-2308)

There was no reliable way to define items (for example, folders) in CloudBees CasC configuration bundles.

A new file, named items.yaml, is now part of CloudBees CasC configuration bundles and it contains the definition of CloudBees CI items, limited to folders for now.

See Creating folders with CasC for more information.

RBAC groups defined in items.yaml folders (CTR-2546)

Role-Based Access Control (RBAC) groups now can be defined in items.yaml folders.

Previously created RBAC groups in folders items managed by items.yaml are deleted.

See items.yaml for more information.

Plugin to add to configuration bundle and parse the items.yaml file (CTR-2307)

Users must install the CloudBees CasC API Plugin to enable the new items.yaml file.

RBAC management with CasC - Roles (CTR-2416, CTR-2417)

A new file, named rbac.yaml, is now part of the CloudBees CasC configuration bundles and it contains the definition of Role-Based Access Control (RBAC) global roles and groups.

Global roles and groups not defined in the rbac.yaml are deleted.

See Configuring RBAC with CasC for more information.

Ability to configure RBAC as an Authorization Strategy using CasC (CTR-2633)

CloudBees Role-Based Access Control (RBAC) authorization strategy was not compatible with Configuration as Code (CasC).

With this update, CloudBees RBAC can be set as the authorization strategy with CasC, by adding the following configuration to the jenkins.yaml file:

jenkins: authorizationStrategy: "cloudBeesRoleBasedAccessControl"

See Configuring RBAC with CasC for more information.

Ability to export folder items into items.yaml file from the existing CasC export page (CTR-2544)

A new items.yaml file containing folders can be exported as part of the configuration bundle in the Configuration as Code management page.

See Exporting a CasC bundle for more information.

Ability to export RBAC global groups and roles into rbac.yaml file from the existing CasC export page (CTR-2545)

A new rbac.yaml file containing global roles and groups can be exported as part of the configuration bundle in the Configuration as Code management page.

See Exporting a CasC bundle for more information.

CasC bundle reload from the CasC bundle page (CTR-2526)

Before adding this feature, only Overall/Administer users were able to apply a new version of the configuration bundle.

Now Overall/Manage users are also able to apply the configuration changes from the CloudBees Configuration as Code bundle management page.

See Updating a CasC bundle for more information.

Improve UX for installing the CloudBees CasC API Plugin (CTR-2614)

The CloudBees CasC API Plugin uses some new APIs from cloudbees-installation-manager, which could lead to java.lang.NoSuchMethodError exceptions during start up.

This improvement adds a version check to avoid the java.lang.NoSuchMethodError exceptions during start up. If cloudbees-installation-manager version is older than required, then the creation of items and RBAC groups/roles is skipped and a SEVERE log message is printed.

Individual masters can now modify opt-out option (CTR-2542)

Prior to this change, users could only configure masters to opt out of having security managed by operations center. With this update, users can configure masters to have authentication and authorization or just authentication managed by operations center by default, but also have select masters manage security independent of operations center or if they are using Configuration as Code, manage the role-based access control (RBAC) groups and roles as code for given masters.

See Configuring options for individual masters for more information.

CloudBees Fast Archiving Plugin Jenkins version update (CTR-2568)

With this update, the minimum Jenkins version is 2.249.2-cb-1 for the CloudBees Fast Archiving Plugin.

Make 5.42 integration of the CloudBees Role-Based Access Control Plugin with folders optional (CTR-2556)

An update to version 5.42 of the CloudBees Role-Based Access Control Plugin (nectar-rbac) caused the plugin to fail unless the user had installed the Folders plugin.

With this update, the CloudBees Role-Based Access Control Plugin will work even if the user has not installed the Folders plugin.

Preparation for upcoming API changes in Jenkins (CTR-2378)

Minor log message updates that do not affect functionality.

Manage Pipeline policies with Configuration as Code (CasC) (NGPIPELINE-1439)

CasC is now supported with Pipeline policies. This is a Preview feature.

Resolved issues

Allow CasC to export current value of anonymous and authenticated RBAC roles (CTR-2668)

With this fix, authenticated and anonymous RBAC roles are exported as part of the rbac.yaml file export.

CasC Groups creation does not work when authorization is pushed from operations center (CTR-2634)

Groups are created by Configuration as Code before CloudBees CI has fully started; however, CloudBees Role-Based Access Control (RBAC) plugin code includes logic that assumes CloudBees CI is fully started, which lead to a runtime exception.

With this fix, the CloudBees RBAC plugin uses a safer API to retrieve the RBAC configuration singleton, which works even when CloudBees CI has not fully started. Also, the new call is more efficient than the previous one, improving performance.

Allow CasC configuration to change permissions of authenticated and anonymous RBAC roles (CTR-2666)

With this update, authenticated and anonymous RBAC roles can be configured in the rbac.yaml file.

operations center /tmp should be mounted from an emptyDir volume (CPLT2-6735)

The /tmp directory in the operations center container was using the union filesystem, which was providing sub-optimal performance.

The operations center /tmp folder is now mounted from an emptyDir volume, which provides better performance.

Compatibility fixes for Kubernetes 1.18+ (CPLT2-6774)

Kubernetes 1.18 requires the use of valid Kubernetes signers to allow Certificate Signing Requests (CSR) to go through. Sidecar injector needs a keypair in order to secure its access using TLS and couldn’t provision its certificate anymore.

Sidecar injector now uses a self-signed certificate whose public key is injected in the Mutating Webhook configuration object so that the API server can contact it securely. It no longer uses CSRs.

Remove usage of extensions.Ingress (CPLT2-6825)

Kubernetes no longer serves Ingress resources from extensions/v1beta1 in v1.19 so migration to the networking.k8s.io/v1beta1 API, which has been available since v1.14, was required.

The Master Provisioning plugin now uses networking.k8s.io/v1beta1 apiVersion to provision Ingress by default. With this change, this plugin can only be used with Kubernetes versions 1.14 and later.

PodTemplateUtils#combine has a side-effect on the parent node properties (CPLT2-6814)

Many occurrences of node properties were observed on some pod templates with no rational explanation. This was causing memory issues because of the high number of elements.

There was a faulty side-effect when pod templates were combined to implement inheritance, which was causing node properties to be added to the parent pod template during that operation. This side-effect was removed.

Managed Master "/tmp" should be mounted from an emptyDir volume (CPLT2-6736)

The Managed Master: /tmp is now mounted from an emptyDir volume to avoid using the overlay filesystem.

Too much indent on Values.Persistence.mounts (CPLT2-6806)

Using Values.Persistence.mounts causes a Helm error with an underlying root cause related to indentation.

Values.Persistence.mounts and Values.Persistence.volumes were removed from the Helm chart because they were not usable and were duplicated by OperationsCenter.ExtraVolumeMounts and OperationsCenter.ExtraVolumes.

Prevent AnonymousClassWarnings when serializing the ToolInstaller (CTR-2552)

Prior to this change, when the CloudFoundry CLI installer was sent to an agent, users received messages about AnonymousClassWarnings. To prevent these messages, the class no longer uses an Anonymous inner class.

Console log flooded with ClassNotFoundException: org.eclipse.jetty.server.LowResourceMonitor (FNDJEN-3108)

The cloudbees-support plugin was flodding the log with "Error getting information from Jetty server. We cannot retrieve the low resource monitor."

This issue has been fixed.

HttpWorkersCounterPeriodicWork is giving false positives with Jetty (FNDJEN-3041)

Since 2.222.x, HttpWorkersCounterPeriodicWork is giving false positives in Jetty Winstone because it is incapable of counting Jetty Winstone threads properly.

False positives are now prevented when checking for Jetty http threads.

Proxy reconfiguration for CasC installation (FNDJEN-3187)

When using plugin-catalog to install plugins on one controller, the proxy configuration is used to access external resources.

Known issues

Using image:tag jenkins/inbound-agent:windowsservercore-1809 can result in an image that will not start due to a password expiration

To fix this issue, specify the version in the image tag. The current version is jenkins/inbound-agent:4.3-9-windowsservercore-1809.

For details about running agents in Windows containers, see Managing agents - Running CloudBees CI build agents in Windows containers

For details about this issue, see the JenkinsCI issue.

Items, including but not limited to folders, shared clouds, shared agents, and shared configurations, defined within a folder do not have their RBAC configuration correctly migrated with version 2.249.3.1 (CTR-2740,CTR-2742)

A fix for this critical regression is included with version 2.249.3.2.

Version 4.0 or higher of .NET Framework is required to launch controller or agents on Windows services

Starting from this release, .NET Framework 2.0 doesn’t work for launching CloudBees controller or agents as Windows services. Microsoft.NET Framework 4.0 or above is now required for using the default service management features.

This release also upgrades Windows Service Wrapper (WinSW) from 2.3.0 to 2.9.0 and replaces the bundled binary from .NET Framework 2.0 to 4.0. There are many improvements and fixes in these versions, big thanks to NextTurn and all other contributors. You can find the full WinSW changelog here, just a few highlights important to CloudBees users:

  • Prompt for permission elevation when administrative access is required. Now CloudBees users do not need to run the agent process as Administrator to install the agent as a service from GUI.

  • Enable TLS 1.1/1.2 in .NET Framework 4.0 packages on Windows 7 and Windows Server 2008 R2.

  • Enable strong cryptography when running .NET Framework 4.0 binaries on .NET 4.6.

  • Support security descriptor string in the Windows service definition.

  • Support 'If-Modified-Since' and proxy settings for automatic downloads.

  • Fix Runaway Process Killer extension so that it does not kill wrong processes with the same PID on startup.

  • Fix the default domain name in the serviceaccount parameter (JENKINS-12660)

  • Fix archiving of old logs in the roll-by-size-time mode.

Use-cases affected by .NET Framework 2.0 support removal

If you use .NET Framework 2.0 to run the CloudBees Windows services, the following use cases are likely to be affected:

  • Installing the CloudBees controller as a Windows service from Web UI. The official MSI Installer supports .NET Framework 2.0 for the moment, but it will be changed in future versions.

  • Installing agents as Windows services from GUI. This feature is provided by the Windows Agent Installer Module from the Jenkins core.

  • Installing agents over Windows Management Instrumentation (WMI) via the WMI Windows Agents plugin

  • Auto-updating of Windows service wrappers on agents installed from GUI.

Upgrade guidelines

If all of your CloudBees controller and agent instances already use .NET Framework 4.0 or above, there are no special upgrade steps required.

If you run the CloudBees controller as a Windows Service with .NET Framework 2.0, this instance will require an upgrade of .NET Framework to version 4.0 or above. .NET Framework 4.6.1 or above is recommended because this .NET version provides many platform features by default (e.g. TLS 1.2 encryption and strong cryptography), and Windows Service Wrapper does not have to apply custom workarounds.

If you want to continue running some of your agents with .NET Framework 2.0, the following extra upgrade steps are required:

  1. Disable auto-upgrade of Windows Service Wrapper on agents by setting the -Dorg.jenkinsci.modules.windows_slave_installer.disableAutoUpdate=true flag on the CloudBees controller side.

  2. Upgrade agents with .NET Framework 4.0+ by downloading the recent Windows Service Wrapper 2.x version from WinSW GitHub Releases and manually replacing the wrapper ".exe" files in the agent workspaces.

Upgrade notes

If upgrading from a rolling release older than 2.387.1.2, customers may experience technical difficulties. CloudBees ensures compatibility only between supported versions of the product and recommends upgrading early and often to avoid these difficulties. If you are having difficulties upgrading, contact CloudBees Support for assistance.

CloudBees Role-Based Access Control Plugin

With this upgrade, for security reasons, we are disabling the ability to configure RBAC groups and role filters at the views level.

See CloudBees Role-Based Access Control Plugin 5.42 for more information about the security vulnerability.

This change means that any previous groups or role filters created in a view will not be applied and you will not be able to configure them.

This update only affects the views themselves, not the items within them. Previous permissions applied to the items are still enforced.

If you were filtering roles on views before this upgrade, these filters will no longer work, so your users may have a more permissive permission scheme on the views.

CloudBees recommends running this script in your script console to determine if you have a configuration on your instance that will be affected by this change.

If you do have a configuration that will be affected by this change, you have two options:

  1. (CloudBees recommended approach) Recreate each view inside a folder and apply the RBAC configuration to the folder. The folder RBAC configuration is propagated to the view since it is inside the folder.

  2. Enable RBAC configuration on views by setting the system property nectar.plugins.rbac.groups.ViewProxyGroupContainer=true.

    This approach is not recommended for security reasons.
End-of-life announcement

After assessing the viability of our supported plugins, CloudBees no longer supports the Visual Studio Team Service Plugin as of September 9, 2020.

This end-of-life announcement allows CloudBees to focus on driving new technology and product innovation as well as maintaining existing products that are actively used by customers.

After September 9, 2020, the plugin will lose functionality when upgraded. Microsoft recommends Creating a service hook for Azure DevOps instead.

Users should uninstall the plugin to avoid a credentials ID enumeration security bug.

For more information regarding this end-of-life announcement, please contact your Customer Success Manager.

Revisions

Revision 2 (2020-11-04)

Fix some vulnerabilities in Active Directory Plugin. Refer to CloudBees Security Advisory 2020-11-04 for details.