Controller slowness and BLOCKED threads on UrlFolderIconContributor after upgrade

Issue

After upgrading CloudBees CI to versions between 2.541.2.35785 and 2.541.3.36069 (inclusive), a controller with a large number of folders and Role-Based Access Control (RBAC) groups becomes slow or unresponsive.

Thread dumps or slow-request traces show a large number of HTTP worker threads with a call stack similar to the following:

"Handling GET /jenkins/ ... Jetty (winstone)-N" ...
    ...
    at nectar.plugins.rbac.groups.GroupContainerACL.hasPermission2(GroupContainerACL.java:158)
    at hudson.security.ACL.hasPermission(ACL.java:137)
    at hudson.security.AccessControlled.hasPermission(AccessControlled.java:72)
    at com.cloudbees.hudson.plugins.folder.AbstractFolder.getItems(AbstractFolder.java:912)
    at hudson.model.Items$AllItemsIterable$AllItemsIterator.hasNext(Items.java:691)
    at java.lang.Iterable.forEach(java.base@21.0.10/Iterable.java:74)
    at com.cloudbees.hudson.plugins.folder.icons.UrlFolderIcon$UrlFolderIconContributor.apply(UrlFolderIcon.java:53)
    at java.lang.Iterable.forEach(java.base@21.0.10/Iterable.java:75)
    at jenkins.security.csp.CspBuilder.withDefaultContributions(CspBuilder.java:69)
    at jenkins.security.csp.impl.CspDecorator.getContentSecurityPolicyHeaderValue(CspDecorator.java:55)
    at jenkins.security.csp.impl.CspFilter.doFilter(CspFilter.java)
    ...▼

The number of threads grows with the number of HTTP requests, making the controller unresponsive under normal load.

Environment

CloudBees CI on modern cloud platforms - managed controller >= 2.541.2.35785 and < 2.555.1.36485
CloudBees CI on traditional platforms - client controller >= 2.541.2.35785 and < 2.555.1.36485
CloudBees Folders Plus plugin
controllers with a large number of folders (typically more than a few thousand)

Explanation

Jenkins 2.541.1 introduced a Content-Security-Policy-Report-Only response header. To generate this header, CloudBees CI invokes registered CSP contributors on every HTTP request, including UrlFolderIconContributor from the CloudBees Folders Plus plugin.

UrlFolderIconContributor of the affected versions iterates over all folders on the controller and performs a per-user RBAC permission check for each one. On controllers with large folder hierarchies and active RBAC groups, this per-request iteration causes HTTP worker threads to contend on the RBAC lock, resulting in a very slow request processing.

Resolution

This issue is resolved in CloudBees CI version 2.555.1.36485.

Workaround

Apply one of the following workarounds. Workaround 1 is preferred because it resolves the root cause without a security impact.

Workaround 1: Exclude UrlFolderIconContributor using the Extension Filter plugin

Install the Extension Filter plugin on the controller.
Navigate to Manage Jenkins Configure System.

In the Extension Filter section, add an exclusion for the following class:

com.cloudbees.hudson.plugins.folder.icons.UrlFolderIcon$UrlFolderIconContributor▼

Select Save.

After applying this workaround, custom URL-based folder icons no longer render and fall back to the default folder icon.

Workaround 2: Disable CSP header computation

Add the following JVM argument to the controller startup options:

-Djenkins.security.csp.CspHeader.headerName=▼

Disabling the CSP header is a security regression. Remove this JVM argument as soon as a permanent fix is applied.

References

This article is part of our Knowledge Base and is provided for guidance-based purposes only. The solutions or workarounds described here are not officially supported by CloudBees and may not be applicable in all environments. Use at your own discretion, and test changes in a safe environment before applying them to production systems.