Items and Global Configurations cannot be saved between version 2.277.3 and 2.289.2 (excluded)

Article ID:4402576980891
2 minute readKnowledge base

Issue

  • TLS termination for Jenkins is setup in Jetty / Winstone

  • Running 2.277.3 / After upgrading CloudBees CI to version 2.277.3 or later:

    • Many items and global configurations cannot be saved from the UI or the REST API

    • Plugin HPI and Files cannot be uploaded

    • External service fail to POST payload to Jenkins endpoints

  • In Chrome, attempts to save configuration in Jenkins results in ERR_CONNECTION_ABORTED, ERR_CONNECTION_RESET, ERR_EMPTY_RESPONSE error pages

  • When enabling FINE logs for org.eclipse.jetty, the following exception can be seen while reproducing the problem:

      javax.net.ssl.SSLHandshakeException: Encrypted buffer max length exceeded
    [...]
    at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540)
    at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395)
    at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
    [...]

Environment

Explanation

This issue happens in Jenkins version 2.277.3 to 2.289.2 (excluded) when Jetty / Winstone (the embedded servlet container for Jenkins) is set up to terminate TLS (commonly using --httpsKeyStore or --httpsCertificate to provide a certificate, see Configuring HTTP).

It is caused by a known issue Jetty Issue #6082 in the Jetty 9.4.39.v20210325 initially introduced while fixing a security vulnerability. When Jetty receives POST requests (when a user saves a configuration in Jenkins), various issues related to compaction and calculation of buffer length now may cause Jetty to abort connection unexpectedly.

The issue is fixed in Jetty 9.4.40.v20210413 used in Jenkins LTS 2.289.2.

A backport has also been provided for the CloudBees CI 2.277 release line in version 2.277.4.4, that is version 2.277.4.3 packaged with the fixed Jetty / Winstone version.

Related Issue(s)

Resolution

The solution is to upgrade CloudBees CI to version 2.289.2.2 or later.

Workaround

If impacted, but the upgrade to 2.289.2 is not an option, the workaround is to upgrade to version 2.277.4.4.