Thread leak in Amazon EC2 Plugin when using Connect by SSH Process

Article ID:4412445646107
1 minute readKnowledge base

Issue

  • I am using the Amazon EC2 plugin to provision ephemeral agents and Jenkins thread count is progressively growing. Threads like the following are being leaked:

      "Thread-10000" daemon prio=5 RUNNABLE
          java.net.SocketInputStream.socketRead0(Native Method)
          java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
          java.net.SocketInputStream.read(SocketInputStream.java:171)
          java.net.SocketInputStream.read(SocketInputStream.java:141)
          com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)
          com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)
          com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)
          com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)
          com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)
          com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:706)
          com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502)
          java.lang.Thread.run(Thread.java:748)

Explanation

When using connectBySSHProcess - labelled as "Connect by SSH Process" in the EC2 Cloud Templates configuration - the EC2 Cloud actually still uses trilead SSH during the early setup of agent and is leaking threads after each provisioning.

Jenkins eventually needs to be restarted to clean up those threads.

Workaround

Set connectBySSHProcess to false in the EC2 Cloud Templates configuration. In the UI, this is equivalent to disabling the "Connect by SSH Process" checkbox.

Solution

Upgrade the Amazon EC2 Plugin to version 1.66 or later.

This article is part of our Knowledge Base and is provided for guidance-based purposes only. The solutions or workarounds described here are not officially supported by CloudBees and may not be applicable in all environments. Use at your own discretion, and test changes in a safe environment before applying them to production systems.