Issue
-
I am using the Amazon EC2 plugin to provision ephemeral agents and Jenkins thread count is progressively growing. Threads like the following are being leaked:
"Thread-10000" daemon prio=5 RUNNABLE java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.socketRead(SocketInputStream.java:116) java.net.SocketInputStream.read(SocketInputStream.java:171) java.net.SocketInputStream.read(SocketInputStream.java:141) com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41) com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52) com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79) com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108) com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232) com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:706) com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502) java.lang.Thread.run(Thread.java:748)
Environment
-
CloudBees CI (CloudBees Core) on modern cloud platforms - Managed controller
-
CloudBees CI (CloudBees Core) on modern cloud platforms - Operations Center
-
CloudBees CI (CloudBees Core) on traditional platforms - Client controller
-
CloudBees CI (CloudBees Core) on traditional platforms - Operations Center
-
Amazon EC2 plugin < 1.66
Explanation
When using connectBySSHProcess - labelled as "Connect by SSH Process" in the EC2 Cloud Templates configuration - the EC2 Cloud actually still uses trilead SSH during the early setup of agent and is leaking threads after each provisioning.
Jenkins eventually needs to be restarted to clean up those threads.
Workaround
Set connectBySSHProcess to false in the EC2 Cloud Templates configuration. In the UI, this is equivalent to disabling the "Connect by SSH Process" checkbox.
Solution
Upgrade the Amazon EC2 Plugin to version 1.66 or later.