Issue
After upgrading the Amazon EC2 plugin to version 2045.v06da_da_a_46422 or later and configuring Windows agents to use SSH connectivity (windows-ssh), the controller accumulates thousands of sshd-SshClient threads. A thread dump shows entries like:
"sshd-SshClient-1234" daemon prio=5 RUNNABLE
org.apache.sshd.common.io.nio2.Nio2Session...
Thread count grows continuously with each failed agent launch attempt.
In observed cases, over 7000 threads accumulated, leading to resource exhaustion and potential OutOfMemoryError: unable to create new native thread.
Environment
-
Amazon EC2 plugin >= 2045.v06da_da_a_46422 and < 2051.v200f12366595
Related Issue(s)
-
BEE-71188 - ec2-plugin: sshd-SshClient thread leak when Windows SSH agent launch fails
Explanation
The EC2 plugin creates an internal SSH client for each agent connection attempt. When the connection fails, the SSH client threads are not properly cleaned up and remain running indefinitely. Windows agents are more likely to trigger this issue because their SSH service takes longer to become available during boot, resulting in repeated failed connection attempts.
Scenarios that can trigger the leak include:
-
SSH not yet reachable on the instance during boot
-
Host key verification failure
-
Authentication timeout
-
Channel open failure
Resolution
Upgrade the Amazon EC2 plugin to version 2051.v200f12366595 or later.
This fix is included in CloudBees CI version 2.555.3.36983 or newer.
After upgrading, restart the controller to reclaim threads leaked by the previous version.
Workaround
If unable to upgrade immediately:
-
Monitor thread count via a thread dump or JMX.
-
Schedule periodic controller restarts to reclaim leaked threads before resource exhaustion occurs.
| This workaround is temporary until you can upgrade to EC2 plugin version 2051.v200f12366595 or later. |
Tested product/plugin versions
-
Amazon EC2 plugin version 2051.v200f12366595
-
CloudBees CI version 2.528.1.29783 (affected)
-
CloudBees CI version 2.555.3.36983 (fixed)