Issue
The Jenkins logs regularly show the following stacktrace:
2020-11-04 06:49:56.143+0000 [id=329776] WARNING i.f.k.c.d.i.WatchConnectionManager$1#onFailure: Exec Failure java.util.concurrent.RejectedExecutionException: Task okhttp3.RealCall$AsyncCall@556ba43e rejected from java.util.concurrent.ThreadPoolExecutor@217680e3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 30] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at okhttp3.RealCall$AsyncCall.executeOn(RealCall.java:183) Caused: java.io.InterruptedIOException: executor rejected at okhttp3.RealCall$AsyncCall.executeOn(RealCall.java:186) at okhttp3.Dispatcher.promoteAndExecute(Dispatcher.java:186) at okhttp3.Dispatcher.enqueue(Dispatcher.java:137) at okhttp3.RealCall.enqueue(RealCall.java:127) at okhttp3.internal.ws.RealWebSocket.connect(RealWebSocket.java:193) at okhttp3.OkHttpClient.newWebSocket(OkHttpClient.java:435) at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager.runWatch(WatchConnectionManager.java:156)
Resolution
This exception is due to the Kubernetes plugin trying to cleanup old Kubernetes clients using an expired authentication token. Version 1.28.1 of the Kubernetes plugin contains a fix that simplifies the Kubernetes client management and should avoid this stack.