All pipeline jobs are getting stuck at git step

2 minute readKnowledge base

Issue

Pipeline jobs involving git are stuck at the checkout phase or the log phase. And we can see in the controller logs a warning like the one shown below:

WARNING c.c.o.c.p.OperationsCenterCredentialsProvider#getCredentials: Cannot lookup credentials of type interface com.cloudbees.jenkins.plugins.awscredentials.AmazonWebServicesCredentials from Operations Center
java.lang.InterruptedException
at java.base/java.lang.Object.wait(Native Method)
at hudson.remoting.Request.call(Request.java:177)
at hudson.remoting.Channel.call(Channel.java:1002)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
Caused: hudson.remoting.RemotingSystemException
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:301)
at com.sun.proxy.$Proxy99.getRemoteCredentials(Unknown Source)
at com.cloudbees.opscenter.client.plugin.OperationsCenterCredentialsProvider.getCredentials(OperationsCenterCredentialsProvider.java:140)
at com.cloudbees.plugins.credentials.CredentialsProvider.getCredentialsInItemGroup(CredentialsProvider.java:1191)
at com.cloudbees.plugins.credentials.CredentialsProvider.lookupCredentialsInItemGroup(CredentialsProvider.java:389)
at com.cloudbees.plugins.credentials.CredentialsProvider.lookupCredentials(CredentialsProvider.java:348)
at com.cloudbees.jenkins.plugins.amazonecr.AmazonECSRegistryCredentialsProvider.getCredentials(AmazonECSRegistryCredentialsProvider.java:68)
at com.cloudbees.plugins.credentials.CredentialsProvider.getCredentialsInItemGroup(CredentialsProvider.java:1194)
at com.cloudbees.plugins.credentials.CredentialsProvider.getCredentialsInItemFallback(CredentialsProvider.java:1301)
at com.cloudbees.plugins.credentials.CredentialsProvider.getCredentialsInItem(CredentialsProvider.java:1296)
at com.cloudbees.plugins.credentials.CredentialsProvider.lookupCredentialsInItem(CredentialsProvider.java:563)
at com.cloudbees.plugins.credentials.CredentialsProvider.findCredentialById(CredentialsProvider.java:931)
at hudson.plugins.git.GitSCM.lookupScanCredentials(GitSCM.java:952)
at hudson.plugins.git.GitSCM.createClient(GitSCM.java:927)
at hudson.plugins.git.GitSCM.createClient(GitSCM.java:847)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1294)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:129)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:97)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:84)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)

Resolution

The problem is caused by a misconfiguration (or the lack of it) for AWS Secrets Manager Credentials Provider plugin which is a community plugin that is not maintained by CloudBees.

This issue will cause the pipelines to get stuck even if they are not using the features provided by the plugin.

We have also seen performance issues caused by this plugin in the past, because it required contacting the AWS API every time a credentials dropdown is loaded in Jenkins, and that can happen in a lot of places. If the connection to AWS fails or times out for any reason, it blocks the rendering of the entire page.

Tested product/plugin versions

  • CloudBees CI - Modern Cloud Platforms - controller 2.426.3.3

  • aws-secrets-manager-credentials-provider:1.214.va_0a_d8268d069