Performance issues caused by excessive amount of Credentials Lookups from Open Telemetry plugin

1 minute readKnowledge base

Issue

  • CloudBees CI controller becomes slow and eventually unresponsive

  • Thread dumps and slow requests reveal stacktraces similar to the following:

    at com.cloudbees.plugins.credentials.CredentialsProvider.lookupCredentials(CredentialsProvider.java:348) at io.jenkins.plugins.opentelemetry.authentication.HeaderAuthentication.getAuthenticationHeaderValue(HeaderAuthentication.java:42) at io.jenkins.plugins.opentelemetry.authentication.HeaderAuthentication.enrichOtelEnvironmentVariables(HeaderAuthentication.java:65) at io.jenkins.plugins.opentelemetry.JenkinsOpenTelemetryPluginConfiguration.getOtelConfigurationAsEnvironmentVariables(JenkinsOpenTelemetryPluginConfiguration.java:405) at io.jenkins.plugins.opentelemetry.job.OtelEnvironmentContributorService.addEnvironmentVariables(OtelEnvironmentContributorService.java:66) at io.jenkins.plugins.opentelemetry.job.OtelEnvironmentContributor.buildEnvironmentFor(OtelEnvironmentContributor.java:31) at hudson.model.Run.getEnvironment(Run.java:2379)

Explanation

The Open Telemetry plugin includes a feature that injects Open Telemetry credentials as environment variables, that may be activated with the option Export OpenTelemetry configuration as environment variables under Manage Jenkins > System > OpenTelemetry. When enabled, the global credentials used by Open Telemetry are resolved for every pipeline step execution. This is particularly impactful in environments that use external Credentials provider implementation.

Resolution

Disable the option Export OpenTelemetry configuration as environment variables under Manage Jenkins > System > OpenTelemetry

This article is part of our Knowledge Base and is provided for guidance-based purposes only. The solutions or workarounds described here are not officially supported by CloudBees and may not be applicable in all environments. Use at your own discretion, and test changes in a safe environment before applying them to production systems.