Issue
-
I am using a separate namespace for build agent pods with the helm attribute
.Agents.SeparateNamespace
but Build Agents do not get provisioned -
When trying to provision a build agent pod, the controller shows issues like the following:
2021-02-01 19:54:15.065+0000 [id=31] WARNING o.c.j.p.k.KubernetesCloud#provision: Failed to count the # of live instances on Kubernetes io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://10.1.2.3/api/v1/namespaces/agents-ns/pods?labelSelector=jenkins%3Dslave. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods is forbidden: User "system:serviceaccount:cloudbees-core:jenkins" cannot list resource "pods" in API group "" in the namespace "cloudbees-core". at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:589) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:526) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:492) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:451) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:433) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:151) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:635) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:82) at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.getPodsWithLabels(KubernetesCloud.java:635) at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.provision(KubernetesCloud.java:550)
Environment
-
CloudBees CI (CloudBees Core) on Modern Cloud Platforms >= 2.263.1.2 and < 2.263.4.1
-
CloudBees CI (CloudBees Core) on Modern Cloud Platforms - Managed controller >= 2.263.1.2 and < 2.263.4.1
-
CloudBees CI (CloudBees Core) on Modern Cloud Platforms - Operations Center >= 2.263.1.2 and < 2.263.4.1
-
Kubernetes Plugin >= 1.28.0
Explanation
This is caused by a change / improvement made to the Kubernetes plugin #835 in version 1.28.0. When the Kubernetes plugin check for existing live instances of agent pods (to calculate if within the limits), it uses the namespace configured in the Kubernetes Cloud. However, CloudBees CI does not explicitly configure the Kubernetes Cloud with the agent namespace when using .Agents.SeparateNamespace
.
Resolution
The issue has been resolved in CloudBees CI 2.263.4.1.
Upgrade CloudBees CI to version 2.263.4.1 or later.