ALB Ingress group name changes for 31st managed controller on AWS EKS with external-dns

When provisioning new managed controllers on AWS EKS, using ALB with operations center Ingress annotations configured to set external-dns.alpha.kubernetes.io/hostname, the 31st managed controller is provisioned with a different Ingress group name (alb.ingress.kubernetes.io/group.name) than the previous managed controllers.

It’s normal for the 31st managed controller to have a different Ingress group name, as CloudBees CI shards ALBs to a different group.name after 30 managed controller by default. However, when using external-dns with the external-dns.alpha.kubernetes.io/hostname annotation on all Ingress resources, this causes issues with external DNS changing the DNS records to point to the new managed controller because the ALB created for the 31st managed controller is not associated with the same Ingress group as the previous managed controllers.

To resolve this issue, remove the external-dns.alpha.kubernetes.io/hostname Ingress annotation from your CloudBees CI OperationsCenter.Ingress.Annotations helm values, as it is not required.

Alternatively, you can prevent the operations center Ingress annotations from being propagated to managed controllers by using the operations center Manage Jenkins  Configure Controller Provisioning and put the desired annotations for managed controller in the Ingress Annotations field, and exclude the alb.ingress.kubernetes.io/group.name annotation.