Troubleshooting CloudBees CI (CloudBees Core) on Modern Cloud Platforms - Operations Center is not accessible

Article ID:360038638992
2 minute readKnowledge base

Issue

Installed CloudBees Core on Kubernetes but I’m having issues accessing the Operations Center UI.

Resolution

For your browser to access the CloudBees Core UI (i.e., Jenkins) requires an HTTP/S request to go through several layers to reach the pod hosting your new Operations Center. To troubleshoot those layers I would start with the inner layer and work our way outward. First start with the pod, then with the service and finally ingress.

  1. Start with the pod and perform a HTTP request locally,

    kubectl exec -it cjoc-0 -- curl -D- https://localhost:8080/cjoc

    Example output,

     HTTP/1.1 302 Found
     Date: Tue, 03 Dec 2019 16:48:24 GMT
     Location: https://localhost:8080/cjoc/
     Content-Length: 0
     Server: Jetty(9.4.z-SNAPSHOT)
  2. Retrieve the pod IP address,

    kubectl exec -it cjoc-0 -- cat /etc/hosts | grep cjoc

    Example output,

    10.244.1.112 cjoc-0.cjoc.cloudbees-core.svc.cluster.local cjoc-0

  3. And, perform the same HTTP request against the IP,

    kubectl exec -it cjoc-0 -- curl -D- https://10.244.1.112:8080/cjoc

    Which results in the same output,

     HTTP/1.1 302 Found
     Date: Tue, 03 Dec 2019 16:52:45 GMT
     Location: https://10.244.1.112:8080/cjoc/
     Content-Length: 0
     Server: Jetty(9.4.z-SNAPSHOT)
  4. Find the cjoc service IP, i.e. ClusterIP of cjoc, with the command,

    kubectl get service

    Example output,

     NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)            AGE
     cjoc         ClusterIP   10.100.224.84    <none>        80/TCP,50000/TCP   4d2h
     kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP            62d
  5. Use curl again, this time against the cjoc service (Notice: cjoc service is listening on port 80)

    kubectl exec -it cjoc-0 -- curl -D- https://10.100.224.84/cjoc

    Example output,

     HTTP/1.1 302 Found
     Date: Tue, 03 Dec 2019 16:57:42 GMT
     Location: https://10.100.224.84/cjoc/
     Content-Length: 0
     Server: Jetty(9.4.z-SNAPSHOT)
  6. After validating the pod and the service, you can now test ingress

    kubectl get ingress

    Example output,

     NAME    HOSTS                        ADDRESS       PORTS   AGE
     cjoc    cloudbees-core.example.com   192.168.1.3   80      4d3h
    In this example, ingress for cjoc has address 192.168.1.3, this is the eth0 physical interface of this node. On a cloud provider the address will be different.
  7. From your workstation, curl the hostname and not the IP, e.g.,

    curl -D- https://cloudbees-core.example.com/cjoc

    Successful output,

     HTTP/1.1 302 Found
     Server: nginx/1.15.9
     Date: Tue, 03 Dec 2019 17:04:09 GMT
     Content-Length: 0
     Connection: keep-alive
     Location: https://cloudbees-core.example.com/cjoc/

    with IP,

    curl -D- https://192.168.1.3/cjoc

    Unsuccessful output,

     HTTP/1.1 404 Not Found
     Server: nginx/1.15.9
     Date: Tue, 03 Dec 2019 17:07:20 GMT
     Content-Type: text/plain; charset=utf-8
     Content-Length: 21
     Connection: keep-alive

Troubleshooting

If you installed CloudBees Core using the CloudBees Helm chart, e.g.,

helm install cloudbees-core cloudbees/cloudbees-core --set OperationsCenter.HostName=cloudbees-core.example.com

you have seen the name cloudbees-core.example.com. In our example, this is the hostname which ingress expects in HTTP requests in order to forward your request. Therefore you must use the hostname in the request. Below are two methods of providing a Host header, e.g.,

curl -D- --resolve cloudbees-core.example.com:80:[LoadBalancer IP or hostname or ingress IP] https://cloudbees-core.example.com/cjoc

OR

curl -D- -H "Host: cloudbees-core.example.com" https://[Load balancer or ingress address]

In both examples above, a host header is included in the request and ingress will pass the request through to the pod. You can run this command both inside or outside of the cluster.