Troubleshooting CloudBees CI (CloudBees Core) on Modern Cloud Platforms - Operations Center is not accessible

Article ID:360038638992
2 minute readKnowledge base

Issue

Installed CloudBees Core on Kubernetes but I’m having issues accessing the Operations Center UI.

Environment

Resolution

For your browser to access the CloudBees Core UI (i.e., Jenkins) requires an HTTP/S request to go through several layers to reach the pod hosting your new Operations Center. To troubleshoot those layers I would start with the inner layer and work our way outward. First start with the pod, then with the service and finally ingress.

  1. Start with the pod and perform a HTTP request locally,

    kubectl exec -it cjoc-0 -- curl -D- https://localhost:8080/cjoc

    Example output,

     HTTP/1.1 302 Found
 Date: Tue, 03 Dec 2019 16:48:24 GMT
 Location: https://localhost:8080/cjoc/
 Content-Length: 0
 Server: Jetty(9.4.z-SNAPSHOT)

  2. Retrieve the pod IP address,

    kubectl exec -it cjoc-0 -- cat /etc/hosts | grep cjoc

    Example output,

    10.244.1.112 cjoc-0.cjoc.cloudbees-core.svc.cluster.local cjoc-0

  3. And, perform the same HTTP request against the IP,

    kubectl exec -it cjoc-0 -- curl -D- https://10.244.1.112:8080/cjoc

    Which results in the same output,

     HTTP/1.1 302 Found
 Date: Tue, 03 Dec 2019 16:52:45 GMT
 Location: https://10.244.1.112:8080/cjoc/
 Content-Length: 0
 Server: Jetty(9.4.z-SNAPSHOT)

  4. Find the cjoc service IP, i.e. ClusterIP of cjoc, with the command,

    kubectl get service

    Example output,

     NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)            AGE
 cjoc         ClusterIP   10.100.224.84    <none>        80/TCP,50000/TCP   4d2h
 kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP            62d

  5. Use curl again, this time against the cjoc service (Notice: cjoc service is listening on port 80)

    kubectl exec -it cjoc-0 -- curl -D- https://10.100.224.84/cjoc

    Example output,

     HTTP/1.1 302 Found
 Date: Tue, 03 Dec 2019 16:57:42 GMT
 Location: https://10.100.224.84/cjoc/
 Content-Length: 0
 Server: Jetty(9.4.z-SNAPSHOT)

  6. After validating the pod and the service, you can now test ingress

    kubectl get ingress

    Example output,

     NAME    HOSTS                        ADDRESS       PORTS   AGE
 cjoc    cloudbees-core.example.com   192.168.1.3   80      4d3h
    In this example, ingress for cjoc has address 192.168.1.3, this is the eth0 physical interface of this node. On a cloud provider the address will be different.

  7. From your workstation, curl the hostname and not the IP, e.g.,

    curl -D- https://cloudbees-core.example.com/cjoc

    Successful output,

     HTTP/1.1 302 Found
 Server: nginx/1.15.9
 Date: Tue, 03 Dec 2019 17:04:09 GMT
 Content-Length: 0
 Connection: keep-alive
 Location: https://cloudbees-core.example.com/cjoc/

    with IP,

    curl -D- https://192.168.1.3/cjoc

    Unsuccessful output,

     HTTP/1.1 404 Not Found
 Server: nginx/1.15.9
 Date: Tue, 03 Dec 2019 17:07:20 GMT
 Content-Type: text/plain; charset=utf-8
 Content-Length: 21
 Connection: keep-alive

Troubleshooting

If you installed CloudBees Core using the CloudBees Helm chart, e.g.,

helm install cloudbees-core cloudbees/cloudbees-core --set OperationsCenter.HostName=cloudbees-core.example.com

you have seen the name cloudbees-core.example.com. In our example, this is the hostname which ingress expects in HTTP requests in order to forward your request. Therefore you must use the hostname in the request. Below are two methods of providing a Host header, e.g.,

curl -D- --resolve cloudbees-core.example.com:80:[LoadBalancer IP or hostname or ingress IP] https://cloudbees-core.example.com/cjoc

OR

curl -D- -H "Host: cloudbees-core.example.com" https://[Load balancer or ingress address]

In both examples above, a host header is included in the request and ingress will pass the request through to the pod. You can run this command both inside or outside of the cluster.

Reference