Operations Center connectivity issues while trying to Push the Connection Details

Article ID:232197548
3 minute readKnowledge base

Symptoms

  • Client controller cannot be connected with Operations Center

Diagnostic/Treatment

  • Pre-condition: Initial connection operations center -> controller

Most of the connectivity issues related to operations center and controller are related to a wrong HTTP or TCP configuration. To diagnosis such kind of issues the best approach is to Push the Connection Details from operations center to controller from Manage in operations center.

cjoc-cm-manage.png

Now, we need to copy/paste the controller URL to finally Push the Connection Details.

cjoc-trouebleshooting-push-connection-details.png

operations center to controller expected healthy status

cjoc-cje-expected-result.png

This is the expected output when we try to connect a controller to an operations center

operations center to controller apparent healthy status

In case the connectivity table shows everything is fine but the connectivity still fails, it is surely there is a Proxy set-up in the controller under Manage Jenkins -> Manage Plugins [Advanced section] and the OC hostname was not added in the Non Proxy Host. i.e If the URL to access OC is https://cjoc.jenkins.example.com, then it is expected to have just cjoc.jenkins.example.com in the No Proxy Host section at controller level.

cjoc-cje-expected-result.png

operations center to controller issue at HTTP level

cjoc-http-connectivity-problem.png
cjoc-http-proxy-connectivity-problem.png

There is a HTTP connectivity issue between operations center and controller.

operations center is not reachable from controller at HTTP level

  • There is a proxy configured in the controller and operations center is not added as No Proxy Host. Ensure that if under Manage Jenkins → Manage Plugins → Advanced Tab there is a proxy configured, then operations center hostname is added to No Proxy Host. i.e cjoc.jenkins.example.com.

  • operations center is not reachable from the controller at HTTP level. From the controller host try to curl operations center instance: curl -I https://cjoc.jenkins.example.com:8888/

    The header X-Jenkins should appear on the output, i.e: X-Jenkins: 2.7.19.0.1 (CloudBees Jenkins Operations Center 2.7.19.0.1-fixed). If this header does not appear it means operations center is not reachable from the controller, which means you need to talk to your network administrator to resolve this issue.

Related KB articles:

Open a Support ticket if you are stuck at this point adding a Screenshot and a support bundle from each end point operations center and controller. Also, add the connection logs (https://cjoc.jenkins.example.com:8888/job/Master-1/log) on a separate file.

operations center to controller issue at TCP level

cjoc-tcp-connectivity-problem.png

There is a TCP connectivity issue between operations center and controller. Usually this happens because an intermediate elements like haproxy, firewall or ELB is blocking the connection. CloudBees recommends to use the System Property -Dhudson.TcpSlaveAgentListener.hostName=<MACHINE_HOSTNAME> on the instance to bypass those intermediate elements.

cjoc-tcp-connectivity-port-problem.png

The above issue happens if the JNLP_PORT advertised by operations center is incorrect. Most likely because the System Property -Dhudson.TcpSlaveAgentListener.port=<JNLP_PORT> has been set in operations center but the <JNLP_PORT> points to an application that is not Jenkins.

Open a Support ticket if you are stuck at this point adding a Screenshot and a support bundle from each end point operations center and controller. Also, add the connection logs (https://cjoc.jenkins.example.com:8888/job/Master-1/log) on a separate file.

operations center to controller issue at SSL level

cjoc-ssl-connectivity-problem.png

This error message means could mean several things:

  1. OC is deployed on an TLS end point using a cert that is not trusted by the client controller JVM

  2. TLS certificate is expired on operations center

  3. The certificate used on OC does not match the Jenkins URL

  4. The reverse proxy uses SNI (see Notes below)

Open a Support ticket if you are stuck at this point adding a Screenshot and a support bundle from each end point operations center and controller. Also, add the connection logs (https://cjoc.jenkins.example.com:8888/job/Master-1/log) on a separate file.

Notes

Operations Center Agent before version 2.222.0.3 does not support TLS SNI

In that particular case, the errors shown in the UI are similar to "operations center to controller issue at SSL level":

cjoc-ssl-connectivity-problem.png

Though the controller logs however show a specific exception about the <operations center_URL>/instance-identity/ endpoint:

WARNING: Pre-validation discovery on https://oc.jenkins.example.com:8888/ failed
javax.net.ssl.SSLHandshakeException: TLS Handshake exception establishing connection to Jenkins server: https://oc.jenkins.example.com:8888/. You might need to trust server's self-signed certificate on global security configuration.
	at com.cloudbees.opscenter.agent.AgentProtocolEndpointLocator.locate(AgentProtocolEndpointLocator.java:530)
	at com.cloudbees.opscenter.client.plugin.OperationsCenterRegistrar$PushRegistrationConfirmation.<init>(OperationsCenterRegistrar.java:500)
	at com.cloudbees.opscenter.client.plugin.OperationsCenterRegistrar$DescriptorImpl.doPushRegistration(OperationsCenterRegistrar.java:316)
	[...]
Caused by: java.util.concurrent.ExecutionException: java.net.ConnectException: General SSLEngine problem to https://oc.jenkins.example.com:8888/instance-identity/

The solution is to upgrade to version 2.222.4.3 or later. A workaround to this problem is to use the certificate CloudBees CI is expecting as the default one in the operations center reverse proxy side. This should work as long as it is the default one.