Client controller cannot connect to Operation Center

3 minute read

Symptoms

For the first time or suddenly, a Client controller (CC) cannot be connected to the Operation Center (OC) instance.

Diagnostic/Treatment

Pre-condition: CC and OC instances you wish to connect are up and running.

These are the steps you should follow to understand why your client controller doesn’t connect correctly with the OC instance.

[OC] Ensure that Jenkins URL is correctly set-up

In the OC instance ensure that under Manage Jenkins -> Configure System the Jenkins URL is the right one. In case OC is accessible from a F5 or an ha-proxy ensure that this is correctly configure to forward the packages to OC from an external machine.

[OC] Ensure that a JNLP port is configured

In the OC instance ensure that under Manage Jenkins -> Configure Global security a JNLP port is configured. If your OC instance is running behind a F5 or ha-proxy then a fixed port is required. In any case, a fixed JNLP port is recommended.

[CC] Ensure that the JNLP port is different

In the CC instance ensure that under Manage Jenkins -> Configure Global security if a JNLP port is configured, it is different from the one configured in OC. In any case, a fixed JNLP port is recommended. Note: Use a different JNLP port for each of the CC instances connected to the OC.

[CC] Check the HTTP Proxy configuration

Go to $CC_URL > Manage Jenkins > Plugin Manager > Advanced tab > HTTP Proxy Configuration section and check if there is any Server and Port configured. In the case they have been set up, FQN of the URL (without the port) must be included into No Proxy Host field (e.g: cjoc.jenkins.example.com)

Alternatively, No Proxy Host can be configured by the JAVA property -Dhttp.nonProxyHosts in the CC instance.

[CC] Check if OC instance is reachable through HTTP/(S)

In case you have access to the Script Console run the following script replacing <OC_URL> (e.g: https://ops-center.example.com)

def url = new URL("<OC_URL>");
def connection = url.openConnection();
println("Response Headers");
println("================");
for (def e in connection.getHeaderFields()) {
  println("${e.key}: ${e.value}");
}
println("\nResponse status: HTTP/${connection.responseCode}\n");
▼
Possible expected responses status are:

  • Response status: HTTP/200 (OK)

  • Response status: HTTP/403 (Forbidden) is due to the OC Security Option is enabled and it is also expected. You can confirm this connection is working by requesting the /login endpoint (e.g: https://ops-center.example.com/login). It should return a HTTP/200 (OK) if the connection is successful.

That gives hints as to what the JVM proxy config is doing

In case you don’t have access to the Script Console, curl command is an alternative to validate if the JNLP port is exposed (header X-Jenkins-CLI-Port) as explained on the sub-sections below, replacing <OC_FQN> (e.g. ops-center.unicorn.beescloud.com )

CC is not a TLS end-point

From the CC instance perform curl -I -v https://<OC_FQDN>/

CC is a TLS end-point

From the CC instance perform curl -I -v --insecure https://<OC_FQDN>/

[CC] Check if OC instance JNLP port is open for CC

From the CC instance perform telnet <OC_IP_ADDRESS> <OC_JNLP_PORT>, replacing <OC_IP_ADDRESS> (e.g. 10.0.0.209) and the obtained <OC_JNLP_PORT> (by curl) in the previous step (e.g. 10000).

Expected output would be similar to:

Trying 10.0.0.209…​ Connected to 10.0.0.209.

[CC] Check if OC can answer to CC instance through jenkins-cli

In the CC instance download <OC_URL>/jnlpJars/jenkins-cli.jar. Then, on the client controller execute the following command to see if it is correctly executed. As an example, for Linux OS: wget <OC_URL>/jnlpJars/jenkins-cli.jar

For the following sections the expected output would be the same: a list and description of all the available commands for Jenkins CLI

[CC] OC is not a TLS end-point

From client controller execute:

java -jar jenkins-cli.jar -s https://<OC_FQN>/ help --username=<USERNAME> --password=<PASSWORD>
▼

[CC] OC is a TLS end-point

From client controller execute:

java -Djavax.net.ssl.trustStore=<PATH_TO_CACERTS> -jar jenkins-cli.jar -s https://<OC_FQN>/ help --username=<USERNAME> --password=<PASSWORD>
▼

In case you are running behind a F5 or an ha-proxy

Ensure that:

  1. The F5 is using raw tcp mode for the CLI port

  2. The F5 is not doing any funky keep-alive stuff on the F5

  3. the F5 is not doing any round robin on the CLI port

[OC] In case your OC instance is a TLS end-point

If the OC instance is deployed on a TLS end-point, you must import the SSL certificate in the Java Keystore of the Client controller. In case the Client controller is deployed on a Tomcat web container, you might need to tell what keystore Jenkins is using. This should verify that Tomcat is using the correct keystore.

Check the following KB articles:

Tested product/plugin versions

The latest update of this article has been tested/validated with