Resolution
This document provides a mechanism to validate customers’s environments before running CloudBees Jenkins Operations Center. The KB proposes a number of tests that should be carried out to assert the underlying infrastructure works as intended.
Introduction: Connection flow between client controllers and the OC instance
A simple connection flow looks like the image below:
-
Client controller sends
HEADrequest to CJOC root URL -
Client controller parses
HEADresponse headers looking forX-JenkinsandX-Jenkins-CLI-Port. -
Client controller initiates a TCP connection to the port specified in
X-Jenkins-CLI-Portagainst the host specified inX-Jenkins-CLI-Host(or the host from the CJOC root URL if theX-Jenkins-CLI-Hostheader is absent)
If from CJE instance under https://<CJE_URL>/script we launch the script below we can see those headers exposed by CJOC.
def url = new URL("https://<CJOC_URL>/");
def connection = url.openConnection();
println("Response Headers");
println("================");
for (def e in connection.getHeaderFields()) {
println("${e.key}: ${e.value}");
}
println("\nResponse status: HTTP/${connection.responseCode}\n");
An output like the one below is expected:
Response Headers ================ null: [HTTP/1.1 200 OK] X-Jenkins: [1.625.16.2 (CloudBees Jenkins Operations Center 1.8)] X-Hudson: [1.395] Content-Length: [16451] Expires: [0] Set-Cookie: [JSESSIONID.d35196d9=x9jsjuq6ugw3x5t38e6ecfjt;Path=/;HttpOnly] X-Jenkins-CLI-Port: [6587] X-Hudson-CLI-Port: [6587] X-SSH-Endpoint: [127.0.0.1:49691] Server: [Jetty(winstone-2.8)] X-Content-Type-Options: [nosniff] Cache-Control: [no-cache,no-store,must-revalidate] X-Frame-Options: [sameorigin] X-Hudson-Theme: [default] X-Instance-Identity: [MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEApZe+Wd2zoAiWalHu/wgZaFzJS56XYxv1XejB/ml8k4tb+AutXvUhDvxlRYvmkp7CV/NvF3e2YfgH5WLnt5IXXxn5a3FsDQ3xDtN/+kv4+jwTqbZnOORtJB0xET7Pak7fpjrTkUpsR8lGhHdHVf2iQxHRJoeHFblYajNHPx9U7zkrXgLA4xUqqwCPmUys/QXAcVEl6IL/1jmxtdLFQb38NNPr6CDfhrH7DNkwvTX6EDmn1Zljw+AsFBmDbfm/uoPqUCNrrQk/OvIrKebABPdJL0YoIY7naehd8ezNiICrf92TkZNopCrTUNuq6kpe8rhHW0oQf5o0KSrSi869nXG8FwIDAQAB] X-Jenkins-Session: [5e3b53cb] Content-Type: [text/html;charset=UTF-8] X-Jenkins-CLI2-Port: [6587] Response status: HTTP/200
In case you get Response status: HTTP/403 (Forbidden) is due to the CJOC Security Option is enabled and it is also expected.
After the explanation above, we should be able to understand how the flow looks like when pushing the connection details from CJOC to CJE
-
Browser POST to JOC
-
JOC redirect via browser to CJE
-
CJE
HTTP HEADto JOC -
CJE extracts
X-Jenkins-CLI-Port -
CJE establish remoting protocol to CJOC through TCP on JNLP port exposed by CJOC
Summary:
-
Communication between CJOC and CJE happens at HTTP level - on the in which CJOC is accessible through the browser - and at TCP level through the JNLP port exposed by CJOC
-
Communication is ALWAYS initiated by CJE trying to connect with CJOC using HTTP and TCP protocols
Customer CJOC Configuration Validation
On this section, we need to ensure that we have set-up everything correctly at Jenkins level. The image below shows a typical customer environment. A similar environment (plus a firewall configuration) can be found on this tutorial for demo purposes.
In both CJOC and CJE level we need to ensure that:
-
The Jenkins URL was set-up under Manage Jenkins -> Configure System
-
Jenkins is exposing a JNLP under Manage Jenkins -> Configure Global Security. The best practice is to set-up a fixed port in both CJOC and CJE instances, but this port should be different in CJOC and CJE Note: Use a different JNLP port for each of the CJE instances connected to the CJOC.
[Support] Customer CJOC Configuration Validation
If you are going to open a support ticket because you cannot figure out why CJE cannot connect to CJOC, then ensure that you provide us a table similar to the one below so we can understand how your environment looks like.
| Node | Type | IP Address | DNS Name | HTTP(S) PORT | JNLP Port | $JENKINS_HOME |
|---|---|---|---|---|---|---|
CJOCa |
CJOC primary |
joc-a.acme.com |
||||
CJOCb |
CJOC backup |
joc-b.acme.com |
||||
CJOC |
CJOC VIP |
jenkins.acme.com/operations-center |
||||
controller1a |
controller1 primary |
je1-a.acme.com |
||||
controller1b |
controller1 backup |
je1-b.acme.com |
||||
controller1 |
controller1 VIP |
jenkins.acme.com/je1 |
| CJOC/CJE backup are available for High Availability (HA) set-up environments |
Customer CJOC Networking Validation
On this section we will continue with the same typical customer’s architecture we used on the section before:
At networking level CJE needs to be able to access CJOC through HTTP - on the port in which CJOC is accessible- and TCP on the JNLP port exposed by CJOC.
CJOC connectivity tests
Please, provide Test ID and Pass/Fail results for the following table:
| Test ID | Details | Notes |
|---|---|---|
cjoc-desktop-http |
From desktop: browser access to HTTP(s) port on |
As shown on the above figure: jenkins.acme.com/operations-center |
cjoc-desktop-http-primary-instance |
From desktop: browser access HTTP(s) port on |
As shown on the above figure: joc-a.acme.com |
cjoc-desktop-http-backup-instance |
From desktop: browser access HTTP(s) port on |
As shown on the above figure: joc-b.acme.com |
cjoc-dns-resolution |
From desktop: |
|
cjoc-http-tcp-vip |
From desktop: |
|
cjoc-http-tcp-primary-instance |
Shutdown CJOC-b and log on CJOC-a machine, and then: |
|
cjoc-http-tcp-backup-instance |
Shutdown CJOC-a and log on CJOC-b machine, and then: |
controller connectivity tests
Please, provide Test ID and Pass/Fail results for the following table:
| Test ID | Details | Notes |
|---|---|---|
cje-desktop-http |
From desktop: browser access to HTTP(s) port on |
As shown on the above figure: jenkins.acme.com/je1 |
cje-desktop-http-primary-instance |
From desktop: browser access HTTP(s) port on |
As shown on the above figure: je1-a.acme.com |
cje-desktop-http-backup-instance |
From desktop: browser access HTTP(s) port on |
As shown on the above figure: je1-b.acme.com |
cje-http-tcp-primary-instance |
Shutdown CJEa and log on CJEb machine, and then: |
|
cje-http-tcp-backup-instance |
Shutdown CJEb and log on CJEa machine, and then: |
CJE validation for non-fresh instances
Typical scenarios are:
-
CJE instance that was never managed by any CJOC and now moves to a CJE-CJOC architecture.
-
CJE instance that was previous managed by one CJOC and now migrates to another CJOC instance (CJE Migration between CJOC instances)
CJE Admin Login validation (Optional)
Just in the case that a user with Administer permission is not able to login into the CJE instance to connect to a CJOC, CJE Security needs to be disable by following this instructions.
For instance, when CJE Jenkins Dashboard ($CJE_URL) redirects to registration page ($CJE_URL/registration).
