Symptoms
-
I am not able to connect my CC (Client controller) instances to CJOC
-
I am experiencing communication issues between CJOC and client controller
-
My client controller gets disconnected from CJOC
-
My shared agents gets disconnected from CJOC
How to use the tool
First of all you need to activate the diagnostic ping option in your Operation Center UI by going to the /configureSecurity page and ticking the Diagnostic-Ping combobox as indicated in the screenshot below:
Then, we will run the diagnostic tool. The tool should be run from the client controller to CJOC to diagnose connection problems but from agents to CJOC for JNLP shared-slaves, can be another use case.
Download and run the diagnostic tool using
java -jar network-diagnostic-tool-jar-with-dependencies.jar JENKINS_URL
where the JENKINS_URL is the url of your CJOC instance. For this tool to work you will need to install operations-center-agent plugin v 2.32.0.2 in your jenkins instance.
Once the execution of the jar file starts, it will attempt to connect to CJOC using a diagnostic protocol which will respond with some information useful for diagnosing the issues. The tool will run continuously until the user manually interrupts it. During the execution the tool will attempt to establish the connection by intervals of time: the first connection is attempted right away, the next one after a minute, the subsequent after 5 mins and so adding 5 mins each time (0, 1, 5, 11, 16 …)
The tool should create a DiagnosticPing.log file which should contain something similar to
***********START***********
[Tue Feb 21 18:22:53 CET 2017] Running Diagnostic Tool ...
[INFO] Keystore accepted issuers:
cn=equifax secure global ebusiness ca-1,o=equifax secure inc.,c=us
cn=jenkins.localhost.local,o=cjoc,st=some-state,c=es
.........
cn=keynectis root ca,ou=root,o=keynectis,c=fr
cn=digicert global root g2,ou=www.digicert.com,o=digicert inc,c=us
[INFO] Could validate certificate chain [[
[
Version: V1
Subject: CN=jenkins.localhost.local, O=CJOC, ST=Some-State, C=ES
Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5
Key: Sun RSA public key, 1024 bits
modulus: 123690420915541172355350985742947998481593665150361523029148588723605634407316579618426939116522313678829343537604890104047645438170736763880086007511696193263834139108303041892996006756004471193740703923993764355081288110242120044899639924271312318738917097269688971780130154842969239702491585774700224570259
public exponent: 65537
Validity: [From: Mon Nov 28 00:40:20 CET 2016,
To: Tue Nov 28 00:40:20 CET 2017]
Issuer: CN=jenkins.localhost.local, O=CJOC, ST=Some-State, C=ES
SerialNumber: [ e0e17304 c103d646]
]
Algorithm: [SHA1withRSA]
Signature:
0000: 77 F4 EF 1F BC 8B AD F3 F3 6D AE 03 DF 2C 16 DB w........m...,..
0010: 9D DC 3B A6 96 D1 0A 4C 96 F4 6B C9 6C 81 E2 84 ..;....L..k.l...
0020: 6B FD 89 68 5F 79 C5 81 1E E7 95 9C CC 2B 3E 32 k..h_y.......+>2
0030: 62 6D 2F 21 35 C7 32 A9 F1 B2 9A 2A 82 B2 34 0D bm/!5.2....*..4.
0040: 91 A0 B7 9F 89 30 4F 62 00 52 6D D5 54 60 BA ED .....0Ob.Rm.T`..
0050: BE E4 F8 90 5D 91 03 B2 C3 E7 DE 09 DE 66 AB 78 ....]........f.x
0060: DC 32 C5 51 01 E7 BB 85 43 6D 31 36 56 3F AA 88 .2.Q....Cm16V?..
0070: 8E 3F 6A 3E 40 FF 73 39 20 26 10 6E D6 5C D4 CC .?j>@.s9 &.n.\..
]]
[INFO] headers received:
"Date=Tue, 21 Feb 2017 17:22:57 GMT";
"X-Content-Type-Options=nosniff";
"Set-Cookie=JSESSIONID.6d9d200b=654xh0p5oeb412rma3bsno44u;
Path=/;
Secure;
HttpOnly";
"Expires=Thu, 01 Jan 1970 00:00:00 GMT";
"Content-Type=text/html;
charset=UTF-8";
"X-Hudson=1.395";
"X-Jenkins=2.19.4.2-rolling";
"X-Jenkins-Session=90bd5cd7";
"X-Hudson-CLI-Port=50000";
"X-Jenkins-CLI-Port=50000";
"X-Jenkins-CLI2-Port=50000";
"X-Jenkins-CLI-Host=cjoc.localhost.local";
"X-You-Are-Authenticated-As=anonymous";
"X-You-Are-In-Group=";
"X-Required-Permission=hudson.model.Hudson.Read";
"X-Permission-Implied-By=hudson.security.Permission.GenericRead, hudson.model.Hudson.Administer";
"Connection=close";
"Server=Jetty(9.2.z-SNAPSHOT)"
[INFO] Jenkins Session ID: 90bd5cd7
[INFO] found jnlp host: cjoc.localhost.local
[INFO] found jnlp port: 50000
[INFO] Trying to connect via TCP/IP tocjoc.localhost.local
[INFO] Trying direct connection, no proxy configured in system properties
[INFO] Sending a PING...
[INFO] remote Jenkins instance answered with a PONG response
[INFO] Remote Diagnostic Protocol Version is: 1.0
[INFO] Jenkins Server Session ID is: 90bd5cd7
[INFO] Jenkins Server Session matches the sessionId retrieved during the Http headers discovery phase, we're talking to the same instance
[INFO] remote socket address 192.168.99.1
[INFO] remote socket address matches local host IP, there is no proxy on the way
[Tue Feb 21 18:23:03 CET 2017] PONG received, jnlp connection to oc-agent established successfully
[Tue Feb 21 18:23:03 CET 2017] trying again in 1 minutes
[INFO] Sending a PING...
[INFO] remote Jenkins instance answered with a PONG response
[INFO] Remote Diagnostic Protocol Version is: 1.0
[INFO] Jenkins Server Session ID is: 90bd5cd7
[INFO] Jenkins Server Session matches the sessionId retrieved during the Http headers discovery phase, we're talking to the same instance
[INFO] remote socket address 192.168.99.1
[INFO] remote socket address matches local host IP, there is no proxy on the way
[Tue Feb 21 18:24:03 CET 2017] PONG received, jnlp connection to oc-agent established successfully
[Tue Feb 21 18:24:03 CET 2017] trying again in 6 minutes
[INFO] Sending a PING...
[INFO] remote Jenkins instance answered with a PONG response
[INFO] Remote Diagnostic Protocol Version is: 1.0
[INFO] Jenkins Server Session ID is: 90bd5cd7
[INFO] Jenkins Server Session matches the sessionId retrieved during the Http headers discovery phase, we're talking to the same instance
[INFO] remote socket address 192.168.99.1
[INFO] remote socket address matches local host IP, there is no proxy on the way
[Tue Feb 21 18:30:03 CET 2017] PONG received, jnlp connection to oc-agent established successfully
[Tue Feb 21 18:30:03 CET 2017] trying again in 11 minutes
whereas the Main class will produce the following output to standard output
Trying to reach endpoint via http at: https://jenkins.localhost.local:8443
DiagnosticHeaders{jnlpPort='50000', jnlpHost='cjoc.localhost.local', jenkinsSessionId='90bd5cd7'}
Received Response, creating a socket using JNLP
Socket established successfully
[Tue Feb 21 18:33:03 CET 2017] PONG received, jnlp connection to oc-agent established successfully
[Tue Feb 21 18:33:03 CET 2017] trying again in 1 minutes
[Tue Feb 21 18:34:03 CET 2017] PONG received, jnlp connection to oc-agent established successfully
[Tue Feb 21 18:34:03 CET 2017] trying again in 6 minutes
[Tue Feb 21 18:40:03 CET 2017] PONG received, jnlp connection to oc-agent established successfully
[Tue Feb 21 18:40:03 CET 2017] trying again in 11 minutes
Normally executing the tool for 25 minutes is more than enough. You would need to send both the .log file and the console output to the support team.