Issue
-
How do I generate a thread dump.
-
Diagnosing performance or execution issues, for instance, thread deadlock related to server hangs.
Resolution
In order to diagnose performance issues we can gather information about what is happening inside the JVM while it is experiencing issues. One of the tools used to help diagnose performance issues is a thread dump which displays all currently running threads.
If Jenkins or its agents are operating normally, you can obtain a thread dump remotely by going to https://your.jenkins.example/threadDump. For an agent named 'xyz', go to https://your.jenkins.example/computer/xyz/systemInfo. You need to have the administrator permission on the system. This is by far the easiest.
Following, alternative methods are provided when Jenkins is not responding to web UI or complementary info was needed.
1. Get the PID of the Java process where Jenkins is running
To generate a thread dump the first part is to obtain the process id (or PID). There are multiple methods to obtain the PID:
-
jps - From Unix Terminal or Windows Cmd with the JDK installed into the OS. It lists the instrumented Java Virtual Machines (JVMs) on the target system.
-
ps -ef **
* *grep jenkins - From Unix Terminal only. It is used to obtain the process id of all java processes.
-
Process explorer - There are task manager UI applications (System Tools) which can be used to obtain the process id. Find the process and view its PID in the corresponding column.
-
On Windows and Linux (Ubuntu) is named as
Task manager -
On Mac OS is named as
Activity Monitor
-
2. Get the thread dump
After obtaining the process id ($PID), the next step is to generate the thread dump. There are multiple ways to obtain the thread dump.
-
kill -3 $PID - From Unix Terminal only. It will NOT kill the process, but instead generate a thread dump. This NOT generate any output which is normal. The output of this command is sent to system err/out logs.
-
jstack $PID - From Unix Terminal or Windows Cmd with the JDK installed into the OS. It prints Java thread stack traces for a Java process, core file, or remote debug server. This commands need to be run with the same user as jenkins. For instance:
sudo -u $JENKINS_USER jstack -F $PID > jenkins-threadump.txt -
jcmd - The release of JDK 8 introduced Java Mission Control, Java Flight Recorder, and
jcmdutility for diagnosing problems with JVM and Java applications. It is suggested to use the latest utility,jcmdinstead of the previousjstackutility for enhanced diagnostics and reduced performance overhead. This commands need to be run with the same user as jenkinssudo -u $JENKINS_USER jcmd $PID Thread.print > jenkins-threadump.txt -
jconsole - From Unix Terminal or Windows Cmd with the JDK installed into the OS. The JConsole graphical user interface is a monitoring tool (real time) that complies to the Java Management Extensions (JMX) specification. JConsole uses the extensive instrumentation of the Java Virtual Machine (JVM) to provide information about the performance and resource consumption of applications running on the Java platform.
Generating a few of these thread dumps during the performance issue would be required. Attach the output generated from these commands to a cloudbees support ticket, so we can help diagnose further.
Notes
-
Some of these commands might require admin permissions.
-
When JDK is required (it does not mean JRE). To run the proposed commands you need to run them from
$JDK_PATH/binor from anywhere by including$JDK_PATH/bininto your System$PATH. On the other hand, since JDK7 Update 21, the server JRE includes these tools IIRC as well. -
For Windows >
Task Managerif PID is not displayed > click View - Select Columns > Select the PID (Process Identifier). -
As an option, for running Unix commands into Windows Cygwin can be installed.
-
Please check the KB CloudBees Jenkins Platform supported Java versions.