Issue
Slow requests files in a Support Bundle or a thread dump file show more than 1 explicit GC call every 5 seconds.
The thread stack is similar to:
RequestHandlerThread[#191]" Id=277 RUNNABLE (in native)
at java.lang.Runtime.gc(Native Method)
Environment
-
CloudBees CI (CloudBees Core) on modern cloud platforms - Managed controller
-
CloudBees CI (CloudBees Core) on modern cloud platforms - Operations Center
-
CloudBees CI (CloudBees Core) on traditional platforms - Client controller
-
CloudBees CI (CloudBees Core) on traditional platforms - Operations Center
Explanation
What is System.gc()
System.gc() is an API provided in Java. When invoked it will make its best effort to clear accumulated unreferenced object from memory.
Why I’m seeing this in my instance
System.gc() calls can be invoked from different places:
-
The application developers might be explicitly calling
System.gc()method. -
Sometimes
System.gc()can be triggered by 3rd party libraries, frameworks, etc.. -
It could be triggered from external tools through use of JMX
What’s the problem calling System.gc()?
Calling System.gc() will trigger a stop-the-world Full GC event. That means that while this event lasts the entire JVM will freeze. Normally, these Full GCs take a long duration to complete (depends on heap size, Garbage collector used, …) resulting in a poor user experiences causing slowness and performance issues.
As a general rule, you will never need to use System.gc() and JVM has already its own mechanisms to trigger the GC when is necessary. There are just a few cases where System.gc() is justified and running job Jenkins is not one of them.
Search for the origin of the System.gc() (recommended)
Check the thread dumps looking for the code where System.gc() is called. It could be code executed as part of a Test, a Shared Library, … Once the code is located, ask the developers to remove it
For instance, given a thread dump like:
java.lang.Thread.State: RUNNABLE at java.lang.Runtime.gc(Native Method) at java.lang.System.gc(System.java:928) at com.cloudbees.plugins.support.test.App.A(App.java) at com.cloudbees.plugins.support.test.App.run(App.java) at java.lang.Thread.run(Thread.java:619)
It will suggest that the code in com.cloudbees.plugins.support.test.App.A(App.java) is invoking System.gc()
Disable explicitly GC calls in the JVM
If you are not able to locate the origin of the System.gc() calls or is not part of your code, then you can setting up the flag -XX:+DisableExplicitGC in the JVM options.
This option will block any System.gc() call invoked from anywhere.
Avoid the stop-the-world event to happen
The JVM provides the option -XX:+ExplicitGCInvokesConcurrent, this option allows to GC collections run concurrently along with application threads, to reduce the lengthy pause time.