HA Failing Due To NoClassDefFoundError

Article ID:217944758
1 minute readKnowledge base

Issue

HA can fail with many java.lang.NoClassDefFoundError related exceptions:

 Exception in thread "ConnectionMap.Acceptor [10.0.0.1:56736],null,null" java.lang.NoClassDefFoundError: org/jgroups/blocks/TCPConnectionMap$TCPConnection$Receiver
 at org.jgroups.blocks.TCPConnectionMap$TCPConnection.start(TCPConnectionMap.java:424)
 at org.jgroups.blocks.TCPConnectionMap$Acceptor.handleAccept(TCPConnectionMap.java:310)
 at org.jgroups.blocks.TCPConnectionMap$Acceptor.run(TCPConnectionMap.java:283)
 at java.lang.Thread.run(Thread.java:745)
 Caused by: java.lang.ClassNotFoundException: org.jgroups.blocks.TCPConnectionMap$TCPConnection$Receiver
 at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 at winstone.classLoader.WebappClassLoader.loadClass(WebappClassLoader.java:83)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 ... 4 more
 Apr 01, 2016 3:13:35 PM org.jgroups.util.TimeScheduler2$RecurringTask run
 SEVERE: failed running task STABLE: StableTask
 java.lang.NoClassDefFoundError: org/jgroups/protocols/pbcast/STABLE$1
 at org.jgroups.protocols.pbcast.STABLE.sendStableMessage(STABLE.java:687)
 at org.jgroups.protocols.pbcast.STABLE$StableTask.run(STABLE.java:829)
 at org.jgroups.util.TimeScheduler2$RecurringTask.run(TimeScheduler2.java:561)
 at org.jgroups.util.TimeScheduler2$MyTask.run(TimeScheduler2.java:493)
 at org.jgroups.util.TimeScheduler2$Entry.execute(TimeScheduler2.java:398)
 at org.jgroups.util.TimeScheduler2$1.run(TimeScheduler2.java:258)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
 Caused by: java.lang.ClassNotFoundException: org.jgroups.protocols.pbcast.STABLE$1
 at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 at winstone.classLoader.WebappClassLoader.loadClass(WebappClassLoader.java:83)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 ... 9 more

The problem impacts older versions of CJP and customers who use custom init scripts that probably don’t take the --webroot switch into account. This problem is happening because the default webroot data resides in the ${JENKINS_HOME}/war directory which is a default setting for single node installations of Jenkins.

Newer versions of CJP include a --webroot switch, but older customers may need to make this adjustment manually if they want to use the HA features properly.

Environment

Resolution

Adjust or add the webroot switch and point it ouside of ${JENKINS_HOME}

To resolve the problem, we advise adding --webroot to your init scripts or Jenkins arguments and point it to a folder that exists outside of your shared ${JENKINS_HOME} data directory. Note that the webroot data needs to be stored and accessible on each individual node on it’s local filesystem, so avoid sharing it over NFS.

Quick example:

java -server -jar jenkins.war --webroot=/var/cache/jenkins/war

Make sure that the new folder you create for the webroot data has read/write permissions for your jenkins service account on each HA node.