Required Data: HA (active/active) Hazelcast EventQueue overloaded

1 minute readKnowledge base

Issue

In your controller logs of an HA (active/active) cluster, you see the following error message:

EventQueue overloaded! Failed to process event packet sent from:
▼

Environment

Resolution

  1. To troubleshoot this when running CloudBees CI version 2.452.3.2 or newer:

    1. Collect a support bundle with the About Jenkins, Information from other replicas, and Hazelcast collection summary options.

    2. The support bundle will contain information about the Hazelcast collections in the file hazelcast-collections.txt.

  2. To troubleshoot this when running CloudBees CI versions older than 2.452.3.2:

    1. Add a FINER level logger for com.cloudbees.jenkins.plugins.replication.sync by following Configure Loggers for Jenkins and CloudBees CI. This will provide more information about the events being processed.

      FINER	c.c.j.p.r.s.MessageProvider$ListenerImpl#isActuallyModified: checksum of ... changed so considering save event
      ▼

    2. Run collectPerformanceData.sh to get consecutive thread dumps when the error is happening, by following Required Data: CloudBees CI hang or high CPU on Kubernetes or Linux.

    3. Collect a support bundle with the About Jenkins, Information from other replicas, and Controller Custom Log Recorders options.

  3. Submit the above data to CloudBees Support for further analysis at https://support.cloudbees.com.