Issue
Triggers defined with Gerrit Trigger plugin are not working.
The Gerrit incoming events queue contains 1008 items! Something might be stuck, or your system can't process the commands fast enough. Try to increase the number of receiving worker threads. Current thread-pool size: 6 The Gerrit incoming events queue contains 6389 items! Something might be stuck, or your system can't process the commands fast enough. Try to increase the number of receiving worker threads. Current thead-pool size: 10
Environment
-
CloudBees CI (CloudBees Core) on modern cloud platforms - Managed controller
-
CloudBees CI (CloudBees Core) on modern cloud platforms - Operations Center
-
CloudBees CI (CloudBees Core) on traditional platforms - Client controller
-
CloudBees CI (CloudBees Core) on traditional platforms - Operations Center
Resolution
The recommended fix is to upgrade Gerrit Trigger plugin to at least version 2.30.1. A downgrade to version 2.28.0 will also fix this issue.
The issue is reported for both 2.29.0 and 2.30.0 versions of Gerrit Trigger plugin. The error happens when disabled jobs are configured with dynamic trigger at Jenkins start time.
Explanation
The executors are waiting for the project to be ready, i.e. the dynamic trigger to be loaded. However, the GerritTriggerTimerTask does a couple of checks (including job.isBuildable()) before updateTriggerConfigURL.
This means the latch introduced in pull request #350 is never released and the executors are blocked forever.