Why are jobs on the queue when there are dedicated agents with free executors

Article ID:235690727
2 minute readKnowledge base

Issue

Jobs stay on the Build queue when there are dedicated agents with free executors.

  • NOTE: This KB article is not applied for shared agents leased from CJOC.

Resolution

There are multiple causes which might produce this issue, but if you want to diagnosis this issue you might want to try running in /script:

for (i in Jenkins.instance.queue.buildableItems) {
  println "considering ${i}"
  for (c in Jenkins.instance.computers) {
    println "found computer ${c}"
    EXEC: for (e in c.executors) {
      if (e.interrupted || !e.parking) continue
      println "with executor ${e}"
      def o = new Queue.JobOffer(e)
      if (!o.canTake(i)) {
        println "${o} refused ${i}"
        def node = o.node
        if (node == null) {
          println "no node associated with ${c}"
          continue
        }
        def cob = node.canTake(i)
        if (cob != null) {
          println "because of ${cob}"
          continue
        }
        for (d in hudson.model.queue.QueueTaskDispatcher.all()) {
          cob = d.canTake(node, i)
          if (cob != null) {
            println "because of ${cob} from ${d}"
            continue EXEC
          }
        }
        if (!o.available) {
          println "${o} not available"
          if (o.workUnit != null) println "has a workUnit ${o.workUnit}"
          if (c.offline) println "${c} is offline"
          if (!c.acceptingTasks) println "${c} is not accepting tasks"
        }
      }
    }
  }
}
This script was proposed in JENKINS-38514

Known causes

Misconfiguration in Authorize Project

In one reported case, the root issue was that the Authorize Project plugin was configured, so Node.canTake was returning anonymous doesn’t have a permission to run on [sic]; yet the build queue (and support bundle) displayed only Waiting for next available executor.

jobs-stay-on-queue.png
considering hudson.model.Queue$BuildableItem:hudson.model.FreeStyleProject@1878318e[test-a]:1
found computer hudson.model.Hudson$MasterComputer@e9bc860
with executor Thread[Executor #0 for master,5,main]
JobOffer[ #0] refused hudson.model.Queue$BuildableItem:hudson.model.FreeStyleProject@1878318e[test-a]:1
because of anonymous doesn’t have a permission to run on
with executor Thread[Executor #1 for master,5,main]
JobOffer[ #1] refused hudson.model.Queue$BuildableItem:hudson.model.FreeStyleProject@1878318e[test-a]:1
because of anonymous doesn’t have a permission to run on