KBEA-00088 - Builds are suddenly slower than expected

Article ID:360032826812
2 minute readKnowledge base

Summary

Build performance has suddenly and unexpectedly degraded.

Solution

Sudden changes in build performance can be caused by a variety of factors. Here are some things to check:

  • Is your build using a good history file?

  • Are you running with the same number of agents in the "fast" and "slow" cases?

  • Are the agents used in the "fast" and "slow" builds comparable in terms of hardware configuration?

  • Are the eMake hosts used in the "fast" and "slow" builds comparable in terms of hardware configuration?

    • Is the license manager accessible from the agents?

    • Is the license manager configuration on the agents correct? For example, on Windows the vxWorks compilers may store configuration information in HKEY_LOCAL_MACHINE\SOFTWARE\FLEXlm License Manager\WRSD_LICENSE_FILE. If this value exists on your agent hosts, verify that it has the correct value for your site.

    • Is the license manager configuration in your environment correct? For example, the vxWorks compilers may use the LM_LICENSE_FILE environment variable to determine how to contact the license manager.

  • Are the "fast" and "slow" builds actually comparable builds? Are they building the same sources? Do they run the same number of jobs?

  • Are you using third-party tools that are explicitly licensed, such as vxWorks compilers?

  • Is disk performance slow on the eMake machine?

    • Use ecdiskperf to determine the speed of the disk subsystem. You should aim for speeds around 100 MB/s for reads and 80 MB/s for writes.

  • Is network performance slow?

    • Are the agents on the same switch?

    • It should be at least a 100 Mbit network (1 Gbit is recommended)

  • Is the NFS/Samba server slow?

    • Try running from a local disk.

  • High levels of logging can affect performance. Try running eMake without high levels of debugging and annotation output.

  • Does the annotation show large times of serial work (use ElectricInsight for this analysis).

  • Is the effective agent allocation 100? 100 means eMake always got all the agents it wanted. If not, was the percentage different when builds were faster?

Related knowledge base articles:

Applies to

  • Product versions: All

  • OS versions: All

This article is part of our Knowledge Base and is provided for guidance-based purposes only. The solutions or workarounds described here are not officially supported by CloudBees and may not be applicable in all environments. Use at your own discretion, and test changes in a safe environment before applying them to production systems.