KBEC-00164 - Setting Linux server and agent file descriptor limits

Article ID:360032830892
2 minute readKnowledge base

Summary

A file descriptor is an object a process uses to read or write to an open file and open network sockets (although there are other uses).

Operating systems place limits on the number of file descriptors a process can open. In addition to per-process limits, an OS also has a global limit on the number of file descriptors that all its processes, together, may consume.

A common bottleneck in the default Linux operating system configuration is a lack of file descriptors.

Server Solution

Description

An CloudBees CD (CloudBees Flow) Server uses approximately one file descriptor per running job step and three per uncompleted job.

Example

  1. You must tell CloudBees CD (CloudBees Flow) to use the new limits. Add the following line to the init script for the CloudBees CD (CloudBees Flow) Server (in /etc/init.d/commander) before the su - command:

     ulimit -n 32768
  2. Restart the CloudBees CD (CloudBees Flow) server:

     /etc/init.d/commanderServer restart

Agent Solution

Description

An CloudBees CD (CloudBees Flow) Agent uses at least two file descriptors per running job step.

It is important to make sure that operating systems on high traffic sites are configured to provide sufficient numbers of file descriptors to CloudBees CD (CloudBees Flow).

Example

The following example describes how to raise the maximum number of file descriptors to 32768 for the CloudBees CD (CloudBees Flow) process on the RedHat Linux distribution:

  1. Allow all users to modify their file descriptor limits from an initial value of 1024 up to the maximum permitted value of 32768 by changing /etc/security/limits.conf. The following two lines should be part of the file contents:

    • soft nofile 1024

    • hard nofile 32768

  2. In /etc/pam.d/login, add the following line if it does not already exist:

     session required pam_limits.so
  3. You must tell CloudBees CD (CloudBees Flow) to use the new limits. Add the following line to the init script for the CloudBees CD (CloudBees Flow) Agent (in /etc/init.d/ecmdrAgent or /etc/init.d/commanderAgent):

     ulimit -n 32768
  4. Restart the CloudBees CD (CloudBees Flow)agent:

     /etc/init.d/ecmdrAgent restart