Remoting Best Practices for CloudBees CI on Traditional Platforms

Article ID:115002654991
3 minute readKnowledge base

Issue

  • This article presents an overview of the best practice for configuring Remoting Agents for CloudBees CI on Traditional Platforms: Permanent Agents (in the Controllers side) and Share Agents/Cloud (in the Operation Center). Note that those types of agents can be configured in Modern Platform too, however CloudBees Modern Platforms use Kubernetes plugin for agent provisioning.

Environment

Resolution

Global Security

A - Agents Protocols: Recommended setup (default in 2.73.1+)

  1. Java Web Start Agent Protocol/4 (TLS encryption)

  2. Diagnostic-Ping

  3. OperationsCenter2

B - Disable Enable CLI over Remoting see Remoting Connection Mode.

C - Enable Agent → Controller Access Control

Agent Daemonization

To setup SSH connected agents, follow How to Connect to Remote SSH Agents?.

To install Windows agents as a service, follow How to install Windows agents as a service?.

Number of executors

Built-In Node (formerly known as "master" node) should be configured as:

  • # of executors as 0

  • label as Only build jobs with label expressions matching this node

Building jobs on the built-in node can cause performance issues on the controller, but it also is a security issue, as anyone who can configure jobs could read/write/delete any files in the JENKINS_HOME of the controller.

For all other build agents:

  • Use 1 executor on each agent as a general recommendation

If you are considering using multiple executor slots, ensure that you have designed your jobs to not overwhelm the build agent and cause build times to be impacted (for example, if you encounter a bottleneck with CPU, disk I/O, RAM, etc). One way to avoid bottlenecks is to split up the machine that is running an agent with 20 executors, into 20 virtual machines, each running one executor each. That would ensure that a consistent amount of CPU/RAM/disk would be available to any build that runs on that agent, and generally (though it depends on the specific settings you used on your virtualization software) your build times would be consistent.

Setting the executor count on an agent to a value too high could cause builds on that agent to behave in an unexpected way when the machine is overloaded (taking longer than normal, OutOfMemory exceptions, too many open files).

Since operations-center-cloud > 1.8.4, OperationsCenterNodeProvisioningStrategy provides much faster provisioning response. This means that nodes are requested as soon as possible for nodes that can be provisioned from operations center. To enable it, use single shot mode (retentionStrategyShotCount=1 and # of executors = 1) see Why do agents show as suspended while jobs wait in the queue.

Websocket for Inbound Agents

For Inbound Agents is recommended to use WebSocket to connect to the Jenkins controller rather than the TCP port. See JEP-222 for background.

Agents Monitoring

Agent remoting and JVM versions

  • Agent Monitoring Versions This plugin offers two Node Monitors: one for the version of the Remoting library, and one for the JVM. This helps be aware and possibility automatically disconnect agents that don’t comply to configuration and avoid issues. Note: JVM Version > Enable disconnect Agent when incompatibility is found.

CloudBees Monitoring Plugin

Ensure Logging is enabled in your Agents

See Remoting Logging.

Agents executing builds from more than one controller

  • Recommendation: Shared Agent/Shared Cloud managed by operations center.

  • For Permanent agents, 2 options:

    • Docker container for Permanent Agents running in the same host.

    • If you are not using Docker containers, create one user per agent and use different user home as Remote FS for each agent. Example for "Remote Machine A":

      • "Permanent Agent 1" defined in "Controller 1" - Remote FS: "/home/user1" (Host: "Remote Machine A")

      • "Permanent Agent 2" defined in "Controller 2" - Remote FS: "/home/user2" (Host: "Remote Machine A")