CloudBees VMware Pool Autoscaling plugin
The CloudBees VMware Pool Autoscaling plugin connects to one or more VMWare vSphere installations and uses virtual machines on those installations for better resource utilization when building jobs. Virtual machines on a vSphere installation are grouped into named pools. A virtual machine may be acquired from a pool when a job needs to be built on that machine, or when that machine is assigned to a build of a job for additional build resources. When such a build has completed, the acquired machine(s) are released back to the pool for use by the same or other jobs. When a machine is acquired it may be powered on, and when a machine is released it may be powered off.
This plugin should work with VMware vSphere 4.0 and later (normally tested on 5.1).
The plugin was introduced in Nectar 10.10.
Before jobs can utilize virtual machines it is necessary to configure one or more machine centers, that connect to vSphere installations, and one or more machine pools from within those machine centers, from which builds may acquire machines.
From the main page click on "Pooled Virtual Machines", then click on "Configure" to goto the plugin configuration page.
After configuration the "Pooled Virtual Machines" page will display the current list of configured machine centers. From this page the click on a machine center to see the list of machine pools, and from that page click on a machine pool to see the list of machines.
To configure a machine center enter a name for the center, which will be referred to later when configuring clouds or jobs, the host name where the vCenter service is located that manages the vSphere installation, and the user name/password of the user to authenticate to the vCenter and who is authorized to perform appropriate actions on machines in pools.
One or more machine centers may be configured to the same or different vCenter. For example, different pools may require different users that have authorized capabilities to the same vCenter, or there may be multiple vCenters that can be used. Details can be verified by clicking on the "Test Connection" button.
One or more machine pools may be added to a machine center. There are currently two types of machines pool that can be added. A static pool and a folder pool. In either case power-cycling and power-on wait conditions may be configured for all machines in such pools.
It is guaranteed that a virtual machine will be acquired at most once, even if that machine is a member of two or more pools of two or more centers.
To configure a static machine pool enter a name for the pool, which will be referred to later when configuring clouds or jobs. Then, add one or more static machines to the pool. The name of the static machine is the name of the virtual machine as presented in the vCenter.
Note that there can be two or more virtual machines present in the vCenter with the same name, for example if those machines are located in separate folders or vApps. In such cases it is undetermined which machine will be associated with the configuration.
If the machine is assigned to a build then a set of optional properties, "Injected Env Vars", can be declared that will be injected into the build as environment variables.
To configure a folder machine pool enter a name for the pool, which will be referred to later when configuring clouds or jobs. Then, declare the path to a folder or vApp where the machines contained in that folder or vApp comprise of the pool of machines. If the "recurse" option is selected then machines in sub-folders or sub-vApps will also be included in the pool.
If the machine is assigned to a build then IP address of the virtual machine will be declared in environment variable "VMIP" that is injected into the build.
Virtual machines may be added and removed from the folder or vApp without requiring changes to the configuration. Such pools are more dynamic than static pools.
Power-on actions can be specified after a machine has been acquired from a pool, but before the machine has been assigned to a build. Power-off actions can be specified after a build has finished with the machine, but before that machine has been released back to the pool.
The set of power-on actions are as follows:
- Power up
Powers up the machine. If the machine is already powered up this action does nothing.
- Revert to last snapshot and power up
Revert the machine to the last known snapshot, and then power up. This can be useful to power up the machine in a known state.
- Do nothing
No actions are performed on the machine and it is assumed the machine is already powered on.
The set of power-off actions are as follows:
- Power off
Powers off the machine. This can be useful to save resources. Note that it can take some time for a machine to be powered on and fully booted, hence builds may take longer if the power-cycling represents a significant portion of the overall build time.
Suspend the machine. This can be useful to save resources while being able to power on the machine more quickly.
- Take snapshot after power off
Power off the machine, and then take a snapshot.
- Take snapshot after suspend
Suspend the machine, and then take a snapshot.
- Do nothing
No actions are performed on the machine, and it will be left powered-on.
After a machine is powered on, if power-on actions are configured, and before the machine is assigned to a build, certain wait conditions may be configured that ensure the machine is in an appropriate state.
The set of power-on wait conditions are:
- Wait for a TCP port to start listening
A timeout, default of 5 minutes, can be specified. If waiting for the TCP port to become active takes longer that the timeout, then the machine will be released back to the pool and an error will result.
- Wait for VMware Tools to come up
Optionally, also wait for the machine to obtain an IP address. A timeout, default of 5 minutes, can be specified. If waiting for VMware Tools takes longer that the timeout, then the machine will be released back to the pool and an error will result.
To declare a machine pool as an agent pool, where machines in the pool are agents used to build jobs, it is necessary to configure a VMware cloud. From the main page of Jenkins click on "Manage Jenkins", click on "Configure", goto the "Cloud" section, click on the "Add a new cloud", and select "Pooled VMware Virtual Machines". Select the machine center and a pool from that machine center that shall be used as the pool of agents. Then, configure the other parameters as appropriate. For example, if all machines are Unix machines configured with SSH with the same SSH credentials then select the option to launch agents on Unix via SSH.
When a build is placed on the build queue, whose label matches the configured VMware cloud, then, a machine will be acquired from the pool, powered-on (if configured), and added as node that is assigned to build the job after the power-on wait conditions have been met.
If you have selected the checkbox Only one build per VM in the cloud configuration, then after the build completes the machine will be powered-off (if configured) and released back to the pool. (In this case it only makes sense to configure one executor.) Otherwise, the agent may accept multiple concurrent builds, according to the executor count, and will remain online for a while after the initial build in case other builds waiting in the queue can use it; Jenkins will release the agent to the pool only after it has been idle for a while and does not seem to be needed.
Note that if there are no machines free in the machine pool then the build will wait in the queue until a machine becomes available.
To reserve a machine for a build go to the configuration page of an appropriate job, go to the "Build Environment" section, select "Reserve a VMWare machine for this build", and select the machine center and a pool from that machine center that shall be used as the pool of reservable machines.
When a build of such a configured job is placed on the build queue then, the build will start, a machine will be acquired from the pool, powered-on (if configured), and the build will wait until the power-on wait conditions have been met. After the build completes the machine will be powered-off (if configured) and released back to the pool.
The build itself is not run on this machine; it is merely made
available. To actually use the VM from your build, you would need to
somehow connect to it, typically using the
VMIP environment variable.
Note that if there are no machines free in the machine pool then the build will wait until a machine becomes available.
You may also select multiple machines. In this case all of the build will proceed once all of them are available. (The build may tentatively acquire some, release them, and later reacquire them, to avoid deadlocks.)
From time to time you may need to perform maintenance work on particular VMs. During this time you would prefer for these VMs not to be used by Jenkins.
Click on Pooled Virtual Machines, click on a (static or folder) pool, and click on a machine in that pool. Just press the Take Offline button to prevent Jenkins from reserving it. (You can enter a description of what you are doing for the benefit of other users.) Later when the machine is ready, go back to the same page and press Take Online.
Offline status is remembered across Jenkins restarts.