Configuring CloudBees Build Acceleration to run in Docker containers

CloudBees Build Acceleration includes an integration to support Docker on Linux platforms. This section describes how to configure a containerized Cluster Manager, eMake, or agents.

Supported use cases

CloudBees Build Acceleration supports running a single agent per container and multiple containers per host.

Installing Docker

To use Docker containers, you must download and install Docker. For instructions, see Docker Enterprise Edition or Docker Community Edition.

(Agent installations only) Installing efs and lofs

CloudBees Build Acceleration normally uses its own kernel modules to provide two file systems (EFS and LOFS) required by agents. During a non-container agent installation, these kernel modules are compiled and loaded into the kernel to provide the necessary file system drivers. But if you plan to run agents in containers, you must install these modules separately.

CloudBees provides an EFS and LOFS installer. It compiles the efs100 and lofs kernel modules if needed, tests them, and loads them into the kernel to provide the drivers. These drivers must have the same build version as your agents.

To install the EFS and LOFS kernel modules:

Log in as root.
Make sure that gcc, make, and kernel headers are installed.

These are required for compiling the kernel modules. For details about installing kernel headers, see Linux Prerequisites.
Download the file system installer. File system installers are available on the CloudBees downloads website at http://downloads.cloudbees.com/cloudbees-build-acceleration/. Navigate to the desired version and then click the installer filename to download.
Double-click the CloudBeesAcceleratorFileSystem-<version>.<buildnumber>-Linux-x86_64-Install installer file.
Follow the prompts to complete the installation.

The installation packages might need a few minutes to extract.
Enter /etc/init.d/ecfs status and make sure that the EFS and LOFS kernel modules are running.

Building your Docker image from a tarball

To build a Docker image for the Cluster Manager, eMake, or agent from a tarball:

Download the Docker image tarball. Docker image tarballs are available on the CloudBees downloads website at http://downloads.cloudbees.com/cloudbees-build-acceleration/. Navigate to the desired version and then click the desired tarball (one for eMake, one for the Cluster Manager, and one for agents) filename(s) to download.

Unzip and untar the file by entering:

tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-Container<component>.tar.gz

where <component> is Emake, ClusterManager, or Agent.

For example:

tar xvzf CloudBeesAccelerator-11.2.93434-ContainerAgent.tar.gz

Build a Docker image by entering:

docker build -t <my_image_tag> -f <path_to_Docker_file> <path_to_folder_with_sources_for_image>

or:

docker build --tag=<my_image_tag> --file=<path_to_Docker_file> <path_to_folder_with_sources_for_image>

For example:

docker build -t cm_img -f cm/Dockerfile-cm-ubuntu cm

The following are specific instructions for downloading a tarball for a specific component and building a Docker image on Ubuntu 16.04.

Component Steps to Download Tarball and Build Image

Component	Steps to Download Tarball and Build Image
eMake	Download the `CloudBeesAccelerator-<version>.<buildnumber>-ContainerEmake.tar.gz` file from the CloudBees downloads site as described above. Enter: tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-ContainerEmake.tar.gz The files are extracted to a directory named `emake`. Enter: docker build -t=<my_image_tag> -f=emake/Dockerfile-emake-ubuntu emake or: docker build -t=<my_image_tag> -f=/path/emake/Dockerfile-emake-ubuntu /path/emake
Cluster Manager	Download the `CloudBeesAccelerator-<version>.<buildnumber>-ContainerClusterManager.tar.gz` file from the CloudBees downloads site as described above. Enter: tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-ContainerClusterManager.tar.gz The files are extracted to a directory named `cm`. Enter: docker build -t=<my_image_tag> -f=cm/Dockerfile-cm-ubuntu cm or: docker build -t=<my_image_tag> -f=/path/cm/Dockerfile-cm-ubuntu /path/cm
Agent	Download the `CloudBeesAccelerator-<version>.<buildnumber>-ContainerAgent.tar.gz` file from the CloudBees downloads site as described above. Enter: tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-ContainerAgent.tar.gz file The files are extracted to a directory named `agent`. Enter: docker build -t=<my_image_tag> -f=agent/Dockerfile-agent-ubuntu agent or: docker build -t=<my_image_tag> -f=/path/agent/Dockerfile-agent-Ubuntu /path/agent

eMake

Download the CloudBeesAccelerator-<version>.<buildnumber>-ContainerEmake.tar.gz file from the CloudBees downloads site as described above.

Enter:

tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-ContainerEmake.tar.gz

The files are extracted to a directory named emake.

Enter:

docker build -t=<my_image_tag> -f=emake/Dockerfile-emake-ubuntu emake

or:

  docker build -t=<my_image_tag> -f=/path/emake/Dockerfile-emake-ubuntu /path/emake

Cluster Manager

Download the CloudBeesAccelerator-<version>.<buildnumber>-ContainerClusterManager.tar.gz file from the CloudBees downloads site as described above.

Enter:

tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-ContainerClusterManager.tar.gz

The files are extracted to a directory named cm.

Enter:

     docker build -t=<my_image_tag> -f=cm/Dockerfile-cm-ubuntu cm

or:

     docker build -t=<my_image_tag> -f=/path/cm/Dockerfile-cm-ubuntu /path/cm

Agent

Download the CloudBeesAccelerator-<version>.<buildnumber>-ContainerAgent.tar.gz file from the CloudBees downloads site as described above.

Enter:

tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-ContainerAgent.tar.gz file

The files are extracted to a directory named agent.

Enter:

    docker build -t=<my_image_tag> -f=agent/Dockerfile-agent-ubuntu agent

or:

    docker build -t=<my_image_tag> -f=/path/agent/Dockerfile-agent-Ubuntu /path/agent

The following are specific examples for downloading a tarball and building a Docker image for a specific component on Red Hat Enterprise Linux 7.x.

Component Steps to Download Tarball and Build Image

Component	Steps to Download Tarball and Build Image
eMake	Download the `CloudBeesAccelerator-<version>.<buildnumber>-ContainerEmake.tar.gz` file from the CloudBees downloads site as described above. Enter: tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-ContainerEmake.tar.gz The files are extracted to a directory named `emake`. Enter: docker build -t=<my_image_tag> -f=emake/Dockerfile-emake-rh emake or: + docker build -t=<my_image_tag> -f=/path/emake/Dockerfile-emake-rh /path/emake
Cluster Manager	Download the `CloudBeesAccelerator-<version>.<buildnumber>-ContainerClusterManager.tar.gz` file from the CloudBees downloads site as described above. Enter: tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-ContainerClusterManager.tar.gz The files are extracted to a directory named `cm`. Enter: docker build -t=<my_image_tag> -f=cm/Dockerfile-cm-rh_cm or: docker build -t=<my_image_tag> -f=/path/cm/Dockerfile-cm-rh /path/cm
Agent	Download the `CloudBeesAccelerator-<version>.<buildnumber>-ContainerAgent.tar.gz` file from the CloudBees downloads site as described above. Enter: tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-ContainerAgent.tar.gz The files are extracted to a directory named `agent`. Enter: docker build -t=<my_image_tag> -f=agent/Dockerfile-agent-rh agent or: docker build -t=<my_image_tag> -f=/path/agent/Dockerfile-agent-rh /path/agent

eMake

Download the CloudBeesAccelerator-<version>.<buildnumber>-ContainerEmake.tar.gz file from the CloudBees downloads site as described above.

Enter:

     tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-ContainerEmake.tar.gz

The files are extracted to a directory named emake.

Enter:

     docker build -t=<my_image_tag> -f=emake/Dockerfile-emake-rh emake

or:

     docker build -t=<my_image_tag> -f=/path/emake/Dockerfile-emake-rh /path/emake

Cluster Manager

Download the CloudBeesAccelerator-<version>.<buildnumber>-ContainerClusterManager.tar.gz file from the CloudBees downloads site as described above.

Enter:

tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-ContainerClusterManager.tar.gz

The files are extracted to a directory named cm.

Enter:

     docker build -t=<my_image_tag> -f=cm/Dockerfile-cm-rh_cm

or:

     docker build -t=<my_image_tag> -f=/path/cm/Dockerfile-cm-rh /path/cm

Agent

Download the CloudBeesAccelerator-<version>.<buildnumber>-ContainerAgent.tar.gz file from the CloudBees downloads site as described above.

Enter:

tar xvzf CloudBeesAccelerator-<version>.<buildnumber>-ContainerAgent.tar.gz

The files are extracted to a directory named agent.

Enter:

     docker build -t=<my_image_tag> -f=agent/Dockerfile-agent-rh agent

or:

     docker build -t=<my_image_tag> -f=/path/agent/Dockerfile-agent-rh /path/agent

Running the Docker Image to Create a Container and Start eMake, the Cluster Manager, or Agents

Component Steps to Run a Docker Image to Create a Container and Start the Component

Component	Steps to Run a Docker Image to Create a Container and Start the Component
eMake	To run a Docker image to create a container and start eMake, enter: docker run -itd --network=host -v <bind_mount_a_volume>:<bind_mount_a_volume> -w [--name=<container_name>] <emake_image_name> where `<container_name>` is your custom name for the container (optional) and `<emake_image_name>` is the name of the Docker image. For example: docker run -itd --network=host -v /home/dev/prj:/home/dev/prj_-w_/home/dev/prj_--name=emake_container1 emake_10.0_ubuntu_alpha The `-d` option runs the container as a daemon.
Cluster Manager	To run a Docker image to create a container and start the Cluster Manager, enter: docker run -idt -p 80:80 -p 8030:8030 -p 8031:8031 -p 443:443 -p 3306:3306 --network=host [--hostname=<container_host_name>] [--name=<container_name>] <cm_image_name> where `<container_host_name>` is the machine on which the container is running (optional), `<container_name>` is your custom name for the container (optional), and `<cm_image_name>` is the name of the Docker image. Specifying the container hostname or IP address on which the container will run ensures that you can use this hostname or IP address (rather than the container name) when using the `--emake-cm=` eMake option to invoke eMake. For example: docker run -idt -p 80:80 -p 8030:8030 -p 8031:8031 -p 443:443 -p 3306:3306 --network=host --hostname=garfield3 --name=cm_container1 cm_10.0_rh_alpha The `-d` option runs the container as a daemon. The `-p` option specifies which ports to expose.
Agent	To run a Docker image to create a container and start the agents, enter one of the following commands based on your version of CloudBees Build Acceleration. docker run --privileged=true -idt -e CM_HOST_AND_PORT=<host>:<port> -e AGENT_RESOURCE=linux [--hostname=<container_hostname>] --device /dev/efs --network=host [--name=<container_name>] <my_image_tag> where `<host>:<port>` is the IP address and port of the Cluster Manager, `--hostname=<container_hostname>` is the user-defined name for the host on which the container is running (optional), `<container_name>` is your custom name for the container (optional), and `<image_name>` is the name of the Docker image. Note: The `-e AGENT_RESOURCE=linux` option is not needed for CloudBees Build Acceleration 11.2 and newer versions. Following is an example: docker run --privileged=true -itd -e CM_HOST_AND_PORT=192.168.1.97 -e AGENT_RESOURCE=linux --hostname=chester-rh --device /dev/efs --network=host --name=agent_rh_container agent_rh You should use the `--hostname=<container_hostname>` option to ensure that the agents have names that are Cluster Manager “resource-friendly.” This lets you create resource definitions to restrict the selection of agents to only those agents running in a container of the desired type. For example, set the container hostname to the base hostname plus an addendum describing the type of container, such as `chester-rh` or `chester-ub1704`. Then the agents would have names such as `chester-rh-1` so that you could define a Cluster Manager resource as `-rh` to match all the Red Hat containers. To check that the agents started, go to the Cluster Manager web interface, then click the Agents* tab, and then make sure that the agents are running.

eMake

To run a Docker image to create a container and start eMake, enter:

docker run -itd --network=host -v <bind_mount_a_volume>:<bind_mount_a_volume> -w  [--name=<container_name>] <emake_image_name>

where <container_name> is your custom name for the container (optional) and <emake_image_name> is the name of the Docker image. For example:

docker run -itd --network=host -v /home/dev/prj:/home/dev/prj_-w_/home/dev/prj_--name=emake_container1 emake_10.0_ubuntu_alpha

The -d option runs the container as a daemon.

Cluster Manager

To run a Docker image to create a container and start the Cluster Manager, enter:

docker run -idt -p 80:80 -p 8030:8030 -p 8031:8031 -p 443:443 -p 3306:3306 --network=host [--hostname=<container_host_name>] [--name=<container_name>] <cm_image_name>

where <container_host_name> is the machine on which the container is running (optional), <container_name> is your custom name for the container (optional), and <cm_image_name> is the name of the Docker image. Specifying the container hostname or IP address on which the container will run ensures that you can use this hostname or IP address (rather than the container name) when using the --emake-cm= eMake option to invoke eMake.

For example:

docker run -idt -p 80:80 -p 8030:8030 -p 8031:8031 -p 443:443 -p 3306:3306 --network=host --hostname=garfield3 --name=cm_container1 cm_10.0_rh_alpha

The -d option runs the container as a daemon. The -p option specifies which ports to expose.

Agent

To run a Docker image to create a container and start the agents, enter one of the following commands based on your version of CloudBees Build Acceleration.

docker run --privileged=true -idt -e CM_HOST_AND_PORT=<host>:<port> -e AGENT_RESOURCE=linux [--hostname=<container_hostname>] --device /dev/efs --network=host [--name=<container_name>] <my_image_tag>

where <host>:<port> is the IP address and port of the Cluster Manager, --hostname=<container_hostname> is the user-defined name for the host on which the container is running (optional), <container_name> is your custom name for the container (optional), and <image_name> is the name of the Docker image.

Note: The -e AGENT_RESOURCE=linux option is not needed for CloudBees Build Acceleration 11.2 and newer versions.

Following is an example:

docker run --privileged=true -itd -e CM_HOST_AND_PORT=192.168.1.97 -e AGENT_RESOURCE=linux --hostname=chester-rh --device /dev/efs --network=host --name=agent_rh_container agent_rh

You should use the --hostname=<container_hostname> option to ensure that the agents have names that are Cluster Manager “resource-friendly.” This lets you create resource definitions to restrict the selection of agents to only those agents running in a container of the desired type. For example, set the container hostname to the base hostname plus an addendum describing the type of container, such as chester-rh or chester-ub1704. Then the agents would have names such as chester-rh-1 so that you could define a Cluster Manager resource as *-rh to match all the Red Hat containers.

To check that the agents started, go to the Cluster Manager web interface, then click the Agents tab, and then make sure that the agents are running.