Configuring CloudBees Build Acceleration to Run in Docker Containers

CloudBees Build Acceleration includes an integration to support Docker on Linux platforms. This section describes how to configure a containerized Cluster Manager, eMake, or agents.

Supported Use Cases

CloudBees Build Acceleration supports running a single agent per container and multiple containers per host.

Installing Docker

To use Docker containers, you must download and install Docker. For instructions, see Docker Enterprise Edition or Docker Community Edition.

(Agent Installations Only) Installing EFS and LOFS

CloudBees Build Acceleration normally uses its own kernel modules to provide two file systems (EFS and LOFS) required by agents. During a non-container agent installation, these kernel modules are compiled and loaded into the kernel to provide the necessary file system drivers. But if you plan to run agents in containers, you must install these modules separately.

CloudBees provides an EFS and LOFS installer. It compiles the efs100 and lofs kernel modules if needed, tests them, and loads them into the kernel to provide the drivers. These drivers must have the same build version as your agents.

To install the EFS and LOFS kernel modules:

  1. Log in as root.

  2. Make sure that gcc, make, and kernel headers are installed.

    These are required for compiling the kernel modules. For details about installing kernel headers, see Linux Prerequisites.

  3. Double-click the CloudBeesAcceleratorFileSystem-<version>.<build_number>-Linux-x86_64-Install installer file.

    This is inside the “containers” folder for the version on ShareFile:

    ./products/accelerator/release_<version>/<version>.<build_number>/linux/containers/

    For example:

    ./products/accelerator/release_11.0/11.0.0.84351/linux/containers/CloudBeesAcceleratorFileSystem-11.0.0.84351-Linux-x86_64-Install

  4. Follow the prompts to complete the installation.

    The installation packages might need a few minutes to extract.

  5. Enter /etc/init.d/ecfs status and make sure that the EFS and LOFS kernel modules are running.

Building Your Docker Image from a Tar File

To build a Docker image for the Cluster Manager, eMake, or agent from a tar file:

  1. Log in (using your personalized account) to the CloudBees ShareFile site at https://cloudbees.sharefile.com/Authentication/Login.

  2. Navigate to the following folder:

    ./products/accelerator/release_<version>/<version>.<build_number>/linux/containers/

    For example, for CloudBees Build Acceleration version 11.2, navigate to

    ./products/accelerator/release_11_2/11.2.0.93434/linux/containers/

  3. Click to download the tar file for eMake, the Cluster Manager, or agents, and then click Download.

  4. Unzip and untar the file by entering:

    tar xvzf CloudBeesAccelerator-<version>.<build_number>-Container<component>.tar.gz

    where  <component> is Emake, ClusterManager, or Agent.

    For example:

    tar xvzf CloudBeesAccelerator-11.2.93434-ContainerAgent.tar.gz
  5. Build a Docker image by entering:

    docker build -t <my_image_tag> -f <path_to_Docker_file> <path_to_folder_with_sources_for_image>

    or:

    docker build --tag=<my_image_tag> --file=<path_to_Docker_file> <path_to_folder_with_sources_for_image>

    For example:

    docker build -t cm_img -f cm/Dockerfile-cm-ubuntu cm

The following are specific instructions for downloading a tar file for a specific component and building a Docker image on Ubuntu 16.x.

Component Steps to Download Tar File and Build Image

eMake

  1. Download the CloudBeesAccelerator-<version>.<build_number>-ContainerEmake.tar.gz file from the CloudBees ShareFile folder as described above.

  2. Enter:

    tar xvzf CloudBeesAccelerator-<version>.<build_number>-ContainerEmake.tar.gz

    The files are extracted to a directory named emake.

  3. Enter:

    docker build -t=<my_image_tag> -f=emake/Dockerfile-emake-ubuntu emake

    or:

      docker build -t=<my_image_tag> -f=/path/emake/Dockerfile-emake-ubuntu /path/emake

Cluster Manager

  1. Download the CloudBeesAccelerator-<version>.<build_number>-ContainerClusterManager.tar.gz file from the CloudBees ShareFile folder as described above.

  2. Enter:

    tar xvzf CloudBeesAccelerator-<version>.<build_number>-ContainerClusterManager.tar.gz

    The files are extracted to a directory named cm.

  3. Enter:

         docker build -t=<my_image_tag> -f=cm/Dockerfile-cm-ubuntu cm

    or:

         docker build -t=<my_image_tag> -f=/path/cm/Dockerfile-cm-ubuntu /path/cm

Agent

  1. Download the CloudBeesAccelerator-<version>.<build_number>-ContainerAgent.tar.gz file from the CloudBees ShareFile folder as described above.

  2. Enter:

    tar xvzf CloudBeesAccelerator-<version>.<build_number>-ContainerAgent.tar.gz file

    The files are extracted to a directory named agent.

  3. Enter:

        docker build -t=<my_image_tag> -f=agent/Dockerfile-agent-ubuntu agent

    or:

        docker build -t=<my_image_tag> -f=/path/agent/Dockerfile-agent-Ubuntu /path/agent

The following are specific examples for downloading a tar file and building a Docker image for a specific component on Red Hat Enterprise Linux 7.x.

Component Steps to Download Tar File and Build Image

eMake

  1. Download the CloudBeesAccelerator-<version>.<build_number>-ContainerEmake.tar.gz file from the CloudBees ShareFile folder as described above.

  2. Enter:

         tar xvzf CloudBeesAccelerator-<version>.<build_number>-ContainerEmake.tar.gz

    The files are extracted to a directory named emake.

  3. Enter:

         docker build -t=<my_image_tag> -f=emake/Dockerfile-emake-rh emake

or:

+

     docker build -t=<my_image_tag> -f=/path/emake/Dockerfile-emake-rh /path/emake

Cluster Manager

  1. Download the CloudBeesAccelerator-<version>.<build_number>-ContainerClusterManager.tar.gz file from the CloudBees ShareFile folder as described above.

  2. Enter:

    tar xvzf CloudBeesAccelerator-<version>.<build_number>-ContainerClusterManager.tar.gz

    The files are extracted to a directory named cm.

  3. Enter:

         docker build -t=<my_image_tag> -f=cm/Dockerfile-cm-rh cm

    or:

         docker build -t=<my_image_tag> -f=/path/cm/Dockerfile-cm-rh /path/cm

Agent

  1. Download the CloudBeesAccelerator-<version>.<build_number>-ContainerAgent.tar.gz file from the CloudBees ShareFile folder as described above.

  2. Enter:

    tar xvzf CloudBeesAccelerator-<version>.<build_number>-ContainerAgent.tar.gz

    The files are extracted to a directory named agent.

  3. Enter:

         docker build -t=<my_image_tag> -f=agent/Dockerfile-agent-rh agent

    or:

         docker build -t=<my_image_tag> -f=/path/agent/Dockerfile-agent-rh /path/agent

Running the Docker Image to Create a Container and Start eMake, the Cluster Manager, or Agents

Component Steps to Run a Docker Image to Create a Container and Start the Component

eMake

To run a Docker image to create a container and start eMake, enter:

docker run -itd -v <bind_mount_a_volume>:<bind_mount_a_volume> -w  [--name=<container_name>] <emake_image_name>

where <container_name> is your custom name for the container (optional), and <emake_image_name> is the name of the Docker image. For example:

docker run -itd -v /home/dev/prj:/home/dev/prj -w /home/dev/prj --name=emake_container1 emake_10.0_ubuntu_alpha

The -d option runs the container as a daemon.

Cluster Manager

To run a Docker image to create a container and start the Cluster Manager, enter:

docker run -idt -p 80:80 -p 8030:8030 -p 8031:8031 -p 443:443 -p 3306:3306 [--hostname=<container_host_name>] [--name=<container_name>] <cm_image_name>

where <container_host_name> is the machine on which the container is running (optional), <container_name> is your custom name for the container (optional), and <cm_image_name> is the name of the Docker image. Specifying the container hostname or IP address on which the container will run ensures that you can use this hostname or IP address (rather than the container name) when using the --emake-cm= eMake option to invoke eMake.

For example:

docker run -idt -p 80:80 -p 8030:8030 -p 8031:8031 -p 443:443 -p 3306:3306 --hostname=garfield3 -name=cm_container1 cm_10.0_rh_alpha

The -d option runs the container as a daemon. The -p option specifies which ports to expose.

Agent

To run a Docker image to create a container and start the agents, enter one of the following commands based on your version of CloudBees Build Acceleration.

docker run --privileged=true -idt -e CM_HOST_AND_PORT=<host>:<port> -e AGENT_RESOURCE=linux [--hostname=<container_hostname>] --device /dev/efs --net=host [--name=<container_name>] <my_image_tag>

where <host>:<port> is the IP address and port of the Cluster Manager, --hostname=<container_hostname> is the user-defined name for the host on which the container is running (optional), --name=<container_name> is your custom name for the container (optional), and <image_name> is the name of the Docker image.

Note: The -e AGENT_RESOURCE=linux option is not needed for CloudBees Build Acceleration 11.2 and newer versions.

Following is an example:

docker run --privileged=true -itd -e CM_HOST_AND_PORT=192.168.1.97 -e AGENT_RESOURCE=linux --hostname=chester-rh --device /dev/efs --net=host --name=agent_rh_container agent_rh

You should use the --hostname=<container_hostname> option to ensure that the agents have names that are Cluster Manager “resource-friendly.” This lets you create resource definitions to restrict the selection of agents to only those agents running in a container of the desired type. For example, set the container hostname to the base hostname plus an addendum describing the type of container, such as chester-rh or chester-ub1704.  Then the agents would have names such as chester-rh-1 so that you could define a Cluster Manager resource as *-rh to match all the Red Hat containers.

To check that the agents started, go to the Cluster Manager web interface, then click the Agents tab, and then make sure that the agents are running.