Configuring the CloudBees Analytics server

Architecture

5 minute read

CloudBees CD/RO supports enterprise-scale software production. Based on a three-tier architecture, CloudBees CD/RO scales to handle large, complex environments. CloudBees CD/RO’s multithreaded Java server provides efficient synchronization even under high job volume.

Local Configuration

The following diagram shows a CloudBees CD/RO architecture configuration at a single site.

Local deployment

new commander arch

In the local configuration:

  • The CloudBees CD/RO server manages resources, issues commands, generates reports.

  • An underlying database stores commands and metadata.

  • Agents execute commands, monitor status, and collect results, in parallel across a cluster of servers for rapid throughput.

Remote Database Configuration

For a production environment, CloudBees recommends that you install the database on a separate machine from the CloudBees CD/RO server to prevent performance issues. It is acceptable for the CloudBees CD/RO server, web server, and repository server to reside on the same machine in a local configuration, but not required. If you are only evaluating CloudBees CD/RO, CloudBees CD/RO, the database, the CloudBees CD/RO server, the web server, and the repository server can reside on the same machine.

Remote CloudBees Analytics Server Configuration

For a production environment, CloudBees recommends that you install the CloudBees Analytics server on a system other than systems running other CloudBees CD/RO components (such as the CloudBees CD/RO server, web server, repository server, or agent). If you must install it on the same system (such as for testing or other nonproduction or trial-basis situations only), see Running on a system with CloudBees CD/RO components for instructions.

Remote Web Server Configuration

The following diagram shows an example of a remote web server architecture configuration.

commander remote setup

In this example remote web server configuration:

  • There are web servers at each site

  • The database and CloudBees CD/RO server is located at your headquarters

  • Proxy resources exist at each site

Benefits of a Remote Web Server Configuration

A remote web server configuration helps prevent network latency. If you have multiple sites, CloudBees CD/RO can be configured in numerous ways to help you work more efficiently.

Central Web Server and a Remote Web Server at Each Site

You should consider installing multiple web servers for different locations in your organization to help handle user web traffic . CloudBees CD/RO supports multiple workspaces, including those co-located on agents that use them. In this architecture, step log files are created locally so even the largest log files can be captured without a performance penalty.

You can view the step log files remotely from the web UI, but performance decreases if the files must be retrieved across the WAN. This means that remote users will experience the penalty when the web server retrieves the step log file contents and when the contents are sent back across the WAN to the browser.

To minimize these performance issues, install one central CloudBees CD/RO server, and then install a CloudBees CD/RO web server at each remote site. The remote web servers should be co-located with the remote agents and workspaces so remote users can log in through their local web server. Any operations initiated from the remote location, including running jobs, are completed by the central CloudBees CD/RO server.

In this configuration, job data is retrieved from the central server when a remote user views the Job Details page. If the job is using a workspace at the remote user’s site, the links to all step log files will refer to local paths.

Also, in this configuration, the log files are accessed only by the remote web server’s agent and not the CloudBees CD/RO server. This eliminates both trips across the WAN, which improves performance. The CloudBees CD/RO web server reads the log file locally (via its local agent) and then displays the page to the user whose browser is also on the same side of the WAN.

Prerequisites for Installing Remote Web Servers

For details about the remote web server prerequisites such as memory, agents, and centralized plugin directory access, see Remote Web Server Installation Prerequisites.

Clustered Configuration

The following diagram shows a CloudBees CD/RO clustered configuration.

Figure 1. Clustered configuration
Figure 1. Clustered configuration

You can also add horizontal scalability and high availability to your CloudBees CD/RO environment by adding additional machines to create a clustered CloudBees CD/RO configuration.

Benefits of a Clustered Configuration

A clustered CloudBees CD/RO configuration has the following benefits:

  • Add fault tolerance by re-routing jobs to running CloudBees CD/RO servers

  • Increase the supported number of simultaneous jobs and corresponding API requests

  • Expand capacity over time by adding additional CloudBees CD/RO servers

  • Distribute API requests across multiple CloudBees CD/RO servers

  • Distribute CloudBees CD/RO requests across multiple web servers

Required Additional Software Components for Clustered Machines

A clustered CloudBees CD/RO configuration requires two additional software components:

  • A centralized service for maintaining and synchronizing group services in cluster

  • A load balancer for routing work to machines in the cluster

Plugins Directory Accessibility Requirement for Clustered Machines

CloudBees strongly recommends that all server machines in a clustered server configuration be able to access a common plugins directory. This avoids the overhead of managing multiple plugins directories. For details, refer to Configuring Universal Access for a Network Location.

Refer to Clustering for additional details and clustered configuration set up procedures.

CloudBees Analytics architecture

With CloudBees Analytics, you can track your workload changes and visualize how efficient, clean, and secure your setup is. Through CloudBees Analytics dashboards, enterprises have the ability to detect:

  • Which CloudBees CI controllers are unhealthy or idle.

  • Where CloudBees CI plugins are used, and their support level.

  • Builds in progress that are deteriorating the quality of the service, and more.

While CloudBees Analytics provides several built-in dashboards and report templates, you can create custom reports and dashboards to visualize your own custom data or to export this data using APIs.

CloudBees Analytics architecture for CloudBees CD/RO

The architecture for CloudBees Analytics with the CloudBees CD/RO capability is as follows:

CloudBees Analytics architecture for CloudBees CD/RO
Figure 2. CloudBees Analytics architecture for CloudBees CD/RO
Where:

1

The CloudBees Analytics server collects data from the following sources:

  • Backend: The CloudBees CD/RO server backend collects build, deployment, pipeline, and release data.

  • Plugin: The set of plugins bundled with CloudBees CD/RO integrates with third-party DevOps tools to collect defect, feature, incident, and quality data.

  • API: The sendReportingData API allows you to inject your own custom data into the Elasticsearch database.

Refer to Sending data from an external source for further information.

2

Collected data is stored in the Elasticsearch database. Refer to Elasticsearch basics for information on accessing the Elasticsearch database directly.

3, 4

The report object definition, together with dashboard filters, defines the data set to extract from the Elasticsearch database. Refer to Report object type reference for available report object definitions bundled with CloudBees CD/RO.

5

The report engine sends the processed data the dashboard widget for rendering. Several report definitions are provided to support built-in dashboards. Additionally, refer to Creating a report and CloudBees Analytics custom dashboards to create custom reports and dashboards.

CloudBees Analytics architecture for CI data

The architecture for CloudBees Analytics for CloudBees CI data is as follows:

CloudBees Analytics architecture for CI data
Figure 3. CloudBees Analytics architecture for CI data
Where:

1

CloudBees CI sends build and other data to the CloudBees Software Delivery Automation server. In turn, it is stored into the CloudBees Analytics server Elasticsearch database where it becomes available for ingestion by reports and dashboards. Refer to Sending data from an external source for further information.

2

Collected data is stored in the Elasticsearch database. Refer to Elasticsearch basics for information on accessing the Elasticsearch database directly.

3, 4

The report object definition, together with dashboard filters, defines the data set to extract from the Elasticsearch database. Refer to Report object type reference for available report object definitions bundled with CloudBees CD/RO.

Glossary

  • Dashboard: Top-level, multi-widget view with related visualizations of interest. Several built-in dashboards are included. Refer to CloudBees Analytics built-in dashboards for a list of pre-configured dashboards bundled with CloudBees CD/RO.

  • Widget: A dashboard tile for UI visualization of report metrics. Refer to Managing widgets for further information.

  • Visualization: Specification of data to display in the UI.

  • Report: The definition of the set of results from which a widget is built. Contains the ElasticSearch report definition. Refer to Elasticsearch basics for further information.

  • Reporting object type: The type of object on which to report. CloudBees CD/RO provides several built-in report object types. Refer to Understanding the CloudBees Analytics data model for available report object definitions.

  • Reporting object attribute: Specific metadata to report on and visualize.