CloudBees CoreCloudBees DevOpticsCloudBees FlowCloudBees AcceleratorCloudBees RolloutCloudBees Jenkins DistributionCloudBees Jenkins X DistributionCloudBees SDMCloudBees Jenkins PlatformCloudBees Jenkins EnterpriseCloudBees LexiconRelease notes
Introduction
What’s new
Installation on Amazon Web Service (AWS) guide
Administration guide
Generating a support bundle
Backup and restore guide
Plugin guides
Pipelines guides
Troubleshooting guides
Miscellaneous references
Amazon Web Services CLI Plugin
Microsoft Azure CLI Plugin
Cloud Foundry CLI Plugin
OpenShift CLI Plugin
CLI reference
Configuration reference
Elasticsearch
Infrastructure security
CloudBees ServiceNow Integration
Support policies

Elasticsearch

The Elasticsearch component provides a repository for various types of CloudBees Jenkins Enterprise data, such as raw metrics, job-related information, and logs. This section provides information about the Elasticsearch component in CloudBees Jenkins Enterprise and the indices of data being persisted into it.

Data Types

CloudBees Jenkins Enterprise collects some internal log data in Elasticsearch, detailed below.

Table 1. CloudBees Jenkins Enterprise-internal data collected
Name Pattern Retention Period Contents Disk usage expectation

topbeat-*

3 days

Performance data for the controller/worker nodes

70MB/node/day

logstash-*

7 days

syslog entries for CloudBees Jenkins Enterprise infrastructure components

800MB/day for the following system: three-node controller, three-node Elasticsearch, one node for master, one node for builds, 2000 jobs/day.

Analytics data are stored in Elasticsearch using time-stamped index names. These are retained at various intervals, after which the index for that day is deleted automatically by Analytics. Future versions of Analytics will allow you to configure the retention interval of each index type. The indexes and their retention period are listed below.

Table 2. Index Purpose and Retention Period
Name Pattern Retention Period Contents Disk usage expectation

metrics-*

3 days

Metric reports from Jenkins masters, which are submitted every 10 seconds.

450 mb (stable) per connected master

metrics-hourly-*

3 years

Hourly summary of the metrics reports for each master, based on the metrics-* index.

210mb per connected master, per year

builds-*

3 years

Builds reported for indexing by Client Masters. Contains minimal data fields needed for reporting. Does not include console logs or artifacts. Starting from CloudBees Jenkins Analytics 1.8.100 does not include detailed info about nodes, this data has been moved to the nodes-* index.

2k per build

nodes-*

3 years

Node configuration changes reported for indexing by Client Masters. Event examples: node config save, creation/deletion of nodes. Such data will be recorded for all node types including ephemeral and cloud-provisioned nodes.

1k per configuration change

items

Forever

Minimal information about each job and folder.

1k per job or folder

kibana-4-cloudbees

Forever

Dashboard and panel definitions.

30k per dashboard

Deleting a build in Jenkins does not result in the build being deleted from Analytics, because this would diminish the accuracy and value of the Build Analytics feature. Instead, builds will be 'forgotten' 3 years after they were last reported — when the datestamped build-* for that day is deleted.

Note that "Reindexing" resets the lifespan of a build. So Jenkins still has a build from 5 years ago and you Reindex that Jenkins, the build will be retained in the builds-* index for another three years, regardless of its lifespan in Jenkins itself.

Default Elasticsearch Configuration

Configuration Attribute Default Value Description

memory

2048

Total amount of memory allocated for Elasticsearch container (2000MB). Elasticsearch, itself, gets 1024MB of heap space

max_instance_count

1

Number of Elasticsearch node

replicas

1

Number of replica shard to keep

Recommended Configuration for Medium Size Deployment

The following recommendation is for a CloudBees Jenkins Enterprise deployment that runs about 2000 jobs a day, with three Jenkins masters and five build worker nodes. Elasticsearch is to run on three dedicated worker nodes. For AWS, it is of instance type "m4.xlarge", which has 16GB of RAM, 4 vCPUs and 200G of disk space. If OpenStack is the platform, then use the equivalent flavor.

Configuration Attribute Default Value Description

memory

14000

Total amount of memory allocated for Elasticsearch container (14000MB). Elasticsearch, itself, gets 7000MB of heap space

max_instance_count

3

Number of Elasticsearch node

replicas

2

Number of shard replicas to keep

shards

5

Number of shard per index

Changing Elasticsearch Configuration

This is typically done when the user wants to re-configure Elasticsearch, e.g, changing from the out-of-the-box configuration to one than can handle a medium size deployment described above.

Please use the following sequence of actions:

  1. Add three dedicated Elasticsearch nodes by running the worker-add operation

  2. Backup Elasticsearch

  3. Ensure that the backup has completed by querying Elasticsearch using the following URL http://<CJE-url>/elasticsearch/_snapshot/_status.

  4. Execute the elasticsearch-update operation. For the medium size deployment recommendation, the contents of the file elasticsearch-update.config should be as follows:

    elasticsearch-update.config
    [elasticsearch]
max_instance_count = 3
memory = 14000
shards = 5
replicas = 2
use_dedicated_worker = true

  5. Restore Elasticsearch indices from the backup in step 2

  6. Restart Operations Center by executing the following commands in the CloudBees Jenkins Enterprise project directory

    dna stop cjoc
dna start cjoc

CloudBees Jenkins Enterprise Elasticsearch Operations

The following sections are specific to those operations that affect Elasticsearch running in the CloudBees Jenkins Enterprise environment.

Backing Up Elasticsearch

elasticsearch-backup

This operation initiates a back-up action for all Elasticsearch indices into a repository called "tiger-backup". This operation is invoked asynchronously, i.e., it returns immediately before the back-up action has finished.

To check the back-up action status, reference the following Elasticsearch API end-point http://<CJE-url>/elasticsearch/_snapshot/tiger-backup/_status.

Restoring Elasticsearch

elasticsearch-restore

This operation is the reverse of the elasticsearch-backup operation. It is also an asynchronous operation and the same API end-point can be used to check for its completion status.

Deleting Elasticsearch Indices

elasticsearch-indices-delete

This operations removes older logstash-* and topbeat-* indices. By default, it will remove data older than three days. However, the user can customize the cutoff point, by modifying the file "elasticsearch-indices-delete.config".

[elasticsearch]

## The number of days worth of elasticsearch indices to keep.
#days_to_keep = 3
Configuration reference
Infrastructure security