Upgrade Kubernetes CloudBees Analytics environments to OpenSearch

In CloudBees CD/RO v2024.06.0, CloudBees Analytics was upgraded from using Elasticsearch to OpenSearch. The data formats of these search engines are not fully compatible. To preserve data from your legacy CloudBees Analytics servers, you must migrate it to an updated server when upgrading from a v2024.03.0 or earlier to v2024.06.0 or later.

This page describes the general processes to test and upgrade legacy CloudBees Analytics servers:

Test upgrade process
Production upgrade process

In the following content, these servers are referenced as:

Source server: v2024.03.0 or earlier CloudBees Analytics servers using Elasticsearch.
Destination server: v2024.06.0 or later CloudBees Analytics servers using OpenSearch.

Test upgrade process

CloudBees strongly recommends to test migrating your data from Elasticsearch to OpenSearch on a VM before attempting it in your production environment.

This approach helps to:

Identify and resolve potential issues with data migration.
Accurately estimate the duration of the migration process.
Develop and refine troubleshooting steps for production upgrades.
Minimize or avoid potential system downtime in your production environment.

To test the migration:

Review the CloudBees Analytics Elasticsearch to OpenSearch migration known issues.
Perform a clean installation of a CloudBees Software Delivery Automation server on a VM.

Perform an installation of v2024.06.0 or later CloudBees Analytics server within the same VM as your CloudBees Software Delivery Automation installation.

You can download the latest installer from the CloudBees downloads site.

The following information is critical for this process:

When launching the CloudBees Analytics installer, include the URL of the source remote server, as described in Specify remote data source servers.
When performing the CloudBees Software Delivery Automation server installation, do not choose the option to specify a remote CloudBees Software Delivery Automation server to interact with the CloudBees Analytics server when installing the destination CloudBees Analytics server.

Migrate the data from the source CloudBees Analytics server to the destination server by following the instructions on Migrate data with CloudBees CD/RO procedure.

Performing a backup of the source data is not required when performing a test run. The source system and its data are still available for normal use during and after the test migration.
To configure the CloudBees Analytics server on the CloudBees Software Delivery Automation server, run:
```
ectool setAnalyticsServerConfiguration --enabled 1▼
```
Log in to your new instance of CloudBees CD/RO.
Verify your CloudBees Analytics data is available and up to date in your upgraded CloudBees CD/RO instance.
1. Navigate to Analytics Dashboards, and verify your dashboards are able to populate accurate data.
2. Navigate to Analytics Reports, and verify your reports are able to populate accurate data.

If you have migrated your data without issues, this VM is no longer needed, and you can proceed to Production upgrade process. If you encounter multiple issues during testing that you cannot troubleshoot, copy the logs from the VM, and contact CloudBees Support.

Production upgrade process

For Kubernetes upgrades from CloudBees Analytics v2024.03.0 or earlier to v2024.06.0 or later, the following steps are required for a successful production upgrade:

Review the CloudBees Analytics Elasticsearch to OpenSearch migration known issues.

Disable the connection between CloudBees Analytics and CloudBees CD/RO servers on your source machine by running:

ectool setDevOpsInsightServerConfiguration --enabled 0▼

For production migrations, once the CloudBees Analytics server is disabled, no data will be available for jobs running in CloudBees CD/RO. Additionally, this data will not be part of the backup created in the next step. If possible, it is suggested for all jobs to complete on the legacy server prior to creating the backup.

Back up the data from your legacy CloudBees Analytics Elasticsearch server to avoid data loss by following the instructions on Maintain CloudBees Analytics server data on Kubernetes.

Failing to back up the source CloudBees Analytics data could result in permanent data loss if issues arise during data migration.
If you have not already done so, add the analytics Helm chart to your values file. For more information, refer to CloudBees Analytics server values.
Configure the analytics Helm chart with any custom values previously used for dois.

To configure CloudBees Analytics credentials or certificates, refer to Update CloudBees Analytics authentication methods.

In your updated values file, set analytics.autoRegister: true to analytics.autoRegister: false.

Setting analytics.autoRegister: false in your values file prevents the CloudBees Analytics server configuration from being created in the CloudBees Software Delivery Automation server. This is critical to prevent unexpected issues while migrating CloudBees Analytics data from Elasticsearch to OpenSearch.

Upgrade your deployment to v2024.06.0 or later. For more information, refer to Upgrade CloudBees CD/RO on Kubernetes.
Migrate your data from the source CloudBees Analytics server to the destination server by following the instructions on Migrate CloudBees Analytics data from Elasticsearch to OpenSearch.

Ensure you review the information in Kubernetes migrations overview before starting. This section contains important information needed during data migration.

Follow the instruction in Enable the CloudBees Analytics configuration for CloudBees Software Delivery Automation to connect the CloudBees Analytics server to CloudBees Software Delivery Automation.

If you previously changed your CloudBees CD/RO server administrator password in the CloudBees CD/RO UI, update the field in your values file before running your helm upgrade command, as described in Enable the CloudBees Analytics configuration for CloudBees Software Delivery Automation.

Verify your CloudBees Analytics data is available and up to date in your upgraded CloudBees CD/RO instance.

Navigate to Analytics Dashboards, and verify your dashboards are able to populate accurate data.
Navigate to Analytics Reports, and verify your reports are able to populate accurate data.

Do not proceed to the next step if your CloudBees Analytics dashboards or reports are not displaying accurate data. Doing so may result in permanent data loss.

To troubleshoot:

Navigate to the Reindex Analytics Data procedure job log. This is described in Data migration log example.
Verify there are no errors reported.
1. If errors are reported, refer to the Known issues for data migration for possible solutions.
2. Do not uninstall the dois service until you are sure your CloudBees Analytics data is accurately being reported in CloudBees CD/RO. Doing so may result in permanent data loss.
If you are unable to get your CloudBees CD/RO instance to display CloudBees Analytics data, contact CloudBees Support.

Uninstall the dois service. For more information, refer to Disable legacy server after migration.

If you have not already done so, it is critical to complete Enable the CloudBees Analytics configuration for CloudBees Software Delivery Automation and to ensure your CloudBees Analytics is being accurately reported in CloudBees CD/RO (as described in previous step) before attempting to uninstall the dois service. Failing to do so may result in permanent data loss.

Update CloudBees Analytics authentication methods

Before deploying your v2024.06.0 or later CloudBees Analytics Helm chart, you must configure an authentication method for the analytics service. This is done by configuring either of the following:

Configure credentials for CloudBees Analytics (analytics.credentials)
Configure Certificates for CloudBees Analytics (analytics.certificates)

Configure Certificates for CloudBees Analytics must be used if analytics.replicas is ≥2.

Configure credentials for CloudBees Analytics

This section guides you through configuring an authentication method for analytics.credentials when migrating from dois to analytics. There are several authentication methods that can be used for analytics.credentials.

Refer to the corresponding section in the following list for your preferred method:

Using an existing secret
Using username and password

These methods can only be used if your environment does not use analytics.replicas.

Using an existing secret

This section details how to configure the analytics.credentials.existingSecret when previously using dois.credentials.existingSecret.

The following applicable credentials variables were updated in v2024.06.0:

Previous dois Updated analytics

CBF_DOIS_PASSWORD

CBF_ANALYTICS_PASSWORD

CBF_DOIS_ADMIN_PASSWORD

CBF_ANALYTICS_ADMIN_PASSWORD

To create a new secret for analytics:

Set the following variables:

# Value of dois.credentials.existingSecret
DOIS_CREDS_SECRET=<YOUR-DOIS-CREDENTIALS-SECRET>

# Value of analytics.credentials.existingSecret
ANALYTICS_CREDS_SECRET=cd-flow-analytics

# Your CloudBees CD/RO namespace
NAMESPACE=<YOUR-NAMESPACE>▼

Create the analytics secret, copying the values from dois by running:

kubectl --namespace $NAMESPACE create secret generic $ANALYTICS_CREDS_SECRET \
 --from-literal CBF_ANALYTICS_ADMIN_PASSWORD=$(kubectl get secret $DOIS_CREDS_SECRET -o jsonpath="{.data.CBF_DOIS_ADMIN_PASSWORD}" | base64 --decode) \
 --from-literal CBF_ANALYTICS_PASSWORD=$(kubectl get secret $DOIS_CREDS_SECRET -o jsonpath="{.data.CBF_DOIS_PASSWORD}" | base64 --decode)▼

In your values file, for analytics.credentials.existingSecret, add cd-flow-analytics.

You can now complete your CloudBees CD/RO Helm upgrade.

Using username and password

This section details how to configure analytics.credentials.adminPassword and analytics.credentials.reportUserPassword. For this method, in your values file, copy:

dois.credentials.adminPassword to analytics.credentials.adminPassword.

dois.credentials.reportUserPassword to analytics.credentials.reportUserPassword.

Example

...
dois:
  credentials:
    existingSecret:
    adminPassword: myDoisAdminPassword
    ## If `reportUserPassword` is an empty string, a random 20 characters password is generated.
    reportUserPassword: myDoisReportUserPassword
...
analytics:
  credentials:
    existingSecret:
    adminPassword: myDoisAdminPassword
    ## If `reportUserPassword` is an empty string, a random 20 characters password is generated.
    reportUserPassword: myDoisReportUserPassword
...▼

You can now complete your CloudBees CD/RO Helm upgrade.

Configure Certificates for CloudBees Analytics

This section guides you through configuring an authentication method for analytics.certificates when migrating from dois to analytics. There are several authentication methods that can be used for analytics.certificates.

For values in dois.certificates, if your base64 encoded data keys are used directly in your values files, you can copy the value directly to the corresponding key in analytics.certificates. Once you have copied the key value, you can continue with the upgrade process.

The following sections explain how to configure the corresponding analytics.certificates if you currently use dois.certificates.exisitingSecret. Refer to the corresponding section for your environment:

Using a certificate bundle
Using a CA certificate and key
Using a CA certificate and key with a signature
Using a CA certificate and key with signature and node and admin certificates and keys

Using a certificate bundle

This section details how to configure analytics.certificates.exisitingSecret if you were previously using a certificate bundle for dois.certificates.exisitingSecret:

In v2024.06.0, the CBF_DOIS_CRT_BUNDLE was updated to CBF_ANALYTICS_CRT_BUNDLE.

Set the following variables:

# Value of dois.certificates.exisitingSecret
DOIS_CERTS_SECRET=<YOUR-CBCD-DOIS-CERTIFICATES>

# Value of analytics.certificate.bundle
ANALYTICS_CERTS_SECRET=cbcd-analytics-certificates-new

# Your CloudBees CD/RO namespace
NAMESPACE=<YOUR-NAMESPACE>▼

Get the certificate bundle value from the dois certificate secret

BUNDLE=$(kubectl -n $NAMESPACE get secret $DOIS_CERTS_SECRET -o jsonpath="{.data.CBF_DOIS_CRT_BUNDLE}" | base64 --decode)▼

Ensure $BUNDLE has a value:
```
echo $BUNDLE▼
```

Create a new secret by setting the value of $BUNDLE to CBF_ANALYTICS_CRT_BUNDLE:

kubectl -n $NAMESPACE create secret generic $ANALYTICS_CERTS_SECRET --from-literal=CBF_ANALYTICS_CRT_BUNDLE=$BUNDLE▼

Ensure the new secret has a value for CBF_ANALYTICS_CRT_BUNDLE:

kubectl -n $NAMESPACE describe secret $ANALYTICS_CERTS_SECRET▼

Example output

kubectl -n $NAMESPACE describe secret $ANALYTICS_CERTS_SECRET

Name:         cbcd-analytics-certificates-new
Namespace:    cd
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data:
CBF_ANALYTICS_CRT_BUNDLE:  5252 bytes▼

In your values file, set analytics.certificates.exisitingSecret to:
- cbcd-analytics-certificates-new

You can now complete your CloudBees CD/RO Helm upgrade.

Using a CA certificate and key

This section details how to configure analytics.certificates.exisitingSecret if you were previously using a CA certificate and key for dois.certificates.exisitingSecret:

The following applicable certificates variables were updated in v2024.06.0:

Previous dois Updated analytics

CBF_DOIS_CA_CRT

CBF_ANALYTICS_CA_CRT

CBF_DOIS_CA_KEY

CBF_ANALYTICS_CA_KEY

Set the following variables:

# Value of dois.certificates.exisitingSecret
DOIS_CERTS_SECRET=<YOUR-CBCD-DOIS-CERTIFICATE>

# Value of analytics.certificate.bundle
ANALYTICS_CERTS_SECRET=cbcd-analytics-certificates-new

# Your CloudBees CD/RO namespace
NAMESPACE=<YOUR-NAMESPACE>▼

Get the ca.crt and ca.key values from the dois certificate secret by running:

CA_CRT=$(kubectl -n $NAMESPACE get secret $DOIS_CERTS_SECRET -o jsonpath="{.data.CBF_DOIS_CA_CRT}" | base64 --decode)
CA_KEY=$(kubectl -n $NAMESPACE get secret $DOIS_CERTS_SECRET -o jsonpath="{.data.CBF_DOIS_CA_KEY}" | base64 --decode)▼

Ensure $CA_CRT and $CA_KEY have values by running:
```
echo $CA_CRT
echo $CA_KEY▼
```

Create a new secret by setting the value of $CA_CRT to CBF_ANALYTICS_CA_CRT and $CA_KEY to CBF_ANALYTICS_CA_KEY:

kubectl -n $NAMESPACE create secret generic $ANALYTICS_CERTS_SECRET \
  --from-literal=CBF_ANALYTICS_CA_CRT=$CA_CRT \
  --from-literal=CBF_ANALYTICS_CA_KEY=$CA_KEY▼

Ensure the new secret has values for CBF_ANALYTICS_CA_CRT and CBF_ANALYTICS_CA_KEY:

kubectl -n $NAMESPACE describe secret $ANALYTICS_CERTS_SECRET▼

Example output

Name:         cbcd-analytics-certificates-new
Namespace:    cd
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data:
CBF_ANALYTICS_CA_CRT:     1996 bytes
CBF_ANALYTICS_CA_KEY:     2472 bytes▼

In your values file, set analytics.certificates.exisitingSecret to:
- cbcd-analytics-certificates-new

You can now complete your CloudBees CD/RO Helm upgrade.

Using a CA certificate and key with a signature

This section details how to configure analytics.certificates.exisitingSecret if you were previously using a CA certificate and key with a signature for dois.certificates.exisitingSecret:

The following applicable certificates variables were updated in v2024.06.0:

Previous dois Updated analytics

CBF_DOIS_CA_CRT

CBF_ANALYTICS_CA_CRT

CBF_DOIS_SIGN_CRT

CBF_ANALYTICS_SIGN_CRT

CBF_DOIS_SIGN_KEY

CBF_ANALYTICS_SIGN_KEY

Set the following variables:

# Value of dois.certificates.exisitingSecret
DOIS_CERTS_SECRET=<YOUR-CBCD-DOIS-CERTIFICATE>

# Value of analytics.certificate.bundle
ANALYTICS_CERTS_SECRET=cbcd-analytics-certificates-new

# Your CloudBees CD/RO namespace
NAMESPACE=<YOUR-NAMESPACE>▼

Get the ca.crt and sign.crt and sign.key values from the dois certificate secret by running:

CA_CRT=$(kubectl -n $NAMESPACE get secret $DOIS_CERTS_SECRET -o jsonpath="{.data.CBF_DOIS_CA_CRT}" | base64 --decode)
SIGN_CRT=$(kubectl -n $NAMESPACE get secret $DOIS_CERTS_SECRET -o jsonpath="{.data.CBF_DOIS_SIGN_CRT}" | base64 --decode)
SIGN_KEY=$(kubectl -n $NAMESPACE get secret $DOIS_CERTS_SECRET -o jsonpath="{.data.CBF_DOIS_SIGN_KEY}" | base64 --decode)▼

Ensure $CA_CRT, $SIGN_CRT, and $SIGN_KEY have values:
```
echo $CA_CRT
echo $SIGN_CRT
echo $SIGN_KEY▼
```

Create a new secret by setting the value of

$CA_CRT to CBF_ANALYTICS_CA_CRT
$SIGN_CRT to CBF_ANALYTICS_SIGN_CRT

$SIGN_KEY to CBF_ANALYTICS_SIGN_KEY

kubectl -n $NAMESPACE create secret generic $ANALYTICS_CERTS_SECRET \
  --from-literal=CBF_ANALYTICS_CA_CRT=$CA_CRT \
  --from-literal=CBF_ANALYTICS_SIGN_CRT=$SIGN_CRT \
  --from-literal=CBF_ANALYTICS_SIGN_KEY=$SIGN_KEY▼

Ensure the new secret has values for the data keys:

kubectl -n $NAMESPACE describe secret $ANALYTICS_CERTS_SECRET▼

Example output

Name:         cbcd-analytics-certificates-new
Namespace:    cd
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data:
CBF_ANALYTICS_CA_CRT:     1996 bytes
CBF_ANALYTICS_SIGN_CRT:   2024 bytes
CBF_ANALYTICS_SIGN_KEY:   2472 bytes▼

In your values file, set analytics.certificates.exisitingSecret to:
- cbcd-analytics-certificates-new

You can now complete your CloudBees CD/RO Helm upgrade.

Using a CA certificate and key with signature and node and admin certificates and keys

This section details how to configure analytics.certificates.exisitingSecret if you were previously using a CA certificate and key with a signature, node and admin certificates and keys for dois.certificates.exisitingSecret:

The following applicable certificates variables were updated in v2024.06.0:

Previous dois Updated analytics

CBF_DOIS_CA_CRT

CBF_ANALYTICS_CA_CRT

CBF_DOIS_SIGN_CRT

CBF_ANALYTICS_SIGN_CRT

CBF_DOIS_NODE_CRT

CBF_ANALYTICS_NODE_CRT

CBF_DOIS_NODE_KEY

CBF_ANALYTICS_NODE_KEY

CBF_DOIS_ADMIN_CRT

CBF_ANALYTICS_ADMIN_CRT

CBF_DOIS_ADMIN_KEY

CBF_ANALYTICS_ADMIN_KEY

Set the following variables:

# Value of dois.certificates.exisitingSecret
DOIS_CERTS_SECRET=<YOUR-CBCD-DOIS-CERTIFICATES>

# Value of analytics.certificate.bundle
ANALYTICS_CERTS_SECRET=cbcd-analytics-certificates-new

# Your CloudBees CD/RO namespace
NAMESPACE=<YOUR-NAMESPACE>▼

Get certificate values from the dois certificate secret:

CA_CRT=$(kubectl -n $NAMESPACE get secret $DOIS_CERTS_SECRET -o jsonpath="{.data.CBF_DOIS_CA_CRT}" | base64 --decode)
SIGN_CRT=$(kubectl -n $NAMESPACE get secret $DOIS_CERTS_SECRET -o jsonpath="{.data.CBF_DOIS_SIGN_CRT}" | base64 --decode)
NODE_CRT=$(kubectl -n $NAMESPACE get secret $DOIS_CERTS_SECRET -o jsonpath="{.data.CBF_DOIS_NODE_CRT}" | base64 --decode)
NODE_KEY=$(kubectl -n $NAMESPACE get secret $DOIS_CERTS_SECRET -o jsonpath="{.data.CBF_DOIS_NODE_KEY}" | base64 --decode)
ADMIN_CRT=$(kubectl -n $NAMESPACE get secret $DOIS_CERTS_SECRET -o jsonpath="{.data.CBF_DOIS_ADMIN_CRT}" | base64 --decode)
ADMIN_KEY=$(kubectl -n $NAMESPACE get secret $DOIS_CERTS_SECRET -o jsonpath="{.data.CBF_DOIS_ADMIN_KEY}" | base64 --decode)▼

Ensure the variables have values:

echo $CA_CRT
echo $SIGN_CRT
echo $NODE_CRT
echo $NODE_KEY
echo $ADMIN_CRT
echo $ADMIN_KEY▼

Create a new secret by setting the following values:

$CA_CRT to CBF_ANALYTICS_CA_CRT
$SIGN_CRT to CBF_ANALYTICS_SIGN_CRT
$NODE_CRT to CBF_ANALYTICS_NODE_CRT
$NODE_KEY to CBF_ANALYTICS_NODE_KEY
$ADMIN_CRT to CBF_ANALYTICS_ADMIN_CRT
$ADMIN_KEY to CBF_ANALYTICS_ADMIN_KEY

kubectl -n $NAMESPACE create secret generic $ANALYTICS_CERTS_SECRET \
--from-literal=CBF_ANALYTICS_CA_CRT=$CA_CRT \
--from-literal=CBF_ANALYTICS_SIGN_CRT=$SIGN_CRT \
--from-literal=CBF_ANALYTICS_NODE_CRT=$NODE_CRT \
--from-literal=CBF_ANALYTICS_NODE_KEY=$NODE_KEY \
--from-literal=CBF_ANALYTICS_ADMIN_CRT=$ADMIN_CRT \
--from-literal=CBF_ANALYTICS_ADMIN_KEY=$ADMIN_KEY▼

Ensure the new secret has values for the data keys:

kubectl -n $NAMESPACE describe secret $ANALYTICS_CERTS_SECRET▼

Example output

Name:         cbcd-analytics-certificates-new
Namespace:    cd
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data:
CBF_ANALYTICS_NODE_KEY:   2464 bytes
CBF_ANALYTICS_SIGN_CRT:   2024 bytes
CBF_ANALYTICS_ADMIN_CRT:  1892 bytes
CBF_ANALYTICS_ADMIN_KEY:  2464 bytes
CBF_ANALYTICS_CA_CRT:     1996 bytes
CBF_ANALYTICS_NODE_CRT:   1952 bytes▼

In your values file, set analytics.certificates.exisitingSecret to:
- cbcd-analytics-certificates-new

You can now complete your CloudBees CD/RO Helm upgrade.