Configure a disaster recovery site for CloudBees Analytics

This guide provides instructions to copy backup snapshots from your primary CloudBees Analytics cluster and restore them in a disaster recovery (DR) site.

Assumptions

The following setup and environment conditions are assumed throughout this procedure:

Snapshot backups on the primary site are stored in /os-backups.
The DR site should not automatically trigger scheduled backups. To prevent this, suspend backup schedules using the following cron expression:
```
schedule_cron: "0 0 31 2 *"▼
```

Copy and restore CloudBees Analytics snapshots

Follow the steps in this section to create a snapshot backup archive on your primary site and deploy it to your DR site for restoration.

Create a backup archive on the primary site

Create a compressed backup archive of snapshot data stored in the primary analytics node.

Define necessary variables:

PRIMARY_NAMESPACE=<primary-namespace>
ANALYTICS_POD=$(kubectl get pod -n $PRIMARY_NAMESPACE -l component=flow-analytics -o jsonpath="{.items[0].metadata.name}")▼

Open a terminal in the primary analytics pod:

kubectl exec -n $PRIMARY_NAMESPACE -it $ANALYTICS_POD -- /bin/bash▼

Inside the container, archive the contents of the backup directory:
```
cd /os-backups
tar czf /tmp/backup.tar.gz .▼
```

Exit the pod, then copy the backup archive to your local machine:

kubectl cp $PRIMARY_NAMESPACE/$ANALYTICS_POD:/tmp/backup.tar.gz ./backup.tar.gz▼

Transfer the backup archive to the DR site

Send the backup archive created on the primary site to your DR cluster analytics pod and prepare the files for use.

Define DR namespace and pod variables:

DR_NAMESPACE=<dr-namespace>
DR_ANALYTICS_POD=$(kubectl get pod -n $DR_NAMESPACE -l component=flow-analytics -o jsonpath="{.items[0].metadata.name}")▼

Copy the backup archive into the DR analytics pod:

kubectl cp ./backup.tar.gz $DR_NAMESPACE/$DR_ANALYTICS_POD:/tmp/backup.tar.gz▼

Open a terminal in the DR analytics pod:

kubectl exec -n $DR_NAMESPACE -it $DR_ANALYTICS_POD -- /bin/bash▼

Clear the existing backup directory (optional but recommended before restore):
```
rm -rf /os-backups/*▼
```

Extract the archive into the backup directory:

tar -xvf /tmp/backup.tar.gz -C /os-backups/▼

Validate snapshot consistency

Ensure that the backup contents match across primary and DR sites by inspecting the file list.

Run the following command in both the primary and DR analytics pods:
```
ls -lrt /os-backups▼
```
Compare output between clusters to confirm timing, file names, and content alignment.

Once you confirmed the consistency of the snapshot between the primary and DR sites, move on to the next section.

Validate or create the snapshot repository on the DR site

To use the restored snapshot files, the DR cluster must have a valid snapshot repository configured.

Check if the snapshot repository exists:

curl -u admin:<analytics-admin-pass> --insecure \
  --header 'Content-Type: application/json' \
  https://flow-analytics:9201/_snapshot/analytics_snap/_all▼

If the repository does not exist, create a new one:

curl -XPUT -u admin:<analytics-admin-pass> --insecure \
  --header 'Content-Type: application/json' \
  https://flow-analytics:9201/_snapshot/analytics_snap \
  -d '{
    "type": "fs",
    "settings": {
      "location": "/os-backups",
      "compress": true
    }
  }'▼

Re-check the repository to confirm creation:

curl -u admin:<analytics-admin-pass> --insecure \
  --header 'Content-Type: application/json' \
  https://flow-analytics:9201/_snapshot/analytics_snap/_all▼

Restore snapshots from the DR pod

Restore snapshots using the snapshot repository now available on the DR cluster.

From inside the DR analytics pod, trigger a restore process using a snapshot restore job or by calling the OpenSearch snapshot API.
Confirm that snapshot files in /os-backups are detected and restored as expected.

For in-depth restore instructions, consult the OpenSearch or Elasticsearch snapshot and restore documentation that aligns with your CloudBees Analytics version.