Backup and restore on Kubernetes

2 minute read
On this page

This chapter describes how to restore manually using kubectl for CloudBees CI on modern cloud platforms deployments in the cloud.

Using a rescue-pod

The approach is to create a "rescue-pod" that mounts the operations center volume via the same persistentVolumeClaim, and restore the backup from there.

  1. Retrieve the security context in the pod to restore

    $ kubectl --namespace=cje-cluster-example get pods cjoc-0 -o jsonpath='{.spec.securityContext}' map[fsGroup:1000]
  2. Scale down operations center

    $ kubectl --namespace=cje-cluster-example scale statefulset/cjoc --replicas=0 statefulset.apps "cjoc" scaled
  3. List the Persistent Volume Claims

    $ kubectl --namespace=cje-cluster-example get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE jenkins-home-cjoc-0 Bound pvc-6b27e963-b770-11e8-bcbf-42010a8400c1 20Gi RWO standard 46d jenkins-home-mm1-0 Bound pvc-b2b7e305-ba66-11e8-bcbf-42010a8400c1 50Gi RWO standard 42d jenkins-home-mm2-0 Bound pvc-6561b8da-c0c8-11e8-bcbf-42010a8400c1 50Gi RWO standard 34d
  4. Run the rescue-pod with the required pvc (jenkins-home-cjoc-0 in this example) and the required security context

    $ cat <<EOF | kubectl --namespace=cje-cluster-example create -f - kind: Pod apiVersion: v1 metadata: name: rescue-pod spec: securityContext: runAsUser: 1000 fsGroup: 1000 volumes: - name: rescue-storage persistentVolumeClaim: claimName: jenkins-home-cjoc-0 containers: - name: rescue-container image: nginx command: ["/bin/sh"] args: ["-c", "while true; do echo hello; sleep 10;done"] volumeMounts: - mountPath: "/tmp/jenkins-home" name: rescue-storage EOF pod "rescue-pod" created
  5. Move the backup file to the rescue-container

    kubectl cp oc-jenkins-home.backup.tar.gz rescue-pod:/tmp/
  6. (Optional) Clean the previous $JENKINS_HOME.

    In the case, you have a complete copy of the $JENKINS_HOME (Manual Backup) and you wish to perform a rollback after a failed update, wipe out the old content.

    # file kubectl exec --namespace=cje-cluster-example rescue-pod -it -- find /tmp/jenkins-home -type f -name '*.*' -delete # folders kubectl exec --namespace=cje-cluster-example rescue-pod -it -- find /tmp/jenkins-home/ -mindepth 1 -type d -name '*' -exec rm -rf {} \;
  7. Uncompress the backup file inside the cjoc Persistent Volume Claim

    kubectl exec --namespace=cje-cluster-example rescue-pod -it -- tar -xzf /tmp/oc-jenkins-home.backup.tar.gz -C /tmp/jenkins-home
  8. Delete the rescue-pod

    kubectl --namespace=cje-cluster-example delete pod rescue-pod
  9. Scale up the cjoc

    kubectl --namespace=cje-cluster-example scale statefulset/cjoc --replicas=1