High Availability (active/active) troubleshooting

3 minute read

If CloudBees High Availability (HA) is not working as expected, use this page to troubleshoot.

High Availability (HA) developer mode

The High Availability (HA) feature provides a developer mode to troubleshoot HA problems in controllers running in HA mode.

To enable the developer mode in a controller.

  • Navigate to Manage Jenkins  CloudBees CI High Availability.

  • Select Status on the left navigation pane. This is the default view when accessing the CloudBees CI High Availability screen.

  • Select the Enable developer mode field and select Save or Apply.

Enable developer mode
Figure 1. Enable developer mode

Developer mode is a powerful tool to troubleshoot and understand High Availability. When enabled, the controller provides additional information about the High Availability mode:

  • A button with the current replica name appears in the page footer.

  • The background color for the replica button changes from one replica to another.

  • When selected, the replica button on the footer redirects to the CloudBees CI High Availability screen.

    Footer in developer mode
    Figure 2. Footer in developer mode.
  • The consolidated queue widget displays the queue items in all the replicas.

  • In developer mode, CloudBees CI adds the replica name to those items queued in other replicas.

  • Names for items queued in the current replica remain the same.

Build name in developer mode
Figure 3. Build name in developer mode

In addition to enabling developer mode, also from the CloudBees CI High Availability screen, users can change the replica they are using by selecting the Reset sticky session button.

Switch to another replica
Figure 4. Switch to another replica

When the Reset sticky session is selected, CloudBees CI randomly assigns a new replica. If there is no change and you are assigned to the same replica, you can reload the page and try again until a new replica is assigned.

The Reset sticky session button only displays if you are using ingress-nginx. However, you may be assigned to a different replicas if, while using your browser developer tools, you remove CloudBees CI cookies and sign in again.

High Availability (HA) Script Console

In addition to the previous tools, the CloudBees CI High Availability screen provides a High Availability Script Console. This console allows CloudBees CI users to run scripts across all the current controller replicas and displays the results. To access the HA Script Console select Script Console on the left.

HA Script Console
Figure 5. HA Script Console
  1. Select Script Console to access the HA Script Console.

  2. Type your scripts in the scripts area.

  3. Run your HA scripts. When selected, the script will be executed in all the controller replicas.

Results for the HA Script Console
Figure 6. Results for the HA Script Console

Include HA information in your support bundle

The CloudBees Support Plugin allows you to generate a support bundle that contains commonly requested diagnostic information used by CloudBees to resolve support issues.

This plugin is installed by default with CloudBees CI. For more information about how to generate support bundles and the information collected by CloudBees in those bundles refer to Generating a support bundle.

To include specific High Availability (HA) information in a controller support bundle, select the Information from other replicas option in the CloudBees Support screen. When this option is selected, the generated support bundle contains the HA information in the replicas/ folder.

Include specific HA information in your controller support bundle
Figure 7. Include specific HA information in your controller support bundle
This option is only available for controllers that run in HA mode.

Problems with a CloudBees CI on modern cloud platforms installations

Controller configuration is not set to Deployment

If the controller fails to provision with the error:

ERROR: Failed to provision controller ... StatefulSet is only for non-replicated controllers

The issue was caused by including kind: StatefulSet when configuring the controller under Advanced configurationYAML. To resolve the issue, select Acknowledge error, Free snapshot, then go to the configuration page and change kind: StatefulSet to kind: Deployment in the YAML field.