With CloudBees CD/RO, you can model a microservice application and its deployment environment using a pre-provisioned Kubernetes cluster definition. Helm and YAML are the supported deployment options for microservice applications.
A CloudBees CD/RO microservice application consists of the microservices and processes that orchestrate the deployment of the microservices to pre-provisioned Kubernetes clusters. Microservice applications consists of one or more microservices, where each microservice is a logical group of containers. All microservices within a microservice application are mapped to at least one cluster within an environment.
To create a microservice application that deploys to a mapped Kubernetes cluster environment via Helm:
-
Ensure all Prerequisites are met.
-
Review Argo Rollouts deployment and delivery specific considerations, if applicable.
-
Define the environment, configure a cluster reference based upon pre-provisioned Kubernetes cluster.
-
Map the microservice to the cluster in the environment.
Prerequisites
These instructions assume you understand the tooling required for provisioning a Kubernetes cluster, are familiar with Kubernetes concepts, and have provisioned your Kubernetes cluster or clusters onto which the microservice application is deployed. Additionally, the following prerequisites must be met:
-
Kubernetes requirements:
-
The Kubernetes clusters used for deployment must be v1.16 or higher.
-
To use Argo Rollouts, you must install the Argo controller in your cluster.
-
-
CloudBees CD/RO plugins requirements:
-
For YAML, the EC-Kubectl plugin v1.4.0 or higher is required.
-
For Helm, the EC-Helm plugin is required.
-
-
CloudBees CD/RO agent requirements:
-
CloudBees CD/RO agent images v2023.03.0 and higher:
-
For CloudBees CD/RO v2023.03.0 or later
cbflow-agentimages, the tools to use this feature come preinstalled. For current tool version information, refer to CloudBees CD/RO agents third-party tools.
-
-
CloudBees CD/RO agent images v2023.02.0 and earlier, you must install the following tools:
-
For Helm-based deployments, you must install the Helm CLI.
-
For YAML-based deployments, you must install kubectl.
-
To use Argo Rollouts, you must install the kubectl-argo-rollout plugin in your agent.
-
-
Additional knowledge-based prerequisites:
-
For YAML-based deployments, refer to YAML-based deployment file naming.
-
For Helm-based deployments, refer to Helm-based deployment flags and options.
YAML-based deployment file naming
When using YAML-based deployments, kubectl is used to process jobs. When kubectl parses over directories, YAML files in the directory are applied in the order they appear in the directory, which is typically alphabetically.
This may cause temporary degraded statuses in your deployment if a file with a dependency is read before its dependency. This is a kubectl known issue, and when deploying directories, applies to both Git-based deployments and local workspace deployments.
Once Argo Rollouts detects the degraded status, it will try to self-repair and pass over the files again which normally fixes the issue in around 20 seconds. CloudBees CD/RO will report the degraded status when it occurs, and updates it once the issue is corrected.
To avoid this issue, you can prefix your file names with a number sequence, so they are read in the correct order. For instance, if you have a my-deployment directory with the following structure:
|-- my-deployment | |-- rollouts.yaml ## Depends on services.yaml, which is not deployed until after. | |-- services.yaml
To read the dependency first, you could rename the files as follows:
|-- my-deployment | |-- 01_services.yaml | |-- 02_rollouts.yaml ## Depends on services.yaml, which is already deployed.
Helm-based deployment flags and options
Helm-based microservice deployments use the helm upgrade --install command to support both fresh and upgrade deployments. To pass flags and options as deployment parameters to helm upgrade --install, use the Additional options field when creating your application.
Parameters are appended to the command in the following order:
| When an option is configured in multiple parameters, the value of the last occurrence is used. |
-
Values file definitions configured with the Values parameter.
-
Microservice definition configuration Repository name, Repository credentials, Release name, and Chart.
-
Microservice definition Additional options.
-
Cluster definition Namespace parameter.
-
Cluster definition Kubeconfig Context parameter.
-
If automatic rollback is configured, the
--atomicoption is also added.
Release manifests are parsed for containers and other Kubernetes resources deployed with your application, which are listed in the environment inventory.
|
Because different YAML libraries implement different YAML specifications, some manifests accepted by Kubernetes may not be parsed, resulting in their containers or Kubernetes resources not appearing in the environment inventory. If you receive warnings about a specific YAML file that has containers, and the chart is under your governance, ensure the chart template is correctly formatted and linted. For charts provided by third-parties, file an issue with that provider. |
Use Argo Rollouts inline analysis
Microservice deployments support using Argo Rollouts inline analysis results. These results are provided by approved Argo Rollouts metric providers, and can be used for YAML and Helm-based deployments.
|
CloudBees CD/RO only reads the analysis results supplied by Argo Rollouts metric providers. For more information on configuring Argo Rollouts inline analysis or approved Argo Rollouts metric providers, refer to the Argo Rollouts documentation. |
To use this feature:
-
Inline analysis conditions must be configured in your YAML or Helm-based deployments prior to running the microservice deployment pipeline in CloudBees CD/RO.
For Helm-based deployments, Argo Rollouts templates must be properly escaped in Helm charts to avoid conflicts between Argo Rollouts and Helm templates. For more information, refer to the Argo Rollouts documentation. -
You must have the minimum plugin version installed for your deployment configuration:
-
For Helm-based deployments, EC-Helm v1.8.0 or later is required.
-
For YAML-based deployments, EC-Kubectl v1.5.0 or later is required.
-
When using Argo Rollouts inline analysis, there are three basic types of results:
-
Passed: The deployment meets the criteria described in the deployment chart.
-
Failed: The deployment fails to meet the criteria described in the deployment chart.
-
Inconclusive: It cannot be determined if the deployment meets or fails to meet the criteria described in the deployment chart.
Based of these results, the CloudBees CD/RO gate logics are:
-
For passing metric results, CloudBees CD/RO proceeds with the deployment.
-
For failing metric results, CloudBees CD/RO stops and rolls back the deployment.
-
For inconclusive metric results:
-
The microservice deployment is paused.
-
A manual approval step is automatically created and applied to the current pipeline step.
This manual approval is created in addition to any other manual approval steps in your pipeline. Once created, this manual approval pauses the pipeline until it is approved or rejected. Even if you approve or reject the deployment via
kubectlcommands, the pipeline remains paused until this step is approved or rejected. -
Based on the configured notifications for Assignees, notifications are sent.
-
A member of the Assignee group approves or rejects the deployment via the manual approval step.
-
CloudBees CD/RO processes the deployment based on the gate logic.
-
For more information on Argo Rollouts inline analysis results, refer to the Argo Rollouts documentation.
Argo Rollouts deployment and delivery
Argo Rollouts is a Kubernetes controller and set of custom resource definitions (CRDs) that provide advanced deployment capabilities for Kubernetes environments, such as:
-
Blue-green
-
Canary
-
Canary analysis
-
Experimentation
-
Progressive delivery
Create a new application or copy an existing application
-
From the CloudBees CD/RO main menu, select . The Applications list displays.
-
Select New application. The New Application dialog displays.
-
Choose an option.
-
Add new application.
-
Select Create new.
-
Complete the following field.
Label Definition Application name
User-defined name for the application.
Project
User-defined name of the project containing the application.
Description
(Optional) User-defined information about this application. Not used by the system.
Application type
The application type. Select Microservice.
Tags
Tags let you group related objects by a user-defined term. To tag this object or to modify or delete a tag, select the Open Dialog arrow at right. Refer to Object Tags for more information.
-
-
Copy an existing application.
-
Select Copy existing.
-
Select the name of the application to copy.
-
Complete the following fields.
Label Definition Project
User-defined name of the project containing the application.
Description
(Optional) User-defined information about this application. Not used by the system.
Tags
Tags allow you to group related objects by a user-defined term. To tag this object or to modify or delete a tag, select the Open Dialog arrow at right. Refer to Object Tags for more information.
-
-
-
Select OK. The application editor displays.
Figure 1. Application editor -
Continue modifying the microservices application by either:
Define microservice details for Helm
When you create a new application, the new application is pre-populated with one microservice tile.
-
In the microservice tile of the Application editor, select Add microservice. The Add microservice dialog displays.
-
Choose an option.
-
Add a new microservice.
-
Select Create new.
-
Complete the following fields.
Label Definition Name
User-defined name for the microservice.
Description
User-defined description for the microservice.
-
-
Copy an existing microservice.
-
Select Copy existing.
-
Complete the following fields.
Label Definition Project
User-defined name of the project containing the application.
Application name
User-defined name for the application.
Microservice
User-defined name for the microservice.
-
Select Next
-
-
-
Select
Helmfrom the Definition type choices. -
Choose a microservice repository source from the Definition source options.
-
Clone and deploy a Git repository:
-
Select
Git repositoryfrom the Definition source menu. -
Complete the following fields.
Label Definition Configuration Name
User-defined name for the Git configuration.
Git Repository
User-defined name of the Git repository.
Remote branch
User-defined name of the remote Git branch.
-
-
Configure a Helm repository.
-
Select
HELM repositoryfrom the Definition source menu. -
Complete the following fields.
Label Definition Repository URL
The URL of the Helm chart repository.
Some custom host or repository configurations for Helm charts may be supported by Kubernetes. CloudBees CD/RO supports most options listed the Helm Chart Repository Guide. The following Kubernetes configurations with CloudBees CD/RO have been tested using Helm charts from the following host types:
-
Helm-based repositories.
-
Git-based repositories.
-
Nexus-based repositories.
IPv6 addresses are only supported for Kubernetes platforms. If using an IPv6 address, enclose the address in square brackets. Example: [<IPv6-ADDRESS>].Repository name
User-defined name of the Helm chart repository.
Repository credentials
Credentials needed for repository connection. Select Browse to locate previously created credentials. For details, refer to Credential management.
-
-
-
-
Enter additional microservice details.
Label Definition Release name
User-defined name of the deployment release.
Chart
User-defined name of the deployment Helm chart.
Chart version
User-defined deployment Helm chart version.
The version is appended to the command line options supplied to the
helm upgradecommand. If the version specified in Chart version differs from the Additional options--versionvalue, then the Additional options parameter--versionvalue is removed.Additional options
Additional options to be provided to the Helm CLI.
Values
A list of values. Values in this file have lower priority than Additional options. The list supplied here is saved to a file and passed to the command with the
-f <fileName>option.Enable automatic rollback on failure?
Select this option to automatically roll back deployments upon failure. When selected, a system-generated rollback microservice process step is created in the microservice process, and the
--atomicHelm option is added to the deployment command.Timeout
Available when rollback is enabled. This is the number of seconds to trigger a rollback timeout.
-
Select Add microservice.
-
Continue modifying the microservices application by configuring the microservice processes, environment, cluster reference, and mapping settings.
Define microservice details for YAML
When you create a new application, the new application is pre-populated with one microservice tile and one environment tile.
-
In the microservice tile of the Application editor, select Add microservice. The Add microservice dialog displays.
-
Choose an option.
-
Add a new microservice.
-
Select Create new.
-
Complete the following.
Label Definition Name
User-defined name for the microservice.
Description
User-defined description for the microservice.
-
-
Copy an existing microservice.
-
Select Copy existing.
-
Complete the following.
Label Definition Project
User-defined name of project containing the application.
Application
User-defined name for the application.
Microservice
User-defined name for the microservice.
-
Select Next
-
-
-
Select
YAMLfrom the Definition type choices. -
Choose a microservice repository source from the Definition source options.
When using YAML-based deployments, ensure you read and understand YAML-based deployment file naming. -
To clone and deploy YAML files from Git repositories:
-
Select
Git repositoryfrom the Definition source menu. -
Complete the following fields.
Label Definition Configuration Name
User-defined name for the Git configuration.
Git Repository
User-defined name for the Git repository.
kubectldoes not deploy YAML files recursively. If you want to deploy specific YAML files, add the relative path to those files in Relative path.Remote branch
User-defined name of the Git remote branch.
Relative path
The relative path to the Git configuration.
kubectldoes not deploy YAML files recursively.Additional options
(Optional) Enter additional options to include with the
kubectl applycommand. You cannot enter thenamespace,context,cluster, oruseras additional options. Instead, you can add these configurations to your plugin configuration or cluster definition.
-
-
To add YAML manual input, a single YAML file, or directory that contains your YAML files:
-
Select
YAML contentfrom the Definition source menu. -
Complete the following fields.
Label Definition Source Type
The source of the YAML content.
-
Select Content to enter the YAML formatted content directly into the YAML Content field.
-
Select YAML Content. Select Path to enter the path to an existing YAML file in the Relative path field.
kubectldoes not deploy YAML files recursively. If you want to deploy specific YAML files, add the relative path to those files in Relative path.
YAML Content
The user-defined name of the YAML content source.
-
-
-
-
Enter additional microservice details.
Label Definition Relative path
The relative path to the YAML content source.
kubectldoes not deploy YAML files recursively.Additional options
(Optional) Enter additional options to include with the
kubectl applycommand. You cannot enter thenamespace,context,cluster, oruseras additional options. Instead, you can add these configurations to your plugin configuration or cluster definition. -
Select Add microservice.
-
Continue modifying the microservices application by configuring the microservice processes, environment, cluster reference, and mapping settings.
Configuring a microservice process
CloudBees CD/RO creates two process steps by default:
-
Retrieve Artifact
-
Deploy Microservice
EC-Helm supports automatic rollback. When you create a microservice for Helm, CloudBees CD/RO creates a third process step, Rollback Microservice, by default.
To configure a microservice process:
-
In the Application tile of the Application editor, select Microservice process from the microservice
menu. The list of microservices for the microservice process displays. -
Select Edit from the microservice
menu. The Edit application processes page display. -
Configure the application process by defining the following:
Label Definition Process name
User-defined name of the microservice process.
Description
User-defined description of the microservice.
Process task type
The task type of the microservice. Select
Deploy,UndeployorOther.Workspace
Name of the microservice workspace.
Working directory
Working directory where the microservice process will run.
Exclusive access
True if environment is exclusive to the application or service process.
Time limit
Units for step time limit: seconds, minutes, or hours.
Credentials
The name of a credential to attach to this process.
Default rollout options
Used to set the default assigned approvers and turn on/off email notifications when rollout deployment is configured with manual approval. If approvers are not specified then the user who launched the deploy process will be assigned as the approver. These default values can be overwritten later upon process run, in pipeline task or release deployer configuration.
-
Select OK.
-
Continue modifying the microservices application by configuring the environment, cluster reference, and mapping settings.
Define the environment
Now you define the cluster environment into which the microservice is deployed. This cluster must have already been provisioned outside of CloudBees CD/RO.
To add an environment:
-
In the environment tile of the Application editor, select +. The New environment dialog opens.
-
Select either Create new or Use existing. If you are creating a new environment, the New environment dialog opens.
-
Enter details about this environment.
Label Definition Environment name
User-defined name of the environment object containing the cluster.
Project name
The user-defined name of the project containing the environment.
Environment description
(Optional) User-defined information about this environment. Not used by the system.
Utility resource name
User-defined name of the resource used to deploy the microservice. This must be a CloudBees CD/RO agent, v10.1 or higher, with the following binaries installed:
-
Helm v3.4.1 or higher.
-
Kubectl v1.16 or higher.
Resource
Select the resource or resource pool.
-
-
Add additional utility resources to the environment if required.
-
Select Add another utility resource.
-
Enter the Utility resource name and select the Resource.
-
-
Select Add environment. The new Utility Resource tile opens in the editor field.
-
Continue modifying the microservices application by configuring the cluster reference, and environment mapping.
Configure a cluster reference
After you create an environment, you can add a cluster reference.
-
In the environment tile of the Application editor, select Add cluster reference. The Add cluster reference dialog opens.
-
Enter details about this cluster reference.
Label Definition Cluster reference name
User-defined name of the cluster.
Cluster reference description
(Optional) User-defined information about this environment. Not used by the system.
Configuration provider
Select a configuration provider.
-
For Helm deployment: Select Kubernetes via Helm (Supports microservice model).
-
For YAML deployment: Select Kubernetes via Kubectl (Supports microservice model).
Configuration Name
A cluster configuration is CloudBees CD/RO object that holds common values, predefined parameter sets, and credentials for your cluster. A cluster configuration leverages the CloudBees CD/RO EC-Helm plugin for Helm deployments or the EC-Kubectl plugin for YAML deployments.
-
Select New configuration to define a new configuration. For more information, refer to Helm plugin or Kubectl plugin.
-
Select the name of a previously defined configuration for the cluster.
Namespace
User-defined name of the Kubernetes namespace. If not supplied, the default namespace is used.
Kubeconfig context
User-defined name of the context to be used for this cluster deployment. If not specified, the one defined by the
current-contextinside thekubeconfigfile is used. -
-
Select Add cluster reference.
-
Continue modifying the microservices application by configuring the environment mapping.
Map the microservice to an environment
For CloudBees CD/RO to correctly deploy the microservice, the microservice needs to be mapped to the environment that defines the cluster.
Configure microservice environment mapping.
-
Select Add mapping or Edit mapping in the Application editor. The Mapping dialog opens.
-
Select the Cluster reference.
-
Select Save mapping.
Deploying the application
Once the application is created and configured, you can deploy it. Refer to Example: Deploy and troubleshoot applications for information about deploying applications.