This content applies only to CloudBees CI on modern cloud platforms and managed controllers.
In a CloudBees CI on modern cloud platforms installation, users with the appropriate CasC permissions can configure managed controllers by adding CasC bundle locations in the operations center. The operations center uses the bundles in the location to configure the managed controllers.
Because the operations center is the only source of CasC bundles for the managed controllers, this setup creates a single point of failure. Additionally, if your organization has a large number of managed controllers, the operations center performance may be impacted as it is responsible for the following operations:
-
Query the bundle locations for new bundles or updates.
-
Check out new bundles and updates to the operations center local storage.
-
Validate the bundles.
-
Distribute the bundles to the managed controllers.
To address these scenarios, CloudBees provides the CasC Controller Bundle Service. This service, disabled by default, is installed as a part of the CloudBees CI on modern cloud platforms Helm chart and, if enabled, frees the operations center from the responsibility of distributing CasC bundles to managed controllers.
|
Using the CasC Controller Bundle Service and using the operations center to distribute CasC bundles are not mutually exclusive. Administrators can use both methods to distribute CasC bundles to managed controllers, according to their needs. However, a managed controller can only be configured to use one method at a time. The CasC bundle comes from the operations center or from the CasC Controller Bundle Service. CloudBees recommends using the CasC Controller Bundle Service, particularly when you have a large number of managed controllers. |
|
The CasC Controller Bundle Service does not support Bundle inheritance or bundle availability. If the |
CasC Controller Bundle Service architecture
-
The CasC Controller Bundle Service is a separate service that runs in the CloudBees CI on modern cloud platforms cluster. It can be enabled during the installation, and it is configured with connectors to the locations where the CasC bundles are stored. A connector is an authenticated connection from the CasC Controller Bundle Service to systems that serve CasC bundles (SCMs in the image).
The CasC Controller Bundle Service runs in the same namespace as the operations center and assumes that managed controllers are provisioned in the same Kubernetes cluster and can reach the CasC Controller Bundle Service service to retrieve the CasC bundles, which happens with the default options set in the Configure Controller Provisioning screen.
-
The CasC Controller Bundle Service periodically queries the connectors to retrieve the CasC bundles.
-
When administrators create or update a managed controller, the operations center queries the CasC Controller Bundle Service and retrieves a list of available bundles.
-
The managed controller is configured to use a CasC bundle provided by the CasC Controller Bundle Service.
-
The managed controller retrieves the CasC bundle from the CasC Controller Bundle Service. After the CasC bundle is applied, the managed controller periodically checks the CasC Controller Bundle Service for updates.
Enable the CasC Controller Bundle Service for CloudBees CI on modern cloud platforms
Install CloudBees CI covers the requirements and the installation process for CloudBees CI on modern cloud platforms.
The installation is performed using a Helm chart and the helm install command.
The CasC Controller Bundle Service is disabled by default in the Helm chart.
To enable the CasC Controller Bundle Service, you must follow these steps:
-
Create a
values.yamlfile that enables the CasC Controller Bundle Service. -
Install the CloudBees CI on modern cloud platforms Helm chart using the
values.yamlfile.
Create a values.yaml file
To enable the CasC Controller Bundle Service, create a values.yaml file with the following content:
CascBundleService: enabled: true (1)
| 1 | Enable the CasC Controller Bundle Service. |
Enable debug logs for the CasC Controller Bundle Service
To test the CasC Controller Bundle Service or troubleshoot issues, enable DEBUG logs by adding the following property to your values.yaml:
CascBundleService: javaOpts: -Dquarkus.log.category."com.cloudbees".level=DEBUG
Security context for the CasC Controller Bundle Service pods
As described in the service architecture, the CasC Controller Bundle Service creates a Kubernetes deployment. Security contexts for the service pods and containers are empty by default. However, the CasC Controller Bundle Service requires a security context configuration if CloudBees CI runs in a Kubernetes cluster that defines Pod Security Standards by using Pod Security Admission (PSA) labels.
Administrators can provide custom pod security contexts or custom container security contexts by adding the PodSecurityContext or the ContainerSecurityContext property to the CasCBundleService section in the values.yaml file, as shown in the following examples:
CascBundleService: ... PodsecurityContext: fsGroup: 1000 runAsUser: 1000 runAsGroup: 1000 ...
CascBundleService: ... ContainerSecurityContext: allowPrivilegeEscalation: false privileged: false runAsGroup: 10001 runAsNonRoot: true runAsUser: 10001 ...
|
If no value is provided for
|
Refer to the Kubernetes documentation for more information on pods and containers security contexts.
Install the CloudBees CI on modern cloud platforms Helm chart using the values.yaml file
To install CloudBees CI on modern cloud platforms with the CasC Controller Bundle Service enabled, use the regular Helm chart, refer to the Install CloudBees CI documentation and use the values.yaml file created in the previous step.
The command must be similar to the one below.
helm install cloudbees-core cloudbees/cloudbees-core -f values.yaml (1)
| 1 | Refer to the Install CloudBees CI documentation for specific installation instructions for your infrastructure. |
|
If the CasC Controller Bundle Service is not enabled during the installation, you can enable it later by updating the Helm chart values and running the |
Add connectors to the CasC Controller Bundle Service
The service, as described in the CasC Controller Bundle Service architecture, uses connectors to retrieve CasC Bundle from SCMs.
After enabling the service, administrators must add the connectors to the service configuration. Those connectors are stored within a Kubernetes secret and provide the authentication information to access the SCM repositories.
The Kubernetes secret with the connectors must have the following structure:
apiVersion: v1 kind: Secret metadata: name: casc-bundle-service-config (1) labels: app: casc-bundle-service (2) type: Opaque stringData: service-configuration.yaml: | (3) connectors: ... <list-of-connectors> (4) ... credentials: ... <list-of-credentials> (5) ... knownHosts: (6) ... <public-keys-for-ssh-connectors> ...
| 1 | Keep casc-bundle-service-config as the name of the secret. |
| 2 | Keep casc-bundle-service as the value for the app label. |
| 3 | The service-configuration.yaml key contains the configuration for the CasC Controller Bundle Service. |
| 4 | List of connectors to the SCM repositories. |
| 5 | List of credentials referenced by the connectors to authenticate to the SCM repositories. |
| 6 | (Optional) List of public keys required for SSH connectors.
Those keys are usually stored in your ~/.ssh/known_hosts file. |
Connectors and credentials properties are listed in the tables below:
| Property name | Required | Description | Example |
|---|---|---|---|
|
Yes |
A unique identifier for the connector. |
|
|
Yes |
The URL of the SCM repository. HTTP and git URLs are supported. HTTP URLs can use token or username and password authentication. Git URLs use SSH authentication. |
|
|
No |
The type of connector.
|
|
|
No |
The branch of the repository to use.
|
|
|
No |
The relative path to the directory containing the CasC bundles. |
|
|
No |
It provides an authentication method for push notifications from the connector. Refer to Notification webhooks for the SCM connectors for more information. |
|
|
Yes |
The credential used for the connector. It contains subproperties of fields described in more detail in the table below. |
credential: token: tokenCred type: token |
| Credential property name | Required | Description | Example |
|---|---|---|---|
|
Only required if used to reference a credential defined in the |
It uses and ID to reference a credential defined in the |
|
|
Only required if the |
The type of credential.
The supported values are |
|
|
Required if the |
The token to use for authentication. |
|
|
Required if the |
The username to use for authentication. |
|
|
Required if the |
The password to use for authentication. |
|
|
Required if the |
The SSH key to use for authentication. |
|
|
No |
The passphrase to use for the SSH key.
Only used if the |
|
|
Only required if the |
This property refers to the GitHub application ID that must be created to display bundle validation results in a GitHub pull request. Refer to Bundle validation using the CasC Controller Bundle Service for more information. |
|
|
Only required if the |
A PKCS8 key used for authentication in the GitHub application. Refer to Bundle validation using the CasC Controller Bundle Service for more information. |
|
|
When defining connectors, if the Each credential must have a unique |
apiVersion: v1 kind: Secret metadata: name: casc-bundle-service-config labels: app: casc-bundle-service type: Opaque stringData: service-configuration.yaml: | connectors: - id: id1 url: https://<your-repository1-url> webhookSecret: <your-webhook-secret> (1) credential: credentialId: cred1 type: reference - id: id2 url: https://<your-repository2-url> branch: testBranch path: testPath type: scm credential: user: bob password: fake type: userPassword - id: id3 url: https://<your-repository3-url> credential: credentialId:cred3 - id: id4 url: git@<your-git-server>:<your-organization-or-username>/<your-repository4-name>.git credential: credentialId: cred4 - id: id5 url: git@github.com:<your-organization-or-username>/<your-repository5-name>.git credential: sshKey: longSSHKey passPhrase: passPhrase type: sshKey - id: id6 url: https://github.com/<your-organization-or-username>/<your-repository6-name>.git credential: credentialId: cred6 credentials: - credentialId: cred1 user: bob password: fake type: userPassword - credentialId: cred3 sshKey: longSSHKey type: sshKey - credentialId: cred4 sshKey: longSSHKey passPhrase: passPhrase type: sshKey - credentialId: cred6 appId: <your-github-app-id> type: githubAppKey privateKey: | -----BEGIN PRIVATE KEY----- key-content -----END PRIVATE KEY----- knownHosts: | (2) # github.com:22 SSH-2.0-7d0489107 github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl # github.com:22 SSH-2.0-7d0489107 github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk= # github.com:22 SSH-2.0-7d0489107 github.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg=
| 1 | The webhookSecret property is used to authenticate the push notifications from the SCM repository. Replace <your-webhook-secret> with the desired secret used to authenticate the push notifications.
Refer to Notification webhooks for the SCM connectors for more information. |
| 2 | github.com public key retrieved using ssh-keyscan github.com.
The secret includes this public key in the knownHosts section because of the id5 SSH connector that refers to a github.com repository. |
To update the list of connectors you must update the values stored in the Kubernetes secret.
kubectl update secret generic casc-bundle-service-config -n cloudbees-core --from-file service-configuration.yaml=<your-secret-yaml-file> (1)
| 1 | Replace <your-secret-yaml-file> with the name of the file containing the secret configuration. |
Once the secret defining the connectors is updated, you can use one of the following options to get the CasC Controller Bundle Service to use the new list.
-
Wait until the next connector polling cycle (defaults to 5 minutes).
-
Restart the CasC Controller Bundle Service with the following command:
kubectl rollout restart deployment casc-bundle-service -n cloudbees-core
Configure a managed controller using a bundle from the CasC Controller Bundle Service
The process to configure a managed controller using a CasC bundle from the CasC Controller Bundle Service is similar to the process described in Set up a managed controller using CasC. The only difference appears in the managed controller configuration screen, where you must select whether you are using a CasC bundle to configure the controller or not, and the source for the CasC bundles (the operations center or the CasC Controller Bundle Service) if you are using one.
To configure a managed controller using a CasC bundle from the CasC Controller Bundle Service, follow these steps:
-
Select New Item from the operations center main menu.
-
Enter a name for the managed controller in the Item name field.
-
Select the Managed Controller option.
-
Click OK.
In the controller configuration screen:
-
Scroll down to the Configuration as Code (CasC) section.
-
Select CasC Bundles From CasC Bundle Service as the source for the CasC bundles.
-
Select the bundle for the new managed controller from the Bundle dropdown list.
-
Set the rest of the configuration options as needed.
-
Select Save.
|
Bundles displayed in the Bundle dropdown list are retrieved from the connectors. The dropdown list displays the value of the Description field in the bundle.yaml file to identify the CasC bundle. If the Description field is not set, the dropdown list displays information from the connector configuration instead. CloudBees recommends using meaningful descriptions for the CasC bundles to make them easier to identify in the dropdown list. |
Notification webhooks for the SCM connectors
When the CasC Controller Bundle Service is enabled, the service automatically creates the following endpoints in your Kubernetes cluster to receive push notifications from the SCM repositories associated with the connectors:
-
/casc-bundle-service/api/v1/github-webhookfor GitHub repositories. -
/casc-bundle-service/api/v1/gitlab-webhookfor GitLab repositories. -
/casc-bundle-service/api/v1/bitbucket-webhookfor Bitbucket repositories.
You must configure your SCM repositories to send push notifications to the CasC Controller Bundle Service endpoints.
If the webhookSecret property is set in the connector configuration, you must also add the secret to the webhook configuration in the SCM repository.
When the CasC Controller Bundle Service receives a push notification due to changes in any of the CasC bundles:
-
It finds the connector associated with the webhook.
-
Authenticates the request using the
webhookSecretproperty. -
Checks out the new CasC bundle.
-
Makes the new bundle available for the managed controllers using the connector.
|
The If this property is set to |
Bundle validation using the CasC Controller Bundle Service
Troubleshoot CasC for controllers describes how to validate and troubleshoot problems with controller CasC bundles.
From the operations center UI, administrators can select a CasC bundle provided by the CasC Controller Bundle Service to configure a managed controller. CasC bundles are automatically validated when applied to managed controllers. The validation results appear in the Configuration as Code export and update screen, specifically in the Bundle update and Update log tabs and in the managed controller logs.
In addition to the validations described in Troubleshoot CasC for controllers, administrators can display bundle validation results using GitHub Checks. When administrators update a CasC bundle and create a new pull request in a GitHub repository, CasC Controller Bundle Service automatically validates the bundle against the first managed controller found using that bundle. After the validation process, the validation results display in the pull request as a GitHub check.
Administrators can review these validation results, and if they are acceptable, merge the pull request to update the bundle in the repository. The CasC Controller Bundle Service then notifies all managed controllers using that bundle to update their configuration.
To display the validation results in a GitHub pull request requires the following configurations.
-
Create a GitHub app with the following permissions:
-
Checks:Read and Write. -
Commit Statuses:Read and Write. -
Contents:Read-only. -
Pull Requests:Read-only.
-
-
Once the GitHub app is ready, generate a private key for the GitHub app and download the
pemfile. -
Transform the private key into a PKCS8 key by using the following
opensslcommand.openssl pkcs8 -topk8 -inform PEM -outform PEM -in <your-github-app-key>.pem -out <your-pkcs8-key>.pem -nocrypt (1)1 Replace <your-github-app-key>with the name of the GitHub application key and<your-pkcs8-key>with the name of the PKCS8 key to be used in the connector configuration. -
Install the GitHub app in the repository or repositories where you want to display the validation results.
-
Add the
appIdandprivateKeyproperties to the connector configuration in the Kubernetes secret. Now the CasC Controller Bundle Service can use the GitHub app to display the validation results in the pull request.
apiVersion: v1 kind: Secret metadata: name: casc-bundle-service-config labels: app: casc-bundle-service type: Opaque stringData: service-configuration.yaml: | connectors: .... - url: https://<your-github-repository-url> credential: appId: <your-github-app-id> type: githubAppKey privateKey: -----BEGIN PRIVATE KEY----- key-content -----END PRIVATE KEY----- ....
Alternatively, administrators can also manually validate the bundle using the Configuration as Code (CasC) CLI or by sending requests to the Configuration as Code HTTP API.