Plugin management with CasC

Starting on the first April 2024 release, CloudBees CI CasC improves and changes how plugins are managed using CasC. Those changes require to use apiVersion: 2 in the bundle.yaml file. CasC bundles using apiVersion:1 can still be used. This documents introduces CasC Plugin Management 2.0 and compares apiVersion:1 and apiVersion:2 behaviors, in terms of plugin management.

CasC Plugin Management 2.0

CasC Plugin Management 2.0 provides the following features:

Plugin dependencies management for CAP and non-CAP plugins.
Alternative plugin download locations for non-CAP plugins.
CAP and non-CAP plugins in the plugins.yaml file.
CasC plugins report with all the effectively installed plugins and their versions in a JSON format.

CasC does not support plugin removal. Therefore, when a plugin is no longer part of a CasC bundle, the plugin will not be removed from the instance.

Table 1. CasC Plugin Management 1.0 vs CasC Plugin Management 2.0
	CasC 2.0 (`apiVersion:2.0`)	CasC 1.0 (`apiVersion:1.0`)
CAP plugins	Listed in the `plugins.yaml` file using the plugin IDs.	Listed in the `plugins.yaml` file using the plugin IDs.
Non-CAP plugins	Non-CAP plugins can be defined in the `plugins.yaml` file for operations center CasC bundles, and in the `plugins.yaml` and `plugin.catalog.yaml` files for controller CasC bundles. The source from where download them can be configured either in the `plugins.yaml` or in the `plugin-catalog.yaml` files. If the same plugin appears in both files, the configuration in the `plugin-catalog.yaml` file prevails.	Only for controller CasC bundles. Plugins to install are listed in the `plugins.yaml` and they must be configured in the `plugin-catalog.yaml` file.
Dependencies	Managed by CasC for CAP and non-CAP plugins. CasC installs all the mandatory dependencies for CAP and non-CAP plugins.	Managed by CasC for CAP plugins (mandatory dependencies). If, in a controller, you plan to install a non-CAP plugin using the `plugin-catalog.yaml` file, dependencies and the plugin itself, must be also included in the `plugins.yaml` file.
Download methods	Plugins included in the `plugins.yaml` file can be retrieved: From the Update Center with no aditional configuration. This is the default method. From an URL using the proper credentials if needed. Only for non-CAP plugins. Using Maven GAV coordinates from a repository. The repository configuration is also part of the `plugins.yaml` file. Only for non-CAP plugins. From the local filesytem. Only for non-CAP plugins. Plugins included in the `plugin-catalog.yaml` file, which is only available for controllers, can be downloaded: From the Update Center if the user sets the version. Directly from an URL if the user configures the URL.	Always from the Update Center for CAP plugins in the `plugins.yaml file`. Plugins included in the `plugin-catalog.yaml` file, which is only available for controllers, can be downloaded: From the Update Center if the user sets the version. Directly from an URL, if the user configures the URL.
Plugin versions and updates	New versions available for non-CAP plugins in the `plugin.yaml` file appear on the Plugin Manager. If the download location for those plugins is the Update Center those plugins are automatically updated on restart. Versions in the `plugin-catalog.yaml` are pinned, and no update appears in the Plugin Manager when a new version is available for the plugins in that file.	The version for CAP plugins is tied to the CloudBees CI version, and the version for non-CAP plugins part of the `plugins.yaml` file, is expressed in the `plugin-catalog.yaml` file. Versions in the `plugin-catalog.yaml` are pinned, and no update appears in the Plugin Manager when a new version is available for the plugins in that file.

Plugin download locations, credentials and repositories

As explained in the previous section, apiVersion:2 bundles allow to download your plugins from different download locations included as a part of the plugin.yaml file.

From the Update Center.
From your CloudBees CI instance local filesystem.
From an URL using credentials in the plugins.yaml file.
Using a Maven GAV coordinates from a Maven repository.

Filesystem location as a plugin download location

- id: "some-plugin-id"
  url: "file://path-to-the-plugin-file/plugin-file.hpi"▼

URL as a plugin download location using credentials

- id: "some-plugin-id"
  url: "http:/some-url/plugin-file.hpi"
  credentialsId: "some-credentials-id"▼

Maven GAV coordinates as a plugin download location

- id: "some-plugin-id"
  groupId: "some-maven-groupId" // Organization in chage of the artifact. I.e: org.jenkins/ci.plugins
  version: "1.0" // Or any other version
  repositoryId: "some-repo-id" // The repository must also be part ot the plugins.yaml file▼

For plugins included in the plugin-catalog.yaml file, you have some limitations related to the supported plugin download locations. The Plugin compatibility page covers these limitations.

Credentials and repositorires

Credentials and repositories used Maven GAV coordinates in the plugins.yaml file must also be included in the plugins.yaml file.

CloudBees CI CasC allows thee different layouts for the repositories used for the Maven GAV coordinates.

JFrog Artifactory
Nexus 2
Nexus 3

Repository examples

repositories:
  - id: "some-artifactory-repo-id"
    layout: "artifactory"
    url: "https://some-artifactory-repo-ul"
  - id: "some-nexus2-repo-id"
    layout: "nexus2"
    url: "https://some-nexus2-url"
    credentialsId: "some-nexus2-credential"
  - id: "some"
    layout: "nexus3"
    url: "https://some-nexus3-url"
    credentialsId: "some-nexus3-credential"▼

CloudBees recommends using the Nexus 3 layout instead of the Nexus 2. Nexus 3 directly downloads the hpi plugin file, whereas Nexus 2 needs to check out all the contents from the repositories and search for the plugin file later. If a Nexus 2 server must be used, then the recommendation is to configure the direct download URL to prevent a performance issue during the start up of the instance.

Credentials can be used to access URLs and repositories, as described in the example above. Supported credentials types in the plugins.yaml file are username and password, and access token. CloudBees recommends injecting the credentials values using system environment variables (or secrets passed as environment vars to the pods) instead of using the values as direct plain text.

Credential examples

credentials:
 - id: "cred1"
   user: "user1"
   password: "${PASSWORD}"  // It can also be a Personal Access Token(PAT)
 - id: "cred2"
   user: "user2"
   password: "PLAIN_TEXT_PASSWORD" // Not Recommended
 - id: token1
   token: "${TOKEN}"▼

Examples and file formats

The `plugins.yaml` file

apiVersion:1 CasC bundles

apiVersion:2 CasC bundles

plugins:
  # In CloudBees Assurance Program (CAP)
  - id: "bitbucket-branch-source"
  - id: "branch-api"
  - id: "cloudbees-casc-client"
  - id: "cloudbees-casc-items-api"
  - id: "cloudbees-casc-items-commons"
  - id: "cloudbees-casc-items-controller"
  - id: "cloudbees-template"
  - id: "cloudbees-workflow-template"
  - id: "configuration-as-code"
  - id: "git"
  - id: "github-branch-source"
  - id: "infradna-backup"
  - id: "support-core"
  - id: "workflow-multibranch"
  # Not in CAP (see plugin-catalog.yaml)
  - id: "manage-permission"▼

For apiVersion:1 CasC bundles, the file only contains plugins IDs. Remember that those plugins included in the plugin-catalog.yaml and their dependencies, must be also part of the plugins.yaml file.

Non-CAP plugins can only be part of the plugins.yaml for controller CasC bundles.

plugins:
  # CAP plugins
  - id: "configuration-as-code"
  - id: "cloudbees-casc-client"
  - id: "cloudbees-casc-items-commons"
  - id: "cloudbees-casc-items-controller"
  # Non-CAP plugin from the Plugin Catalog
  - id: "beer"
    url: "https://repo.jenkins-ci.org/artifactory/releases/org/jenkins-ci/plugins/beer/1.2/beer-1.2.hpi" # (URL ignored as this plugin is defined in the plugin-catalog.yaml)
  # Non-CAP plugin using the URL to download as file
  - id: "ansicolor"
    url: file:///some-path/ansicolor-1.0.1.hpi # Notice the file:// protocol needed to differentiate from the http(s) protocol
  - id: "chucknorris"
  # Non-CAP plugin using GAV coordinates
  - id: "extended-read-permission"
    groupId: "org.jvnet.hudson.plugins"
    version: "53.v6499940139e5"
    repositoryId: "myRepo"
  # Non-CAP plugin using the UC
  - id: "pipeline-graph-view"
  # For the update allowing the reload
  - id: "git"
#Repositories
repositories:
  - id: "myRepo"
    layout: "nexus3"
    url: url: "https://some-nexus3-url"
    credentialsId: "me"
# Credentials
credentials:
  - id: "me"
    user: "some-user"
    password: ${PASSWORD} # Read from Environment Variables - It can be the password or a PAT▼

The `plugin-catalog.yaml` file.

The plugin-catalog.yaml contains the plugin catalog definitions that should be created in the controller.

apiVersion:1 CasC bundles

apiVersion:2 CasC bundles

type: "plugin-catalog"
version: "1" (1)
name: "my-plugin-catalog"
displayName: "My Plugin Catalog"
configurations:
  - description: "Extensions to CAP"
    includePlugins:
      manage-permission: (2)
        version: 1.0.1   (3)
      my-custom-plugin:
        url: http://www.example.org/jenkins-plugins/my-custom-plugin-1.2.3.hpi (4)
    beekeeperExceptions: (5)
      support-core:
        version: "2.70"▼

1	`version` (required) - must currently have the value 1, which defines the current metadata format for the plugin catalog.
2	Add the `manage-permission` plugin to enable Overall/Manage permission.
3	Install version `1.0.1`. As this example is an `apiVersion:1` CasC bundle, the plugin ID must be part of the `plugins.yaml` file.
4	Plugin direct download URL.
5	Beekeeper plugin exception to install a different version for a plugin under CAP.

type: plugin-catalog
version: '1' (1)
name: casc-catalog
displayName: casc Catalog
configurations:
- description: casc
  includePlugins:
    beer:  (2)
      version: "1.0"  (3)
   beekeeperExceptions: (4)
      support-core:
        version: "2.70"▼

1	`version` (required) - must currently have the value 1, which defines the current metadata format for the plugin catalog.
2	Add the `beer` plugin.
3	This plugin was previously defined as a part of the `plugin.yaml` file using and URL and version `1.2`. Version `1.0`, which is defined in this plugin catalog, prevails.
4	Beekeeper plugin exception to install a different version for a plugin under CAP.

The plugin-catalog.yaml is only part of the controller CasC bundles

Bundle inheritance and CasC plugin management

By default parent and child CasC bundles must have the same value for the apiVersion property. Different apiVersion values are only allowed if the property com.cloudbees.casc.server.mergeStrategy.allowMultipleApiVersions is set.

Plugin management with CasC allows you to create CasC bundles with differente values for the apiVersion property in the bundle.yaml file. A different apiVersion implies a different structure for the files. This difference can impact bundle inheritance when ancestors and descendants use different apiVersion values.

Regarding the apiVersion values and bundle inheritance:

Bundle inheritance is only allowed if the descendants’s apiVersion is equal or higher than the ancestor’s.
If parent(ancestor) and child(descendant) bundles have the same apiVersion, the expected behavior is the default behavior for the used apiVersion.
When ancestors and descendant are merged to create and effective bundle, the effective bundle has the same apiVersion as the latest child in the hierarchy.

As expressed here, when the effective bundle is calculated, the child and parent bundles are merged following some strategies. As child and parent can now have different values for the apiVersion property, additional aspects must be considered when the merge is performed.

If the child bundle has apiVersion:1, the effective bundle also has apiVersion:1, and the plugins.yaml file contains a list of non-duplicated plugin IDs.
Merge strategy for the plugin-catalog file remains as described here. If present on both bundles, only the plugin-catalog file in the child bundle is processed.
If the child bundle has apiVersion:2, the effective bundle also has apiVersion:2. If a plugin is present on both bundles, the download method defined at the child bundle prevails.

Instance hot reload with CasC plugin management

When your CloudBees CI instance has been configured using a CasC bundle, and a new version for that bundle is available, you can update your instance utilizing this bundle new version.

Under some circumstances, you can perform what is called a Hot Reload by selecting the Reload Configuration button, if present, on the Bundle update tab of the CloudBees Configuration as Code export and update screen.

Figure 1. Use the Reload Configuration button, if present, to perform a Hot Reload

When you select that button, a Hot Reload is performed, and CloudBees CI applies the new configuration described in the CasC bundle update without restarting the instance.

It doesn’t matter if you use apiVersion:1 or apiVersion:2 bundles, a Hot Reload is only possible if none of the plugins installed gets a new version with the new CasC update process.

For apiVersion:1 bundles, new versions for non-CAP plugins may appear only in the plugin.catalog.yaml file, where you pin the desired version for the plugin.

For apiVersion:2 bundles, new versions for non-CAP plugins may also appear in the plugins.yaml file by means of changes in the different download locations or even changes in the version provided by the Update Center.

Versions for CAP plugins are tied to the CloudBees CI instance version and can only change when there is a CloudBees CI upgrade that, in any case, requires a restart.

CasC plugin report

When you configure your CloudBees CI instance using CasC, a CasC plugin report with all the effectively installed plugins is created in the plugin-installation-report.json file within you $JENKINS_HOME folder.

This report has a JSON format and contains:

The apiVersion used for the bundle in the bundle-api-version property.
A list with ids and the versions for the bootstrap plugins.

A requested list containing an entry for the rest of the installed plugins with:

The plugin id.
The plugin version.
If the plugin is CAP or non-CAP.
The dependencies for the plugin.

The reason why the plugin has been installed: As a part of the bundle, as a dependency, or installed outside CAP.

plugin-installation-report.json example

{
  "bundle-api-version" : "2",
  "bootstrap" : {
    "apache-httpcomponents-client-4-api" : "4.5.14-208.v438351942757",
    "asm-api" : "9.6-3.v2e1fa_b_338cd7",
    "bouncycastle-api" : "2.30.1.77-225.v26ea_c9455fd9",
    "caffeine-api" : "3.1.8-133.v17b_1ff2e0599",
    "cloudbees-administrative-monitors" : "1.0.13",
    "cloudbees-analytics" : "1.59",
    "cloudbees-assurance" : "2.276.0.35",
    "cloudbees-folder" : "6.879.990.cb-v8176690e44b_9",
    "cloudbees-folders-plus" : "3.31",
    "cloudbees-license" : "9.80",
    "cloudbees-platform-common" : "1.27",
    "cloudbees-platform-data" : "1.39",
    "cloudbees-plugin-usage" : "2.20",
    "cloudbees-uc-data-api" : "4.59",
    "cloudbees-unified-ui" : "1.31",
    "command-launcher" : "107.v773860566e2e",
    "commons-lang3-api" : "3.13.0-62.v7d18e55f51e2",
    "commons-text-api" : "1.11.0-95.v22a_d30ee5d36",
    "credentials" : "1337.v60b_d7b_c7b_c9f",
    "display-url-api" : "2.200.vb_9327d658781",
    "handy-uri-templates-2-api" : "2.1.8-30.v7e777411b_148",
    "instance-identity" : "185.v303dc7c645f9",
    "ionicons-api" : "56.v1b_1c8c49374e",
    "jackson2-api" : "2.17.0-379.v02de8ec9f64c",
    "jakarta-activation-api" : "2.1.3-1",
    "jakarta-mail-api" : "2.1.3-1",
    "javax-activation-api" : "1.2.0-6",
    "javax-mail-api" : "1.6.2-9",
    "jaxb" : "2.3.9-1",
    "jdk-tool" : "73.vddf737284550",
    "jjwt-api" : "0.11.5-77.v646c772fddb_0",
    "joda-time-api" : "2.12.7-29.v5a_b_e3a_82269a_",
    "jquery3-api" : "3.7.1-2",
    "json-api" : "20240303-41.v94e11e6de726",
    "json-path-api" : "2.9.0-58.v62e3e85b_a_655",
    "mailer" : "472.vf7c289a_4b_420",
    "mapdb-api" : "1.0.9-28.vf251ce40855d",
    "metrics" : "4.2.21-449.v6960d7c54c69",
    "nectar-license" : "8.43",
    "nectar-rbac" : "5.91",
    "okhttp-api" : "4.11.0-172.vda_da_1feeb_c6e",
    "operations-center-agent" : "3.26942",
    "operations-center-client" : "3.26942",
    "operations-center-context" : "3.26942",
    "scm-api" : "689.v237b_6d3a_ef7f",
    "script-security" : "1326.vdb_c154de8669",
    "snakeyaml-api" : "2.2-111.vc6598e30cc65",
    "structs" : "337.v1b_04ea_4df7c8",
    "support-core" : "1421.v1a_899a_c02a_54",
    "token-macro" : "400.v35420b_922dcb_",
    "user-activity-monitoring" : "1.18",
    "variant" : "60.v7290fc0eb_b_cd",
    "workflow-api" : "1291.v51fd2a_625da_7",
    "workflow-step-api" : "657.v03b_e8115821b_",
    "workflow-support" : "881.v7663695646cf"
  },
  "requested" : {
    "ansicolor" : {
      "id" : "ansicolor",
      "version" : "1.0.1-rc422.90f4bfb7b5a7",
      "cap" : false,
      "dependencies" : {
        "workflow-api" : "2.42",
        "workflow-step-api" : "2.23"
      },
      "reason" : "bundle"
    },
    "ant" : {
      "id" : "ant",
      "version" : "497.v94e7d9fffa_b_9",
      "cap" : true,
      "dependencies" : {
        "structs" : "324.va_f5d6774f3a_d"
      },
      "reason" : "dependency or installed outside CasC"
    }
    // ...
  }
}▼