DSL Git Synchronization

YAML formatted DSL files is a Preview feature.

A Preview feature:

Has not undergone end-to-end testing with CloudBees products.
Is provided without service-level agreements (SLA), and therefore does not include CloudBees' commitment to functionality or performance.
May impact other stable areas of the product when used.
May have limited documentation.
May not be feature-complete during the Preview period.
May graduate from Preview to a supported feature or be removed from the product.
May introduce breaking changes that prevent upgrades due to incompatibility with future development.

Product features and documentation are frequently updated. If you find an issue or have a suggestion, please contact CloudBees Support.

As of CloudBees CD/RO v2024.03.0, DSL Git Synchronization is installed as part of the standard CloudBees CD/RO Tools. For installation instructions, refer to the CloudBees Tools installation.

The DSL Git Synchronization CLI utility extends the functionality of the generateDsl and evalDsl APIs to help manage and promote DSL across multiple files and directories in a GitOps and IDE friendly manner. This feature applies intelligence to the DSL evaluation based upon Git logs, delete or modify actions and apply only changes.

Additionally, unlike EC-DslDeploy, which requires an agent to run, the dslsync commands are run directly from a CLI without requiring a CloudBees CD/RO agent. While both file formats are similar, the two are not compatible. For more information about EC-DslDeploy, refer to DslDeploy plugin.

The CLI also supports YAML formatted DSL files in addition to the existing Groovy formatted DSL files. However, if no format is specified, the CLI defaults to Groovy formatted DSL files. Format specified examples include:

Groovy format: --format=groovy
YAML format: --format=yaml
Default format (Groovy): --format or no parameter included.

Repositories currently storing configuration data in Groovy DSL should not be migrated to YAML within the same repository. This migration path is unsupported and may lead to unpredictable results. To adopt YAML, CloudBees recommends starting a new repository for the YAML files.

The latest version of dslSync supports exporting in either YAML or Groovy format. However, when migrating between Groovy DSL and YAML DSL, the DSL files are renamed. In Git, this renaming is tracked as file deletions, which may appear as though all objects have been removed. If the --commit-hash-property (used to track changes) is not initialized, this migration can result in the deletion of the entire top-level object being synchronized.

To avoid this issue:

Update the repository with the YAML code (if transitioning to this format from Groovy).
Clear the existing value of the --commit-hash-property, before running dslSync apply.
Re-run with --commit-hash-property to update it correctly.

DSL Git Synchronization configurations

Use the information in these sections to configure the DSL Git Synchronization CLI.

Enable bash-completion for dslsync.
Generate the DSL Git synchronization configuration file.

Enable `bash-completion` for `dslsync`

Use the generate-completion command parameter to enable Bash-completion.

Generate the bash-completion script file by running the following command. Replace <dslsync_completion_script_dir> with the directory where the bash completion script will be generated.
```
dslsync generate-completion > <dslsync_completion_script_dir>/dslsync_completion.sh▼
```
Copy the bash-completion script.
- If bash-completion is installed, copy the bash-completion script file into one of the following directories:
  - /etc/bash_completion.d
  - /usr/local/etc/bash_completion.d
- If bash-completion is not installed.
  1. Create directory bash_completion.d in your home directory.
    
    mkdir ~/bash_completion.d
    ▼
  2. Copy the bash-completion script file into the new directory.
  3. Enter the command below into the ~/.bashrc or ~/.zshrc file.
    
    for bcfile in ~/bash_completion.d/* ; do . $bcfile done
    ▼
Start using bash-completion in one of the following ways.
- Source the ~/.bash_profile or ~/.zshrc file.
- Launch a new terminal.

Generate the DSL Git synchronization configuration file

Use the information in this section to generate the dslsync.yml configuration file. This file enables DSL code synchronization between the CloudBees CD/RO servers and Git repositories.

Check out the Git repository <initial-source> branch where the CloudBees CD/RO DSL files will be generated.
- If the <initial-source> branch exists. e.g. dev
  cd <git-repo-directory> git checkout <initial-source>
  ▼
- If the <initial-source> branch does not exist.
  cd <git-repo-directory> git branch <initial-source> git push -u origin <initial-source> git checkout <initial-source>
  ▼

Generate DSL for a project object from the CloudBees CD/RO initial-source instance into the dsls sub-directory of the Git repository.

dslsync generate \
 --target-directory=dsls \
 --server=<cdro-initial-source-instance> \
 --object-type=project \
 --object-name="<project-name>" \
 --overwrite-object-directory
 --user=<admin> --password▼

dslsync generate \
 --target-directory=dsls \
 --server=<cdro-initial-source-instance> \
 --object-type=project \
 --object-name="<project-name>" \
 --format=yaml \
 --overwrite-object-directory
 --user=<admin> --password▼

Commit and push the generated DSL to the Git repository.

git add dsls
git commit -m "DSLs from initial source instance"
git push▼

Run the git log command to get the latest commit SHA.

git log -1 --stat # to get <latest-commit-SHA-for-repo-branch>▼

Locate and copy the commit SHA for use in the next step.
Run the command to create a configuration file in the directory specified by --path with the SHA content file path specified by --commit-hash-property.

If --path=dsls and --commit-hash-property=/server/dslsync/<file-name>, the following file and property will be created:
- System-generated configuration file: <git-repo-directory>/dsls/.cloudbees/dslsync.yml
- SHA content: /server/dslsync/<property-name>
  Groovy format
  YAML format
  dslsync apply \ --path=dsls \ --server=<cdro-initial-source-instance> \ --commit-hash=<latest-commit-SHA-for-repo-branch> \ --commit-hash-property=/server/dslsync/<property-name>
  ▼
  dslsync apply \ --path=dsls \ --server=<cdro-initial-source-instance> \ --commit-hash=<latest-commit-SHA-for-repo-branch> \ --format=yaml \ --commit-hash-property=/server/dslsync/<file-name>
  ▼

Commit the dslsync.yml file into the Git repository <initial-source> branch.

git add dsls/.cloudbees/dslsync.yml
git commit -m "dslsync config with tracker"
git push▼

Promote DSL configuration file from <initial-source> branch to Git branches and CloudBees CD/RO instances (for example, production or QA).

Repeat this step for all environment branches and CloudBees CD/RO instances to be used as target or source of dslsync commands.
```
cd <git-repo-directory>
git checkout <initial-source>
git pull

git checkout <target-branch>
git merge <initial-source>
git push
dslsync apply --path=dsls --server=<cdro-target-branch-instance>▼
```

DSL Git Synchronization commands

`dslsync generate` command

Use dslsync generate command with the parameters listed below to generate DSL scripts for CloudBees CD/RO objects.

Table 1. generate command
Command parameters	Description
`-- h — help`	Show this message and exit.
`-- format`	Specify YAML or Groovy formatted DSL files. If `-- format` is not used, the CLI defaults to Groovy formatted DSL files.
`-- object-name`	The name of the CloudBees CD/RO object to be DSL generated.
`-- object-type`	The type of CloudBees CD/RO object.
`-- overwrite-object-directory`	Delete the parent directory of the object before generating the DSL. If this option is not used and the parent directory is not empty, the DSL will return an error.
`-- password`	The authentication password for CloudBees CD/RO.
`-- port`	Port number to use when connecting to the agent. The default is 8443.
`-- server`	The domain name or IP address of the CloudBees CD/RO server. Enclose the IPv6 address in square brackets. Example: `[<IPv6-ADDRESS>]`
`-`- target-directory`	The directory where the generated DSL files will be created in a nested file structure.
`-- timeout`	The maximum duration of the command execution in seconds.
`-- token`	The authentication access token for CloudBees CD/RO.
`-- user`	The authentication username for CloudBees CD/RO.
`-- v — verbose`	Include verbose output.

`dslsync apply` command

Use dslsync apply to apply DSL scripts from a Git repository to a CloudBees CD/RO server.

Table 2. apply command
Command parameters	Description
`-- dryrun`	Print the evaluated dsl without sending it to the server.
`-- force`	Applies the changes to the CloudBees CD/RO server without prompting or utilizing the contents of the `dslsync_pre.dsl` file.
`-- format`	Specify YAML or Groovy formatted DSL files. If `-- format` is not used, the CLI defaults to Groovy formatted DSL files.
`-- generate-prerun-dsl`	Generates the `dslsync_pre.dsl` file containing changes in the current working directory. These changes are not applied to the server.
`-- h — help`	Show this message and exit.
`-- password`	The authentication password for CloudBees CD/RO.
`-- path`	The Git repository directory containing scripts. The default is the current directory.
`-- resume-from`	Apply DSL starting with the selected script. DSL scripts run prior to this one are ignored. Use this option if the previous run stopped due to an unexpected error.
`-- timeout`	The maximum duration of the command execution in seconds.
`-- token`	The authentication access token for CloudBees CD/RO.
`-- user`	The authentication username for CloudBees CD/RO.
`-- v — verbose`	Include verbose output.

List of compatible DSL API objects

The following DSL API objects are valid values for -- object-type. For DSL API object detail refer to: CloudBees CD/RO DSL Objects

Compatible DSL API Commands
archiveConnector	artifact
ciConfiguration	complianceConfiguration
dataRetentionPolicy	directoryProvider
emailConfig	gateway
group	openIDConnectConfiguration
persona	personaCategory
personaPage	project
reportObjectType	repository
resource	resourcePool
samlIdentityProvider	samlServiceProvider
serviceAccount	tag
user	workspace
zone

Compatible DSL API Commands

archiveConnector

artifact

ciConfiguration

complianceConfiguration

openIDConnectConfiguration

Usage examples

Use the following usage examples to:

Add DSL from CloudBees CD/RO instance to Git repository
Apply DSL from the Git repository to CloudBees CD/RO instance
Promote DSL changes from Git to CloudBees CD/RO instances and a Git repository

Add DSL from CloudBees CD/RO instance to Git repository

Use generate to create DSL of a top-level CloudBees CD/RO object in a Git branch.

Check out the Git repository <target> branch where the CloudBees CD/RO DSL files will be generated.

If the <target-branch> exists. e.g. dev

cd <git-repo-directory>
git checkout <target-branch>▼

If the <target> branch does not exist.

cd <git-repo-directory>
git branch <target-branch>
git push -u origin <target-branch>
git checkout <target-branch>▼

Generate DSL for a CloudBees CD/RO object into the dsls sub-directory of the Git repository.

dslsync generate \
 --target-directory=dsls \
 --server=<cdro-source-instance> \
 --object-type=<top-level-object-type> \
 --object-name="<object-name>" \
 --user=<admin> --password▼

dslsync generate \
 --target-directory=dsls \
 --server=<cdro-source-instance> \
 --object-type=<top-level-object-type> \
 --object-name="<object-name>" \
 --format=yaml \
 --user=<admin> --password▼

Apply DSL from the Git repository to CloudBees CD/RO instance

Use these steps to apply committed DSL file changes from the Git repository branch to a CloudBees CD/RO instance.

Check out the Git repository <source> branch containing the CloudBees CD/RO DSL to be applied.
```
cd <git-repo-directory>
git checkout <source>
git pull▼
```

Apply DSL to the CloudBees CD/RO instance.

dslsync apply --path=dsls \
--server=<cdro-target-branch-instance>▼

dslsync apply --path=dsls \
--server=<cdro-target-branch-instance>
--format=yaml▼

Promote DSL changes from Git to CloudBees CD/RO instances and a Git repository

Check out the Git repository <source> branch containing the CloudBees CD/RO DSL to be promoted.

cd <git-repo-directory>
git checkout <source>
git pull

git checkout <target-branch>
git merge <source>
git push
dslsync apply --path=dsls --server=<cdro-target-branch-instance>▼

Apply DSL to the target CloudBees CD/RO instance.

dslsync apply --path=dsls \
--server=<cdro-target-branch-instance>▼

dslsync apply --path=dsls \
--server=<cdro-target-branch-instance> \
--format=yaml▼