Move/Copy/Promote

Starting with the operations center Context plugin version 1.7.1, enhanced Move and Copy operations for Jobs, Folders, and other items are available. This functionality enhances and complements the facility to "promote" job configurations from one location to another — between client controllers or within the same client controller added in operations center 1.7.

Each one of the three operations may or may not be supported for a given item type. The following table gives a summary of this support:

Table 1. Supported operations per item type
Item type	Move	Copy	Promote ^[1] 1. will be replaced by "copy without build history"
Freestyle Job
Maven Job
Multi Configuration Job (Matrix)
Pipeline Job
Pipeline Multi Branch Job
Folder
GitHub Org
Bitbucket Team / Project
Backup
Long Running Project
Template

Table 2. Supported operations per item type, specific to operations center
Client controller
Shared Agent
Shared Cloud
Update Center
AWS EC2 Cloud Configuration
Cluster Operation

The Move operation is not supported for managed controllers within the operations center. Attempting this type of Move operation could result in the loss of functionality and data. However, the Move operation is supported when moving items within a managed controller.

Jenkins does not have the Folders plugin installed by default. Without any folders to move jobs between, it does not make sense to have a move concept. When you install the Folders plugin, this plugin adds a Move operation to all items that can be moved.

Out of the box, Jenkins does not have capabilities for copying jobs between controllers. Again, in the absence of folders and when limited to a single controller, the major use case for copying a job is to create a new job with the same configuration as an existing job.

That use case is provided for by Jenkins using New Item Copy an existing Item. When you install the Folders plugin in Jenkins, there are some use cases for copying a job with its entire build history. Typically those use cases are solved by the Jenkins administrator manually copying the job within the JENKINS_HOME directory structure and restarting Jenkins.

With the introduction of operations center, however, there now are a lot more use cases for Move, Copy, and Promote operations:

Moving jobs between controllers to distribute the load between multiple controllers
Copying a job from one controller to another to verify that the job and its associated history is transferred correctly before removing the original
Copying an example folder with jobs to seed a project initiation
Copying a job from a production controller to a test controller to assess the impact of a plugin upgrade
Copying a job from a test controller to a production controller and skip builds from the copy

Within the above context there is also an additional set of use cases for some customers who have more strict change control processes. Such customers typically will allow users to configure a job in one environment and then move the changes to that job to production. In essence they are configuring the job and then promoting the configuration of that job towards production (typically through a test environment).

Operations center 1.7 introduced the Promote Configuration operation to support this last use case. Due to the nature of a "promotion" operation, the support for this has to be specifically developed for each job type. At the time of writing, configuration promotion support has only been implemented for Freestyle jobs.

With Operations Center Context version 1.7.1, the Promote Configuration operation has been subsumed by the new Jenkins Move/Copy/Promote operation. This new operation also subsumes the Move operation that is provided by the Folders plugin.

Figure 1. The Move/Copy/Promote operation provided by the Operations Center Context plugin, replacing the Move operation provided by the Folders plugin.

Bear in mind the following before you proceed:

Upgrade all client controllers to at least Operations Center Context 1.7.1 before using the move/copy/promote functionality between client controllers.
To be able to move an item from one controller to another, you would need to update the client controller security section on operations center, setting the authentication mapping to any of the 'trusted with equivalent' security realm options (both restricted controller with equivalent security realm and trusted controller with equivalent security realm will work). You can choose the one you need in the JENKINS_URL/configureSecurity page.

Moving, copying, or promoting items using the web UI

To move/copy/promote a job / folder / item — browse to the item and select the Move/Copy/Promote action from the menu. You will see a screen similar to the following:

Figure 2. The initial screen. The default destination reflects the current location of the item to be moved (in this case a folder called "widgets")

Note the following about this screen:

The path browser defaults to the current location of the item.
The path browser always corresponds to the location containing the item. If there is already an existing item with the same name at the selected destination, then the move operation fails.
The three buttons at the bottom of the screen reflect your capabilities for the current item. For example, if support for promotion of the specific type of item has not been written, then the Promote button will be disabled.

Using the path browser you can navigate to the location you want to move/copy/promote the item to.

This functionality is available even on a standalone controller, although in such cases the operation is limited to that controller.

Once the correct destination has been selected, click on the desired operation:

The Move button will move the current item with all its build history to the new location. The original item will be removed on confirmation of the item having been successfully received at the destination.
The Copy with builds (just Copy in older versions) button will copy the current item with all its build history to the new location. The original item with all its build history will also be retained at its original location.
When using Copy without builds the build history is not copied to the new location. This option has better in performance when there are lots of builds and artifacts to copy and there is no interest in copying them.
The Promote button will copy the configuration without any build history of the item to the destination. If an item of the same name already exists at the destination, that item’s configuration will be replaced by the source item’s configuration. The build history of the source item and any pre-existing destination item will remain unchanged by the promotion.

The first thing that will happen is that Jenkins will attempt to validate the operation. There are a list of checks that are performed. These checks depend on the type of operation, the type of item, as well as whether the destination is to a different Jenkins instance. The checks are sent to the destination controller for validation.

Figure 3. Initial validation checks in progress

If the destination controller reports all checks were successful then the operation will be submitted to the Jenkins build queue.

Figure 4. A move/copy/promote request having been completed validation successfully and accepted into the build queue automatically.
If the destination controller reports warnings for some checks, then a confirmation screen will be shown:

Figure 5. A move/copy/promote request where the validation identified some problems (in this case there are some missing plugins and the destination controller is running an older version of Jenkins)

The user can then decide whether to continue the operation or abort.

If the user selects Continue anyway, all warnings will be ignored.

If the destination controller reports errors for some checks, then a confirmation screen will be shown:

Figure 6. A move/copy/promote request where the validation indicates that the request will fail (the error handling defaults to the safe option of "Stop on warnings or errors" to prevent accidental selection of the Continue button)

This screen differs from the case of warnings as the user must explicitly select the error handling behavior. The default error handling on this screen is to stop for warnings or errors found during the pre-flight check phase.

If validation errors are reported then the operation will almost certainly not succeed.

In certain cases it may be possible to copy or promote by ignoring warnings or ignoring both errors and warnings.

Do not attempt a move operation when ignoring errors and warnings unless you have a current backup of the item being moved as when errors are ignored the move operation will always delete the source item.

Figure 7. A move/copy/promote operation in the running post-validation checks.

The move/copy/promote operations are injected into the Jenkins build queue as they can be potentially long running operations:

Move operations need to ensure that the job is not currently building and also that the job does not try to start building during the move operation.
Move operations acting on a folder need to ensure that all jobs contained in the folder are not currently building.
Copy and move operations may involve sending significant quantities of data between controllers, this can take quite some time if there are a lot of large archived artifacts.

Figure 8. A move/copy/promote operation waiting for confirmation that the files have been received by the destination.
Any of the move/copy/promote operations, when interacting with a remote controller, need to allow for the possibility that the destination controller may experience a temporary network outage during the operation.

Once the operation has been submitted to the Jenkins build queue, the progress dialog can be closed if the user wants to do other things.

When the operation completes, then progress dialog will report the results.

Figure 9. A successfully completed move/copy/promote request. Clicking Close will redirect the browser to the destination item.

Figure 10. A failed move/copy/promote request. If the destination item exists then clicking Close will redirect the browser to the destination item, otherwise you will be returned to the source item.

If the destination item exists then the user will be taken to the destination item when they close the progress dialog.

Moving, copying, or promoting items using the Jenkins CLI

The Operations Center Context plugin also adds three commands to the Jenkins CLI:

cjp-move-item
cjp-copy-item
cjp-promote-item

All three of these commands use the same syntax:

java -jar jenkins-cli.jar -s _JENKINS_URL_ cjp-____-item _/path/to/source/item_ _uri/of/destination/item_▼

They also support the same command line options:

-q: Submit the request and return immediately.
-w: Wait until the request has started.
-v: Display verbose output of progress.
-e IGNORE_ALL | IGNORE_WARNINGS | IGNORE_NONE: Select the error handling mode.

Except for --skip-builds which is only valid for the cjp-copy-item command. This option will skip the build history from the copy operation (as it does "Copy without builds") in the UI.

The source item is specified by providing the path to the item on the source controller. This is a normal path for selecting items using the Jenkins CLI.

The destination folder is specified by providing a URI for the destination. At present there are two supported URI schemes:

jenkins://instanceid/path/to/folder.

If you are referencing a path on the same controller as the source you can use jenkins://./path/to/folder.
cjp:///path/from/root/of/cjoc

In general, with all of these paths can be constructed by building the names of each item. Most Jenkins items support using a different display name for the item from the actual name that the item is known as. This can complicate determining the actual path or URI.

The Operations Center Context plugin adds a hidden action to all items: /platform-uri/ that returns the uri of the item.

For example, if you use a web browser and navigate to the destination folder, the browser location will be something like:

https://jenkins.dev.example.com/job/widget/job/blue▼

By changing the location to:

https://jenkins.dev.example.com/job/widget/job/blue/platform-uri/▼

Get back the jenkins:// URI for the "Jenkins » Widgets » Blue" folder.

If you change the location to

https://jenkins.dev.example.com/job/widget/job/blue/platform-uri/?scheme=cjp▼

Get back the cjp:/// URI for that folder. On a standalone controller that would be cjp:///widget/blue but if the controller is joined to operations center then the path will be relative to the root of operations center, for example: cjp://controllers/dev/widget/blue.

cjp:// URIs are much easier to work with, but can be broken by moving the client controllers within operations center.

jenkins:// URIs can be cumbersome to work with but have the advantage that they are not affected by moving the controller within operations center as the controller is looked up using its instance id.

Promoting a job from one controller to another

First determine the URI of the destination, in this case use the curl command:

$ curl http://jenkins.prod.example.com/job/widgets/job/blue/platform-uri/?s\
cheme=cjp
cjp:///masters/production/widgets/blue

$▼

Now that you have the URI of the destination use the CLI command to promote the job configuration:

$ java -jar jenkins-cli.jar -s http://jenkins.dev.example.com/ cjp-promote-\
item /widgets/blue/deploy cjp:///masters/production/widgets/blue
Started masters » dev » widgets » blue » deploy ↑ masters » production » widget
s » blue » deploy
Completed masters » dev » widgets » blue » deploy ↑ masters » production » widg
ets » blue » deploy : SUCCESS

$▼

To roll the two steps into one use $(curl …):

$ java -jar jenkins-cli.jar -s http://jenkins.dev.example.com/ cjp-promote-\
item /widgets/blue/deploy $(curl -L http://jenkins.prod.example.com/job/wid\
gets/job/blue/platform-uri/?scheme=cjp)
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed
0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
0     0    0    39    0     0   3296      0 --:--:-- --:--:-- --:--:--  3296
Started masters » dev » widgets » blue » deploy ↑ masters » production » widget
s » blue » deploy
Completed masters » dev » widgets » blue » deploy ↑ masters » production » widg
ets » blue » deploy : SUCCESS

$▼

To ask for detailed progress use -v:

$ java -jar jenkins-cli.jar -s http://jenkins.dev.example.com/ cjp-promote-\
item /widgets/blue/deploy cjp:///masters/production/widgets/blue -v
Started masters » dev » widgets » blue » deploy ↑ masters » production » widget
s » blue » deploy
Promotion Starting at: 14/08/15 09:12 triggered by: anonymous
Source: masters » dev » widgets » blue » deploy
Destination: masters » production » widgets » blue » deploy
Error Handling: IGNORE_NONE
        Running pre-flight checks...
        Pre-flight checks: PASS
        Sending files...
        Tray created: DumbWaiterTray{shelf='c033b4a3-1462-4b54-a2f2-10be3c57e30
e', urls=[]}
        Tray is prepared: DumbWaiterTray{shelf='c033b4a3-1462-4b54-a2f2-10be3c5
7e30e', urls=[]}
        Requesting post-validation...
        Post-flight Validation completed.
        Performing Backup
        Backup completed
        Running promotion...
        Promotion completed
        Tidying up...
        Disposing of tray: DumbWaiterTray{shelf='c033b4a3-1462-4b54-a2f2-10be3c
57e30e', urls=[]}

Promotion finished with success at 14/08/15 09:12.
Completed masters » dev » widgets » blue » deploy ↑ masters » production » widg
ets » blue » deploy : SUCCESS

$▼

To request that the command returns as soon as the operation starts rather the default of than waiting for the operation to complete use the -w option:

$ java -jar jenkins-cli.jar -s http://jenkins.dev.example.com/ cjp-promote-\
item /widgets/blue/deploy cjp:///masters/production/widgets/blue -w
Started masters » dev » widgets » blue » deploy ↑ masters » production » widget
s » blue » deploy

$▼

To ask for the operation to be queued only use the -q option:

$ java -jar jenkins-cli.jar -s http://jenkins.dev.example.com/ cjp-promote-\
item /widgets/blue/deploy cjp:///masters/production/widgets/blue -q

$▼

You can also control the error handling behavior. By default the operations will abort if there are any errors or warnings. When transitioning between masters it is not uncommon to find plugin version differences between the masters. Plugin version skew will typically manifest as a WARNING.

$ java -jar jenkins-cli.jar -s http://jenkins.dev.example.com/ cjp-promote-\
item /widgets/blue/deploy cjp:///masters/production/widgets/blue -v
Started masters » dev » widgets » blue » deploy ↑ masters » production » widget
s » blue » deploy
Promotion Starting at: 14/08/15 09:29 triggered by: anonymous
Source: masters » dev » widgets » blue » deploy
Destination: masters » production » widgets » blue » deploy
Error Handling: IGNORE_NONE
        Running pre-flight checks...
        Pre-flight checks: WARNING
Aborting
        Tidying up...

Promotion finished with problems at 14/08/15 09:29
Completed masters » production » widgets » blue » deploy ↑ masters » dev » widg
ets » blue » deploy : FAILURE

$▼

The Jenkins Administrator can inspect the detailed logs of the request using Jenkins Manage Jenkins Move/Copy/Promote History on the source Jenkins instance.

Figure 11. The detailed logs of a failed promote operation accessed via Jenkins Manage Jenkins Move/Copy/Promote History

Figure 11. The detailed logs of a failed promote operation accessed via Manage Jenkins Move/Copy/Promote History

In this case there are some missing plugins.

There are some plugins that store side information about a project in a job property.

Typically, these can be identified by examining the job configuration or the config.xml of the job itself.

A job that is deploying things may actually be using the Deployer framework plugin to do the deployment.

While, in general, this specific plugin may inject a mostly unused property into jobs, you need confirmation that this deploy job is not actually using the Deployer framework plugin.

The easiest way to get that confirmation would be to consult with the creator / maintainer of the job. If the creator / maintainer is unavailable or unsure of the answer, determine the answer by inspecting the configuration of the job.

Look at the deploy job’s configuration:

Figure 12. Example deploy job’s configuration

In this case, the job is using the Copy artifacts from another project and Execute shell build steps. It doesn’t seem likely that this job is actually using the Deployer framework plugin.

Look at the deploy job’s raw config.xml just to get final confirmation:

Figure 13. Example deploy job’s config.xml

Looking at the config.xml you can see that the main functionality is actually provided by three steps:

The SSH Agent plugin provides a build wrapper that makes the deploy-key credentials available - this is using a credential with an ID that has been explicitly specified so that development can provide a credential for their server while production can, using the same job configuration, provide the production credentials on the production server.
The Copy Artifacts plugin copies the widget from the upstream build job into the workspace.
The Execute shell build step uses scp to copy the built widget to the ${TARGET_HOST} - which is presumably an environment variable set on a parent folder so that development can test against a non-production server.

There is a property in the configuration file from the Deployer framework and that is what is causing the warning. The property itself, though, does not seem to contain any significant configuration:

The single click deployment functionality has been recorded as disabled.
There are no default targets for the the Deploy Now action.

After analysis, you can conclude that this job is safe to promote into production even with the reported warnings.

$ java -jar jenkins-cli.jar -s http://jenkins.dev.example.com/ cjp-promote-\
item /widgets/blue/deploy cjp:///masters/production/widgets/blue -v -e IGNO\
RE_WARNINGS
Started masters » dev » widgets » blue » deploy ↑ masters » production » widget
s » blue » deploy
Promotion Starting at: 14/08/15 10:03 triggered by: anonymous
Source: masters » dev » widgets » blue » deploy
Destination: masters » production » widgets » blue » deploy
Error Handling: IGNORE_WARNINGS
        Running pre-flight checks...
        Pre-flight checks: WARNING
        Sending files...
        Tray created: DumbWaiterTray{shelf='ba906833-1e1b-4e85-8f9c-5477f50865d
5', urls=[]}
        Tray is prepared: DumbWaiterTray{shelf='ba906833-1e1b-4e85-8f9c-5477f50
865d5', urls=[]}
        Requesting post-validation...
        Post-flight Validation completed.
        Performing Backup
        Backup completed
        Running promotion...
        Promotion completed
        Tidying up...
        Disposing of tray: DumbWaiterTray{shelf='ba906833-1e1b-4e85-8f9c-5477f5
0865d5', urls=[]}

Promotion finished with success at 14/08/15 10:03.
Completed masters » dev » widgets » blue » deploy ↑ masters » production » widg
ets » blue » deploy : SUCCESS

$▼

Resolving common issues

The following issues may be encountered:

Moving templates

If you move a template from one controller to another controller, jobs on the source controller that were based on the template become disconnected from the template, just as if you had deleted the template on a standalone controller.

Moving items based on templates

If you move a job based on a template from one controller to another, the job on the destination controller will become disconnected from the template, just as if you had deleted the template on a standalone controller.

Plugin version skew

The most common warning is from plugin version skew, i.e. where the destination controller has an older version of a plugin used on the source controller.

Typically such skew will not be a major issue as the general principle under which plugins are developed is that you should be able to roll back a plugin upgrade without loosing configuration.

If a plugin maintainers has to break that contract, they are expected to set a flag in the plugin’s manifest that triggers a warning in the update center, e.g.

Figure 14. Warning about a plugin upgrade where the plugin’s configuration may not be retained if the upgrade is rolled back.

The Deployer framework plugin is not installed on the receiving Jenkins

Plugin removal from CAP announcement

CloudBees removed this plugin from the CloudBees Assurance Program (CAP) in February 2023. Please contact CloudBees Support if you have any concerns or questions.

The Deployer framework plugin can often show up as a missing plugin.

For most jobs, the Deployer framework is storing a cache of whether the job has any archived artifacts that can be auto detected as being supported for deployment with any of the installed deployment engines. The cache is required in order to ensure that the views render quickly as the Deploy Now button should only be rendered for jobs that have at least one artifact that can actually be deployed.

Figure 15. A view with two jobs, the most recent build of the build job has archived artifacts that can be deployed now and as a result the Deploy Now button is visible beside the Build Now button. The deploy job does not have any archived artifacts and therefore there is nothing to for the Deployer framework plugin to deploy from that job.

The best way to know if a job is actually using the Deployer framework plugin is to ask the people using the job.

If all of them do not know, then it is probably safe to conclude that they are not using it.

If you have any remaining doubt, to confirm if a job is actually using the Deployer framework you need to check three things:

There are no Deploy applications build steps

Figure 16. A Deploy applications build step.
There is no Deploy applications step in the Post-build Actions

Figure 17. A Deploy applications post-build action.
The Deploy Now defaults do not contain any significant configuration. The most reliable indicator of significant configuration is if the One-Click Deployment option has been enabled, as that is normally only selected when the defaults have been configured with a valid configuration.

If the One-Click Deployment option has not been selected, check if the Advanced button is indicating that there is edited configuration within (i.e. the notepad icon to the left of the button)

Figure 18. An Advanced button where the values differ from the default configuration.

Different deployer engines will inject a sample starting configuration, so often it will be necessary to expand the Advanced button.

If none of the Host service entries contain an Application then the Deploy Now functionality has not been configured.

The deployer engines may auto-detect applications, for example the AWS deployer engine knows that WAR files are supported for deployment onto Elastic Beanstalk. In order to simplify the use of the Deploy Now action, it will configure an empty application if there is a WAR archived artifact, e.g.

Figure 19. An auto-detected application entry

In such cases the lack of customization of the application configuration, example no Application Name or Version specified, the default hint of BucketName/ in the S3 Bucket, etc., all point to this configuration either not being significant or being trivial to reconstruct.

Pipelines

Because a large part of the definition of a Pipeline can be stored in source control, it may not be possible for Jenkins to determine the full list of plugins that are used by the actual Pipeline definition.

This is typically not a serious issue as any of the explicit Jenkins configuration in the Pipeline job (i.e. the parts of the configuration that would be discarded if saved on a Jenkins instance missing a required plugin) will be covered by the validation checks so worst case the Pipeline job can simply be moved to a controller that has the plugins required by the Pipeline script.