Managing artifacts with CloudBees Fast Archiving plugin

2 minute read

The CloudBees Fast Archiving plugin uses an rsync-inspired algorithm to transfer archives from agents to a controller. The result is that builds complete faster and network bandwidth usage is reduced.

After a job is built on an agent selected build artifacts in the workspace may be copied from the agent to the controller so those artifacts are archived. The CloudBees Fast Archiving plugin takes advantage of the fact that there are usually only incremental changes between build artifacts of consecutive builds. Such incremental changes are detected and only those changes that need to be sent from the agent to the controller are transferred, thus saving network bandwidth. The algorithm is inspired by rsync’s weak rolling checksum algorithm, which can efficiently identify changes. The changes are gzipped to further reduce network load.

Simply use the regular Archive artifacts option in the Post-build Action section for a build. You can see the result in Running.

Figure 1. Running

Fast archiving only occurs on builds run on an agent, not those run on the Jenkins controller. Also there must have been previous builds which produced artifacts for the new artifacts to be compared to. Otherwise, Jenkins will perform a regular complete artifact transfer. It will do the same if the fast archiver detects that data is corrupted.

Fast archiving is enabled by default when the plugin is first loaded. You can remove this strategy and later re-add it from Manage Jenkins Configure system in the Artifact Management for Builds section.

CloudBees developed an Artifact Manager on S3 plugin. It permits you to archive artifacts in an S3 Bucket, where there is less need to be concerned about the disk space used by artifacts. Refer to Kubernetes on AWS - Cloud ready Artifact Manager for AWS for details.