Long-running builds

What happens to builds that were running when Jenkins crashes, or is restarted not in "safe" mode (waiting for running builds to complete)? Whether you are using High Availability (active/passive) to start another Jenkins controller, builds of regular projects that were already running will be aborted. The Restart Aborted Builds plugin allows you to find and reschedule them, but for builds of projects which normally take a long time, perhaps hours or even days, this is not enough.

Builds that need to run for an extended period are best implemented with Pipeline and Inserting checkpoints. If builds cannot be implemented with Pipeline and need to run for an extended period and continue running even if the controller restarts, they can be defined as a "long-running build" type.

To address the needs of people who have legacy builds that are too long to interrupt every time a Jenkins agent is reconnected or Jenkins is restarted for a plugin update, CloudBees CI includes the Long-Running Build plugin that provides a "long-running project" type. The configuration is almost the same as for a standard free-style project, with one difference: the part of your build that you want to run apart from Jenkins should be configured as a (Unix) shell or (Windows) batch step. Of course this script could in turn run Maven, Make, or other tools.

If the agent is reconnected or Jenkins restarted during this "detached build" phase, your build keeps on running uninterrupted on the agent machine (so long as that machine is not rebooted of course). When Jenkins makes contact with the agent again, it will continue to show log messages where it left off, and let the build continue. After the main phase is done, you can run the usual post-build steps, such as archiving artifacts or recording JUnit-style test results.

Make a new job and select Long-Running Project and note the Detached Build section. Pick a kind of build step to run—Bourne shell for Unix agents, or batch script for Windows agents—and enter some commands to run in your build’s workspace.

When the project is built, initially the executor widget will look the same as it would for a freestyle project. During this initial phase, SCM checkouts/updates and similar pre-build steps may be performed. Soon you will see a task in the widget with the (detached) annotation. This means that your main build step is running, and should continue running even if the Jenkins server is halted or loses its connection to the agent. (So long as the connection is open, you should see any new output produced by the detached build step in your build log, with a delay of a few seconds.)

The task label will show post steps while any post-build actions are performed, such as archiving artifacts or recording JUnit test results. This phase does not survive a crash: it requires a constant connection from the Jenkins controller to the agent.

There are a number of limitations and restrictions on what Jenkins features work in long-running builds. Generally speaking, anything that works in a freestyle project in the pre-build or post-build phase should also work. But general build steps are not available except as part of the non-detached pre-build phase, and build wrappers will generally not work. Also surviving a restart can only work if Jenkins can reconnect to the exact same agent without that machine having rebooted, so this will generally not work on cloud-provisioned agents. Consult the release notes for more information.