eMake simulates a serial GNU Make or NMAKE build. Though eMake runs many commands in parallel, the command output (including text written to standard streams and changes to the file system, such as creating or updating files) appears serially in the original order without overlapping. This feature is called transactional output (or “serial order execution”) and is unique to CloudBees Build Acceleration among parallel build systems. This feature ensures standard output streams and underlying file systems always reflect a consistent build execution state, regardless of how many jobs eMake is actually running concurrently on the cluster.
Transactional output is achieved by buffering the results of every command until the output from all preceding commands is written. Buffering means that while the output contents on the standard streams matches GNU Make or NMAKE exactly, the timing of its appearance might be a little unexpected. For example:
-
“Bursty” output – One of the first things you notice when running a build with CloudBees Build Acceleration is that it appears to proceed in bursts, with many jobs finishing in quick succession followed by pauses. This type of output is normal during a highly parallel build because many later jobs might have completed and output is ready to be written as soon as longer, earlier jobs complete. The system remains busy, continuously running jobs throughout the build duration, even if the output appears to have paused momentarily.
-
Output follows job completion – GNU Make and NMAKE print commands they are executing before they are invoked. Because eMake is running many commands in parallel and buffering results to ensure transactional output, command-line text appears with the output from the command after the command has completed. For example, the last command printed on standard output is the job that just completed, not the one currently running.
-
Batch output – As a way to provide feedback to the user during a long-running execution, some commands might write to standard output continuously during their execution. Typically, these commands might print a series of ellipses or hash marks to indicate progress or might write status messages to standard error as they run. More commonly, a job might have several long-running commands separated with echo statements to report on progress during build execution:
For example, consider a rule that uses rsync to deploy output:
install: @echo "Copying output into destination" rsync -vr $(OUTPUT) $(DESTINATION) @echo "done"
With GNU Make, users first see the Copying output echo, then the state information from rsync as it builds the file list, copies files, and finally, they see the done echo as the job completes.
With eMake, all output from this job step appears instantaneously in one burst when the job completes. By the time any output from echo or rsync is visible, the entire job has completed.