Parallelizing your test suite with the CloudBees Smart Tests CLI

2 minute read

To parallelize your test suite, you can take advantage of the parallelization feature built into smart-tests subset . smart-tests subset 's primary purpose is Predictive Test Selection, but here, we’re using --target 100% to skip the test selection part but keep the parallelization feature.

  1. Record your test results via Sending data to CloudBees Smart Tests .

  2. Kick off the process by running smart-tests subset with the --split option and --target 100% . The command will output an ID string you should save and pass into each runner. See Requesting and running a subset of tests for more details about this command.

  3. Start up your parallel test worker(s).

  4. Request the bin of tests that the worker should run. To do this, run smart-tests split-subset with: . the --subset-id option set to the ID you saved earlier, and

  5. the --bin value set to bin-number/bin-count . For example, to split your test suite across 3 workers, use 1/3 , 2/3 , etc.

  6. If you’re using Zero Input Subsetting , add the --output-exclusion-rules option.

  7. Run the tests on each worker as outlined in Requesting and running a subset of tests .

  8. After each run finishes in each worker, record test results using smart-tests record tests with the --subset-id option set to the ID you saved earlier.

In pseudocode:

# main
$ smart-tests record build --build $BUILD_ID --source src=.
$ smart-tests record session --build $BUILD_ID --session $SESSION_ID
$ smart-tests subset bazel --split --target 100% --build $BUILD_ID --session $SESSION_ID .
subset/12345

...

# worker 1
$ smart-tests split-subset bazel --subset-id subset/12345 --bin 1/3 > worker.txt
$ bazel test $(cat worker.txt)
$ smart-tests record tests --build $BUILD_ID --session $SESSION_ID --subset-id subset/12345 bazel .

# worker 2
$ smart-tests split-subset bazel --subset-id subset/12345 --bin 2/3 > worker.txt
$ bazel test $(cat worker.txt)
$ smart-tests record tests --build $BUILD_ID --session $SESSION_ID --subset-id subset/12345 bazel .

# worker 3
$ smart-tests split-subset bazel --subset-id subset/12345 --bin 3/3 > worker.txt
$ bazel test $(cat worker.txt)
$ smart-tests record tests --build $BUILD_ID --session $SESSION_ID --subset-id subset/12345 bazel .
▼
Introduction