Ensuring record tests always runs

This page relates to Recording test results with the CloudBees Smart Tests CLI.

The smart-tests record tests command must be executed after you run tests.

However, some tools exit the build process as soon as the test process finishes, preventing this from happening.

The way to fix this depends on your CI tool:

Jenkins

For declarative Pipeline jobs, use the post { always { … } } option:

pipeline {
  ...
  sh 'bundle exec rails test -v $(cat smart-tests-subset.txt)'
  ...
  post {
    always {
      sh 'smart-tests record tests <BUILD NAME> [OPTIONS]'
    }
  }
}▼

For scripted Pipeline jobs, the catchError step should be used as described here: https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#catcherror-catch-error-and-set-build-result-to-failure.

Note that this is unnecessary for Maven builds that use the -Dmaven.test.failure.ignore option.

CircleCI

CircleCI has when: always option:

- jobs:
  - test:
    ...
    - run:
        name: Run tests
        command: bundle exec rails test -v $(cat smart-tests-subset.txt)
    - run:
        name: Record test results
        command: smart-tests record tests <BUILD NAME> [OPTIONS]
        when: always▼

GitHub Actions

GitHub Action has if: ${{ always() }} option:

jobs:
  test:
    steps:
      ...
      - name: Run tests
        run: bundle exec rails test -v $(cat smart-tests-subset.txt)
      - name: Record test result
        run: smart-tests record tests <BUILD NAME> [OPTIONS]
        if: always()▼

Bash

If you run tests on your local or other CI, you can use trap :

function record() {
  smart-tests record tests <BUILD NAME> [OPTIONS]
}
# set a trap to send test results to {PRODUCT} for this build either tests succeed/fail
trap record EXIT SIGHUP

bundle exec rails test▼