CloudBees Smart Tests’s * Test Notifications* feature notifies developers via Slack when their test sessions finish so they can immediately take action on the results, whether to triage failures or merge a PR. Developers can create subscriptions to receive personal notifications about test sessions run against their branches/pull requests or other test sessions they care about.
Getting started
To set up notifications:
-
Start sending data to CloudBees Smart Tests
-
Install the CloudBees Smart Tests Slack app in your Slack workspace
-
Link your CloudBees Smart Tests account and your Slack account
-
Create your first notification subscription in Slack
Installing the CloudBees Smart Tests Slack app
-
Log into the CloudBees Smart Tests dashboard .
-
Go to Test Notifications.
-
Click the Install Slack App button in the Slack app section. This will open the Slack app authorization flow. You may need to log in to Slack at this stage.
-
Authorize the app to install it.
-
Done! 🎉
Linking your CloudBees Smart Tests and Slack accounts
Once the CloudBees Smart Tests Slack app has been installed, you and your teammates can link your CloudBees Smart Tests and Slack accounts. This lets you set up subscriptions via the app.
|
If you haven’t signed up for CloudBees Smart Tests yet, or if you’re not a member of your team’s CloudBees Smart Tests organization yet, ask a teammate to send you an /docs/concepts/organization/#organization-invitation-link[#Organization invitation link] so you can sign up (if needed) and join. |
You communicate with the app through direct messages:
-
In Slack, click the New message icon in the top section of the left navigation (or use the Command+N or Control+N keyboard shortcut) to compose a new message.
-
In the "To:" field, type CloudBees Smart Tests and select the app from the dropdown list.
-
In the message text field, type /smart-tests link and hit Enter.
-
Click the button in the response message to initiate the flow to connect your CloudBees Smart Tests account to your Slack account.
Creating a notification subscription
After you’ve linked your CloudBees Smart Tests and Slack accounts, you can create your first notification subscription.
Via UI
After linking your account, you can create a new subscription from the home tab of the CloudBees Smart Tests app in Slack:
Via message
You can also create a subscription by sending a message to the CloudBees Smart Tests app. The syntax for creating a subscription is:
/smart-tests subscribe <WORKSPACE> <KEY>=<VALUE>
-
<WORKSPACE>is the workspace’s name containing the test sessions you want to be notified about. -
<KEY>=<VALUE>is a key-value pair that contains the CI environment variable and value that indicates the test sessions you want to subscribe to (e.g.,GITHUB_ACTOR=octocat). More on this below ⤵
Common key-value pairs for subscriptions
Typically, you’ll want to be notified about your test runs. Each CI system has an environment variable that indicates the user that kicked off a build or pipeline.
For example, GitHub Actions has an environment variable called GITHUB_ACTOR . So if you use GitHub Actions, you can subscribe to your test runs using a command such as:
/smart-tests subscribe <YOUR_SMART_TESTS_WORKSPACE> GITHUB_ACTOR=<YOUR_GITHUB_USERNAME>
The tabbed section below describes how to compose /smart-tests subscribe for major CI tools:
|Environment variable |BUILD_REQUESTEDFOREMAIL
|Description |The person who pushed or checked in the changes.
Example:
|
[cols="1,1"] |
|Environment variable |CIRCLE_USERNAME
|Description |The GitHub or Bitbucket username of the user who triggered the pipeline (only if the user has a CircleCI account).
Example:
|
[cols="1,1"] |
|Environment variable |GITHUB_ACTOR
|Description |The name of the person or app that initiated the workflow. For example, octocat.
Example:
|
[NOTE] — GitLab CI has 2 environment variables you can use. — [cols="1,1"] |
|Environment variable |GITLAB_USER_EMAIL
|Description |The email of the user who started the job.
Example:
[cols="1,1"] |
|Environment variable |GITLAB_USER_LOGIN
|Description |The login username of the user who started the job.
Example:
|
[NOTE] — Requires the build user vars Jenkins plugin: https://plugins.jenkins.io/build-user-vars-plugin/ — [cols="1,1"] |
|Environment variable |BUILD_USER_EMAIL
|Description |Email address of the user who started the build.
Example:
|
Other key-value pairs
Since the subscription mechanism is based on CI environment variables, you have a lot of flexibility regarding subscriptions. CI tools expose many environment variables (and you can add your own) that you can pass into /smart-tests subscribe to create custom subscriptions. For more info, check out your CI tool’s documentation.
Receiving notifications
Once you’ve set up a subscription, the CloudBees Smart Tests app will send you a personal message each time a new test session matching the subscription criteria is recorded:
| Passing notification | Failing notification |
|---|---|
|
|
Failing notifications include a link to view test results in CloudBees Smart Tests and a quick summary of failing tests, including annotations for Unhealthy Tests. This helps you get started triaging without checking your email over and over.