Test insights

The CloudBees platform Test insights dashboard provides users with a comprehensive view of all testing metrics in their projects. Software testing is invaluable to improve code quality, performance, and security.

Test results from the following sources can publish their data to Test insights:

CloudBees platform workflows.
CloudBees CI or Jenkins pipelines.

Test insights charts display components, workflows, and runs with and without test suites. An overview table of each test suite contains links to more information.

Use Test insights to:

Better understand test execution behaviors, especially which components currently have failing tests and must be resolved as soon as possible.
Detect and fix components with commonly failing or unstable test results.
Detect components that have not yet been tested.
Determine which test cases take the longest to run. Optimize these to reduce build times and speed up deployments.

Collect test results from platform workflows

Add the Publish test results action to a CloudBees platform workflow that executes a test. The action collects test results from the workflow job, then publishes the data in the Test insights dashboard.

Popular testing tool types are supported, including:

JUnit
MSTest
ProdPerfect
TestNG
Jmeter
Selenium
Tosca
Go
Jest

Collect test results from CloudBees CI and Jenkins pipelines

Add the Run a CloudBees CI job action or the Run a Jenkins job action to your workflow to collect test data from CloudBees CI or Jenkins pipelines, respectively. You must set the test-result-location parameter to enable the action to collect test results from the pipeline’s workflow and publish the test results back to the platform for use in generating test insights.

Access and filter test insights charts

Select the time frame of data for analysis for charts in the Test insights dashboard. By default, the last seven days of data are displayed. An option to select a component to filter on is available only if you have not yet selected a specific component.

Select Analytics Test insights.

(Optional) Filter for the chart data you want to display by selecting the following:

Select the FILTER above the charts.
Select one or more Components from the options, if available. If you have already selected a component prior to selecting Test insights, that selection is maintained.

Select a Duration from the following options:

Table 1. **Duration** filter definitions
Duration	Definition
Current week	Current week in the month, Monday to Sunday schedule. For example, if current day is Tuesday, only data from Monday and Tuesday are displayed.
Previous week	Previous week in the month, Monday to Sunday schedule.
Two weeks back	Two weeks prior in the month, Monday to Sunday schedule.
Current month	First day of current month up to current day.
Previous month	First day to last day of previous month.
Two months back	First day to last day of two months prior.
Last 7 days	The past seven days.
Last 30 days	The past 30 days.
Last 90 days	The past 90 days.
Custom range	Select a custom range

Select APPLY.

The data are filtered accordingly and displayed in the test insights charts.

Set a custom date range

To set a custom date range:

Select FILTER.
Select Custom range.
Select dates for the time frame start and end.

The custom date range is set accordingly and displayed in blue on the date picker. You can view the analytics data for any desired time frame, as in the example below:

Figure 1. Example Custom range selected.

For any data that is truncated with an ellipsis (…), hover to display the full content. Use the scroll bars (vertical and horizontal), if necessary, to view the full content.

Components, workflows, and runs information

Get an overview of components, workflows, and workflow runs for the filtered component data in a specified time frame.

The dashboard charts display the following with and without test suites:

Components
Workflows
Workflow runs

Figure 2. Example Components, Workflows, and Workflow runs charts.

Each chart displays the following details (highlighted in the Workflows chart above):

Total number.
A donut chart of those with and without test suites (in percentages).
Numbers of those with and without test suites.

The Components chart also displays the number of associated repositories, and the Workflows chart displays the number of associated branches.

Components chart links

Select a number, or a section of the donut chart, to list the:

Component name
Repository URL
Number of test suites

If you select a section of the donut chart, or the with or without test suites numbers, the data displayed is for only that subset of components.

In the list, perform any of the following:

Select FILTER to filter by with or without test suites.
Search for specific components by entering all or part of any column item into Search.
Select or next to a column heading to sort on that heading.
Select a component name to display its component summary in a new browser tab.
Select a repository URL to display the repository in a new browser tab.

Workflows chart links

Select a number, or a section of the donut chart, to list the:

Workflow name
Component name
Branch name
Number of workflow runs
Number of test suites

If you select a section of the donut chart, or the with or without test suites numbers, the data displayed is for only that subset of workflows.

In the list, perform any of the following:

Select FILTER to filter by with or without test suites.
Search for specific workflows by entering all or part of any column item into Search.
Select or next to a column heading to sort on that heading.
Select a component name to display its component summary in a new browser tab.

Workflow runs chart links

Select a number, or a section of the donut chart, to list the:

Run ID
Workflow name
Component name
Branch name
Run status
Number of test suites

If you select a section of the donut chart, or the with or without test suites numbers, the data displayed is for only that subset of workflow runs.

In the list, perform any of the following:

Select FILTER to filter by with or without test suites.
Search for specific runs by entering all or part of any column item into Search.
Select or next to a column heading to sort on that heading.
Select a run ID to display the run details in a new browser tab.
Select a component name to display the component summary in a new browser tab.

Figure 3. Example Workflow runs list with two filters highlighted.

Tests overview

The filterable tests overview lists details of all test suites run. Display either by Test suites or by Components views.

Table 2. Details of each **Tests overview** view
Detail listed	in Test suites view?	in Components view?
Test suite name
Component name
Default branch name
Workflow name
Last run date and time
Total test cases
Average run time
Total test case runs
Failure rate on last run
Failure rate (overall)

Select

next to a column heading to sort on that heading.

Select from any or all of the following to filter for a specific subset of tests:

Test suite name
Component name
Workflow name
Branch name

To access and filter the tests overview:

Select Analytics Test insights.
Select a view:
- TEST SUITES
- COMPONENTS
(Optional) Select FILTER, and then select one or more of the options.
(Optional) Search for specific test suites by entering all or part of any column item into Search.

The data are filtered accordingly and displayed in Tests overview.

In the Components view, hover over Failure rate to display the totals of:

Successful test case runs (green bar)
Failed test case runs (red bar)
Skipped test case runs (yellow bar)

Figure 4. Example Tests overview list.

Test cases details

In either view, select an item from the Total test cases column to display the following:

Test case name
Average test run duration
Total test runs
Test failure rate
Test run activity link

Figure 5. Example test cases list with run activity link highlighted.

Test run activity

Select Total test cases in either view, and then select View run activity next to a test case to display a bar graph of test case activity.

The following information is listed above the test run activity graph:

Workflow name
Test average run time
Number of total, successful, failed, and skipped runs.

Each bar on the graph represents a workflow run. Select a bar to display the:

Workflow run ID
Run date and time
Run duration

Below the graph is the run ID linking to the run details, and any test output.

Figure 6. Example of selecting a bar on the test run activity graph.

Total runs details

In the Test suites view, select an item from the Total runs column to display the following:

Run time
Run ID: select to display run details in a new browser tab.
Run status
Total test cases
Number of failed test cases

Compare metrics

Use this feature to compare metrics among all organizations (orgs) and sub-organizations (sub-orgs) in the tenant. You can compare metrics within select charts in the analytics dashboards. In the generated list of all orgs/sub-orgs, sort by the org/sub-org name or its status. Select an org to drill down to the component level to display a more fine-grained status list. Hover over a status item to display more information.

You can only compare metrics for charts with

on the upper right.

To compare metrics:

Select Analytics on the left pane, and then select a dashboard.
Select any on the top right of a chart to display the list of orgs/sub-orgs and their status.

The number of items in the compare metrics list equals the total displayed in the chart.
(Optional) Hover over a status to display more information.
(Optional) Select or next to a column heading to sort on that heading.
(Optional) Select an org to list more information about each sub-org and/or component within that org.
(Optional) Select a sub-org to list more details about each sub-org and/or component within that sub-org.

The list of the status of all orgs/sub-orgs/components for the specific metric is displayed.