Introduction
IntroductionUse the CLI with a public repositoryUpdate tokenless authentication to use GitHub OIDCUse the CLI with Docker image
IntroductionChoose a value for build nameRecord builds from multiple repositoriesRecord builds for microservicesRecord builds for multistage buildRun under restricted networksDeal with shallow clones
Record sessions
IntroductionRequest and run a subset of testsSubset with the CloudBees Smart Tests CLIChoose a subset optimization targetObserve subset behaviorSmart subset optimization targetsZero Input SubsettingReplace static parallel suites with a dynamic parallel subsetUse groups to split subsetsCombine with rule-based test selectionCloudBees Smart Tests extension for VS code
IntroductionConvert test reports to JUnit formatEnsure record tests always runsManage complex test session layoutsUse 'flavors' to run the best tests for an environmentSeparate out test suitesAttach log files

AI Assistance

2 minute read

If you have questions about CloudBees Smart Tests, ask your AI agent to answer them by using Agent skills combined with locally searchable product documentation.

Installation

Claude Code users can run the following slash command:

/plugin marketplace add cloudbees-oss/agent-skills
/plugin install smart-tests@cloudbees
▼

For other agents, use the skills CLI from your terminal:

npx skills add cloudbees-oss/agent-skills
▼

Usage

Once you install the skill, ask questions about CloudBees Smart Tests and get answers based on the documentation. For example:

What are the use-cases for Predictive Test Selection?

Use cases with the view command

The smart-tests view command outputs JSON directly to stdout, making it easy to feed into an AI coding agent. This enables automated workflows for investigating and fixing test issues.

Identify and fix flaky tests

Flaky tests that intermittently fail waste CI time and erode developer trust. With smart-tests view flaky-tests, your AI coding agent receives the list of current flaky tests, then locates the test code and suggests or applies fixes.

smart-tests view flaky-tests --weeks 4 > flaky-tests.json
▼

Pass the output to your AI coding agent with a prompt like:

Here is a JSON list of flaky tests from the last 4 weeks. For each test, find the source file, analyze why it might be flaky (timing issues, shared state, external dependencies), and suggest a fix.

You can also scope the query to a specific test suite and use --limit to focus on the worst offenders:

smart-tests view flaky-tests --test-suite integration-tests --limit 10 > flaky-tests.json
▼

Here are the top flaky integration tests. Identify common patterns (e.g., missing test isolation, hardcoded timeouts) and propose concrete fixes.

Investigate a specific test’s failure history

When a test is failing on a branch, smart-tests view test-results lets you pull the recent execution history for that test so an AI agent can look for patterns—such as failures only on certain branches or time windows.

smart-tests view test-results \
  --test-path "class=PaymentServiceTest#testcase=testProcessPayment" \
  --status FAILED \
  --logs > test-results.json
▼

Here are the recent failures for this test including execution logs. What is the root cause and how can we fix it?

Triage flaky tests by trend

A rising weeklyScoreDelta signals a test that is becoming more flaky over time. Use the multi-week output to ask an agent to prioritize which tests need immediate attention.

smart-tests view flaky-tests --weeks 8 > flaky-trends.json
▼

Here is flakiness score data over the past 8 weeks. Which tests are getting worse (rising weeklyScoreDelta)? Prioritize them and explain what each test does based on its name.

CLI reference