Managing policies with the policy editor

CloudBees SDM is a preview, with early access for select preview members. Product features and documentation are frequently updated. If you find an issue or have a suggestion, please contact CloudBees Support.

Policies enable engineering managers and leaders to define rules related to the development process of a product or feature. Organization-wide policies provide easy to create global policies to define common best-practices or rules matching their software delivery methodology. A policy applies to the organization associated with your admin account.

Sample policies are included with CloudBees SDM and can be used to create your own policies.

Policy queries

Each policy has a rule and associated actions. Rules can enforce specific steps that drive best practices, compliance, or governance requirements for the process, which help ensure higher quality, more stable software for the customer. If a policy has matches, actions associated with the policy are triggered. For example, a status label defined in the action can appear on the Pull requests screen when conditions for the policy are met.

Only an administrator for CloudBees SDM can create, edit, or modify policies and rules using the Policies screen. Users who are non-administrators can then view the results of policies from generated notifications when a policy’s rules are violated.

Policy rules

To create a policy, you have to define the rules and actions taken if a policy has matches. Rules are defined using the Rules tab of the policy editor.

Rules have three components:

Data type

Determines the type of data to which the policy applies and depends upon the services integrated with CloudBees SDM.

Target filter (optional)

Filters the data to be evaluated. By default, the policy will be evaluated for all instances of the given data type. If a target filter is specified, the policy is only evaluated for instances for which the target filter evaluates to true.

Criteria

Defines what status must be met for the actions associated with the policy to be triggered. The actions are triggered for instances where the expression defined by the criteria evaluates to true.

As an example, if GitHub is integrated, the GithubPullRequest data type can be selected. A target filter of state = 'OPEN' could be specified so that the policy is only evaluated for open pull requests. A criteria of mergeStateStatus = 'DIRTY' would then cause the policy to evaluate to true for open pull requests that have a merge status of DIRTY. Any associated actions would be triggered for these pull requests. The policy would evaluate to false for all other open pull requests.

To create a policy rule, you would enter these values: * Target filter: `state = 'OPEN' * Criteria: `mergeStateStatus = 'DIRTY'

Refer to SDM query language for information about building queries.

Viewing policies

All created policies are listed on the Policies tab. You can view them by selecting Policies to the right of Product hub. If no policies have been created, you will be prompted to create your first policy.

Each policy can have multiple icons.

  • A CloudBees logo (1) to the right of the policy name indicates that this policy is provided by CloudBees.

  • A small tag or label icon (2) indicates that the policy’s action adds one or more status labels.

  • The target filter shield icon (3) indicates the number of records selected by the policy’s target filter.

  • The criteria icon (3) indicates the number of records that match the policy’s criteria and would trigger the policy’s actions.

Sample policy icons

Creating a new policy

To create a new policy:

  1. From Product hub, select Policies.

  2. Select Create policy.

  3. Enter a policy name and description. Select Save changes.

  4. You should see the policy’s Rule tab. If not, select the new policy name and select the Rule tab.

  5. Select a data type from the Data type drop-down box.

  6. Optional: Enter a filter in the Target filter text box. For example, to select all open pull requests, you can use:

    state = 'OPEN'
    Refer to SDM query language for information about building queries.
  7. Enter criteria in the Criteria text box.

  8. Select Save changes.

Policy data types

You can use four policy data types to query data from integrated services like GitHub. These policy data types provide the building blocks to construct policies.

Policy data type Source Type of information available When to use

GithubPullRequest

GitHub

Pull request status, review state, assigned reviewer, time since last activity

GitbubRepository

GitHub

Repository information

JenkinsBuild

Jenkins

Job or build state, pipeline issues

JiraIssue

Jira

Jira ticket number, assigned person

Refer to SDM query language for information about building queries.

Defining policy actions

Once a policy is created, the next step is to establish actions that should happen when the policy is triggered. Each policy can have one or more actions.

Currently, only status labels can be assigned as an action. A status label appears as colored text next to an item that matches a policy’s rules. Status labels use three colors:

  • Red: Needs immediate attention. Work is blocked.

  • Yellow. Needs attention soon. Work might be blocked.

  • Green: Requires action. Work is not blocked.

To define an action for a policy:

  1. Select the policy’s Actions tab.

  2. Select Add action drop-down and choose Status label.

  3. Choose an option from the Label type drop-down: Needs attention, Needs action, or Needs attention soon.

  4. Enter a Label name.

  5. Select Save changes.

Once an action is created, you can toggle it on or off using the slide. The slider is blue when an action is active and light gray when inactive.

Example: Product-specific policies

Let’s say that you want to create a policy to check for open pull requests related to a particular product that have no existing reviews, and no reviews requested. You can create a policy with a rule that targets GitHub pull request data, filters to include only those that are in the open state and in a repository that is associated with the required product, and where the criteria checks that there are no reviews and no review requests.

When creating the rule for this policy, the following values would be entered on the policy’s Rules tab, where PRODUCT NAME is the name of a product you have defined:

Data type

GithubPullRequest

Target filter

state = 'OPEN' and repository.products.any(name = 'PRODUCT NAME')

Criteria

pullRequestReviews.size() = 0 and reviewRequests.size() = 0

For more information about rules, see SDM query language.

For information about specific data types:

Example: Finding pull requests with missing builds

There are occasions when notifications from automated systems can be missed. For example, if there is a network outage between your build server and GitHub, commit notifications may not arrive to your build server. Normally when this happens, notifications for follow-up commits propagate through and builds are triggered.

This example walks you through how to build and refine a query, starting with a simple query and ending with a more complex query using time since build, size of Jenkins builds, and more.

Let’s say you want to identify pull requests that should have been built and, for whatever reason, haven’t. You can use a policy in CloudBees SDM to help identify pull requests without builds.

To start, you can identify pull requests that have not been built In the policy editor:

Data type

GithubPullRequest

Target filter

(blank)

Criteria

state= 'OPEN' and headCommit.statusContexts.size()=0

This rule selects all open pull requests that do not have a build. This rule will return false positives because you are filtering by build structure will take a bit of time to start a build after a PR commit has been updated.

This query can be refined to give your infrastructure some time to run the build. To account for this, you can update the Criteria to (with Target filter blank) to:

Data type

GithubPullRequest

Target filter

(blank)

Criteria

state = 'OPEN' and headCommit.committedDate < now('-1 hour') and headCommit.statusContexts.size()=0

The issue with this rule is that not every repository requires a build. There is no way to distinguish between those PRs that do not have a build because they are not configured to have one and those that do not because the configured build never happened. You need a way to determine PRs that should have been built. A reasonable approximation would be if the default branch of the repository has build results, then probably the PRs should have build results.

The rule can be updated to match only those repositories where the default branch has build results:

Data type

GithubPullRequest

Target filter

(blank)

Criteria

state = 'OPEN' and headCommit.committedDate < now('-1 hour') and headCommit.statusContexts.size()=0 and repository.defaultBranchRef.headCommit.statusContexts.size() > 0

If you set up GitHub actions on a repo, that may also have build results. You may have other processes contributing build results. You can end up with false positives on the cases you are trying to filter for.

To look at build results from a specific source, you can use the statusContexts information like Jenkins rather than the size. For example, Jenkins by default specifies a specific identifier in statusContexts. If you are looking for all of the jobs which should have been built by Jenkins but haven’t, you set the query to look for the identifier that Jenkins is using.

If Jenkins is using the Custom GitHub Notification context plugin, then statusContexts may be different from the default values illustrated below.
Data type

GithubPullRequest

Target filter

(blank)

Criteria

state = 'OPEN' and headCommit.committedDate < now('-1 hour') and headCommit.statusContexts.none(context=jenkins) and repository.defaultBranchRef.headCommit.statusContexts.any(context='jenkins')

This example matches all pull requests that are:

  • open

  • the commit is at least an hour old

  • the commit has not been built by Jenkins, and

  • the repository default branch has been built by Jenkins.

In the case of Jenkins, this refinement can end up giving you false violations. Some Jenkins job type events will only trigger against a specific branch. If you are using old-style Jenkins jobs like freestyle jobs, the default branch will likely have a commit status from Jenkins but PRs will not have a commit status from Jenkins. By changing the rule, you can identify only those repositories that are being monitored by a Jenkins multi-branch project.

Example 1. Jenkins Multibranch Project notification contexts

The multi-branch project will add 2 types of statusContexts notifications:

Branches

Adds continuous-integraiton/jenkins/branch.

Pull requests

Adds either continuous-integration/jenkins/pr-merge or continous-integration/jenkins/pr-head.

These patterns can be customized using the Custom Github Notification context plugin.

If the rule is changed to the example below, then the policy will identify open pull requests in the repository which have a multi-branch project building the default branch where the pull request itself has not been built. In most cases, multi-branch projects are configured to build PRs. This rule would be entered in the Criteria field with the Target filter field left blank.

Data type

GithubPullRequest

Target filter

``

Criteria

state = 'OPEN' and headCommit.committedDate < now('-1 hour') and headCommit.statusContexts.none(context=‘continuous-integration/jenkins/pr-merge’ or context=‘continuous-integration/jenkins/pr-head’) and repository.defaultBranchRef.headCommit.statusContexts.any(context='continuous-integration/jenkins/branch')

Next, let’s say that you want to know whether the number of violations is significant or not. One number on its own is meaningless because you can’t compare it [1]. Ideally, you would like to know how many PRs should be built and what fraction are missing a build. To do this, move some of the criteria into the target filter. CloudBees SDM can report which ones are in compliance with the policy and which ones are in violation.

Data type

GithubPullRequest

Target filter

state = 'OPEN' and headCommit.committedDate < now('-1 hour') and repository.defaultBranchRef.headCommit.statusContexts.any(context='continuous-integration/jenkins/branch')

Criteria

headCommit.statusContexts.none(context=‘continuous-integration/jenkins/pr-merge’ or context=‘continuous-integration/jenkins/pr-head’)

Now, the target filter is all open PRs where the default branch in the repository is being built by the multi-branch job type and the criteria is the subset where the PR has not been built.

Updating existing policies

Once a policy is created, you can change the policy’s name or description, update a policy’s rules, or delete a policy.

Changing a policy’s rules

To change a policy’s rules:

  1. From the Product hub, select Policies.

  2. Select the policy you wish to modify.

  3. On the Rule tab, Make any desired changes to the policy’s data type, target filter, or criteria.

  4. Select Save changes to preserve the updates or Revert to discard them.

For more information about rules, see SDM query language.

Changing a policy’s actions

To change a policy’s actions:

  1. Select the policy’s Actions tab.

  2. Select Add action drop-down and choose Status label.

  3. Choose an option from the Label type drop-down: Needs attention, Needs action, or Needs attention soon.

  4. Enter a Label name.

  5. Select Save changes.

Changing a policy’s name or description

To change a policy’s name or description:

  1. From the Product hub, select Policies.

  2. Select the policy you wish to modify.

  3. Select the Manage tab.

  4. Update the policy name or description and select Save changes.

Deleting a policy

To delete a policy:

  1. From the Product hub, select Policies.

  2. Select the policy you wish to delete.

  3. Select the Manage tab.

  4. Select Delete policy.

  5. Select Delete policy to confirm removing the policy or select Cancel to retain the policy.


1. unless the number of violations is 0!