Using Pipeline Policies

Organizations often need to enforce standards across their pipelines. As the number of projects and repositories increase within an organization, there is a higher chance of teams deviating from these rules or best practices developed by the shared services team. While administrators would like to enable their developers to use pipelines freely, they still may need to set some restrictions based on industry-specific regulatory compliance or general best practice principles. Pipeline Policies are runtime validations that work for both scripted and declarative pipelines and provide administrators a way to include warnings for or block the execution of pipelines that do not comply with certain regulatory requirements rules or best practice guidelines.

Adding a Pipeline Policy

Administrators can add multiple policies to govern pipeline execution. Each policy can have multiple rules for limiting timeouts either for the entire pipeline, paused actions, or for specific agents. The order in which rules are applied depends on where in the pipeline the policy applies. If a policy’s action is set to Fail, the pipeline stops executing at the point where the rule violation occurs and the proceeding rules are not be applied. If a policy’s action is set to Warning, the pipeline continues to execute and a message is displayed in the logs indicating a rule violation.

Prerequisites

Pipeline Policies plugin
Pipeline plugins

To add a Pipeline Policy

Click Pipeline Policies in the left navigation pane.

You must have the Pipeline Policies plugin installed for this option to appear.
Click New Policy.

Enter a Name.

Policy names must be unique and should be relevant to the policy referenced as they will appear in logs for troubleshooting. Limit policy names to 120 characters. For example: “Avoid executors getting stuck, Cap wait time globally".

Enter a Description.

Include meaningful details in policy descriptions that explain the reason for the policy. For example, if the policy is to ensure adherence to a particular regulation, it is helpful to include it in the description. Limit description to 400 characters.

Enter a Filter.

Filters are case-sensitive.

Write a filter using Apache Ant Pattern to limit the scope of the policy to certain projects. Filters are applied to full job names, which can potentially be composed of folder, organization, and/or branch names, and so on. For instance:

Job Name: Risk-Management/ReportHandler/main

Example filters:

To capture all pipeline jobs, you can leave the filter empty or use **/*
**/default meaning any job titled default (for example, jobs building default branches)
Risk-Management/ReportHandler/main just for this particular job

Select an Action for the Policy.

Actions are how a policy’s rules are enforced. Select from two actions:

Fail: If a rule in the policy is violated, the build fails at that point in the Pipeline.
Warning: If a rule in the policy is violated, a warning appears in the log at that point in the pipeline.

Add Rule(s).
- Agent Timeout: Maximum timeout duration allowed for the execution on an agent. The Agent Timeout rule validates that a timeout period was set for tasks executed on agents.
- Paused Action Timeout: Maximum timeout duration allowed for the execution of paused actions.
  - The Paused Action Timeout rule validates that pipeline steps that require an external condition must be defined within a timeout.
  - Supported paused actions are the following pipeline steps: “input” step, “build” step with property “wait:true”, and “waitForQualityGate” step.
- Paused Action in Agent: The Paused Action in Agent rule validates that a paused action step isn’t executed on an agent, so further activities requiring the agent are not blocked waiting for an external condition.
  - Supported paused actions are the following pipeline steps: “input” step, “build” step with property “wait:true”, and “waitForQualityGate” step.
- Pipeline Timeout: Maximum timeout duration allowed for the execution of pipelines. The Pipeline Timeout rule validates that a timeout period was set for the entire Pipeline execution.
  
  The Timeout property accepts timeout values in the format of number and unit. For example, “1 HOURS”, “90 minutes”, etc.
  
  Units can be provided in both lowercase and uppercase and we support all standard units including: DAYS, HOURS, MINUTES, SECONDS, etc.
Click Save.

Managing Pipeline Policies with Configuration as Code

Pipeline Policies with Configuration as Code is a Preview feature.

A Preview feature:

Has not undergone end-to-end testing with CloudBees products.
Is provided without service-level agreements (SLA), and therefore does not include CloudBees' commitment to functionality or performance.
May impact other stable areas of the product when used.
May have limited documentation.
May not be feature-complete during the Preview period.
May graduate from Preview to a supported feature or be removed from the product.
May introduce breaking changes that prevent upgrades due to incompatibility with future development.

Product features and documentation are frequently updated. If you find an issue or have a suggestion, please contact CloudBees Support.

Starting with version 1.5 of CloudBees Pipeline Policies, Pipelines can be configured using Configuration as Code.

This is an example of a configuration file:

cloudbees-pipeline-policies:
  config:
    policies:
      - action: "warning"
        customMessage: "Please, check your pipeline as it should use less than 20m in our infra"
        description: "this policies helps to reduce the use of agent for long time"
        filter: "acme/dev-team/*"
        name: "Avoid build running for more than 20m"
        rules:
          - "pausedActionInAgentRule"
          - entirePipelineTimeoutRule:
              maxTime: 20
          - agentTimeoutRule:
              maxTime: 20
          - pausedActionTimeoutRule:
              maxTime: 20▼

For details of the properties used in the example above, please see the Pipeline Policy properties section below.

The difference between using the Jenkins CLI and Configuration as Code is that there is no class in Configuration as Code, instead the property contains the class name. The options available to you are:

pausedActionInAgentRule
entirePipelineTimeoutRule
agentTimeoutRule
pausedActionTimeoutRule

Managing Pipeline Policies in bulk

Managing Pipeline Policies across a large number of Jenkins controllers using the graphical user interface (GUI) is time consuming and prone to human error due to the repetitive nature of the task.

Using the provided CLI commands allows the administrator to automate the management of Pipeline Policies across multiple controllers, which reduces efforts and consistency across all development teams. There are two provided CLI commands for working with Pipeline Policies, listing and creating/updating.

Listing all Pipeline Policies

Prerequisites

Pipeline Policies plugin
Pipeline plugins

Command

java -jar jenkins-cli.jar -s https://my-jenkins-instance.example.com/ policies > policies.json

You can include a -auth option or environment variables if you are using HTTP. You must use -user if you are using SSH. See Using the CLI Client for more information.

If you do not include > policies.json - the JSON will print out in the console. Otherwise, if you include it and you have a file with that name, the JSON will replace the contents of the file. If you do not already have a file with that name, the file will be created.

Success

You will receive a response with a list of imported Pipeline Policies in JSON format. Information about the Pipeline Policies in the response includes:

Example output.json with action, custom message, description, filter, name, and rules including any properties associated with the rules like $class and max time, etc.

Failure

You will receive an exception message dependent on what the issue involves and the status of “FAILURE”.

Creating or updating all Pipeline Policies

CloudBees recommends creating a backup copy of your Pipeline Policies before creating or updating. When you upload a JSON file using the put argument on the command, the JSON file of new policies will replace the current imported policies, which could result in accidental deletion of Pipeline Policies.

Command

java -jar jenkins-cli.jar -s https://my-jenkins-instance.example.com/ policies -put < policies.json

You can include an -auth option or environment variables if you are using HTTP. You must use -user if you are using SSH. See Using the CLI Client for more information.

The policies.json file in the command above is a sample file name - replace it with the name of the file you are uploading.

Success

You will receive a message stating the Pipeline Policies have been imported successfully and the status will be “SUCCESS.”

Failure

You will receive an exception message dependent on what the issue involves and a status of “FAILURE”.

Common errors

Action is missing or invalid - Action can be only “warning” or “fail”, and is mandatory.
- Error if missing: Missing [action] field configuration
- Error if invalid: action can only be: 'warning' or 'fail'
Policies name cannot be duplicated
- Error: Policy 'random name' is duplicated. Duplicate policy names are not allowed
Rules cannot be duplicated in the same rule
- Error: Rule Agent Timeout in policy name is duplicated. Duplicate rule names are not allowed within a policy.
Timeouts (if applicable) cannot be zero or negative
- Error: maxTime for Agent Timeout cannot be zero or negative
Using illegal arguments such as invalid Rules or Time Units
- Invalid time units
  - Error: Error invalid timeUnit (INVALID UNIT): "Could not instantiate {name=name, description=some random description, action=warning, rules=[{$class=AgentTimeoutRule, maxTime=3, maxUnit=INVALID UNIT}]} for com.cloudbees.pipeline.policy.Policy: java.lang.IllegalArgumentException: Could not instantiate {maxTime=3, maxUnit=INVALID UNIT} for com.cloudbees.pipeline.rules.rules.AgentTimeoutRule: java.lang.IllegalArgumentException: No enum constant java.util.concurrent.TimeUnit.INVALID UNIT"
- Invalid rules
  - Error: Error invalid Rule (InvalidRule): “Could not instantiate {name=name, description=some random description, action=warning, rules=[{$class=InvalidRule, maxTime=3, maxUnit=DAYS}]} for com.cloudbees.pipeline.policy.Policy: java.lang.UnsupportedOperationException: no known implementation of class com.cloudbees.pipeline.rules.RulesInterceptionRule is named InvalidRule”

Pipeline Policy properties

The following are policy properties used to configure Pipeline Policies.

`action`

The action property is required. There are two options: fail and warn. Actions are how a policy’s rules are enforced. Select from two Actions:

Fail: If a rule in the policy is violated, the build will fail at that point in the Pipeline.
Warning: If a rule in the policy is violated, a warning will appear in the log at that point in the Pipeline.

`customMessage`

The customMessage property is required. It is a description that displays in the Policy report to explain why a policy failed or does not meet standards.

`description`

The description property is required. It is a description that displays in Jenkins to describe the reason for the policy.

`filter`

The filter property is not required. Filters are case-sensitive. Write a filter using Apache Ant Pattern to limit the scope of the policy to certain projects. Filters will be applied to full job names which can potentially be composed of folder, organization, and/or branch names, etc. For instance:

Job Name: Risk-Management/ReportHandler/main

Example filters:

To capture all pipeline jobs, you can leave the filter empty or use **/*
**/default meaning any job titled default (for example, jobs building default branches)
Risk-Management/ReportHandler/main just for this particular job

`name`

The name property is required. It is the display name for the policy that should help to identify the policy.

`rules`

`$class`

The $class property is required if you have rules on your policy. This property is the system name for the rule. There are 4 choices: AgentTimeoutRule, PausedActionTimeoutRule, PausedActionInAgentRule, and EntirePipelineTimeoutRule.

AgentTimeoutRule: Maximum timeout duration allowed for the execution of an agent. The Agent Timeout rule validates that a timeout period was set for tasks executed on agents.
PausedActionTimeoutRule: Maximum timeout duration allowed for the execution of paused actions.
- The Paused Action Timeout rule validates that pipeline steps that require an external condition must be defined within a timeout.
- Supported paused actions are the following pipeline steps: “input” step, “build” step with property “wait:true”, and “waitForQualityGate” step.
PausedActionInAgentRule: The Paused Action in Agent rule validates that a paused action step isn’t executed on an agent, so further activities requiring the agent are not blocked waiting for an external condition.
- Supported paused actions are the following pipeline steps: “input” step, “build” step with property “wait:true”, and “waitForQualityGate” step.
EntirePipelineTimeoutRule: Maximum timeout duration allowed for the execution of pipelines. The Pipeline Timeout rule validates that a timeout period was set for the entire Pipeline execution.

`maxTime`

The Timeout property accepts timeout values in the format of number and unit. For example, “1 HOURS”, “90 minutes”, etc. Units can be provided in both lowercase and uppercase and we support all standard units including: DAYS, HOURS, MINUTES, SECONDS, etc. The default unit is MINUTES.

Example Pipeline Policies JSON file format

The following example shows how to configure a Pipeline Policies JSON file.

[ {
 "action" : "fail",
 "customMessage" : "Paused Actions CANNOT be under nodes, must be executed in controllers only",
 "description" : "some description",
 "filter" : "Jenkinsfile.violation.input-step-on-executor",
 "name" : "Avoid Paused actions ",
 "rules" : [ {
   "$class" : "PausedActionInAgentRule"
 } ]
}, {
 "action" : "fail",
 "customMessage" : "a pipeline has to have a timeout",
 "description" : "a pipeline has to have a general timeout",
 "filter" : "Jenkinsfile.violation.no-timeout",
 "name" : "Entire Pipeline Timeout",
 "rules" : [ {
   "$class" : "EntirePipelineTimeoutRule",
   "maxTime" : 30
 } ]
}, {
 "action" : "fail",
 "customMessage" : "Agent must have timeouts",
 "description" : "Max timeout for agent",
 "filter" : "Jenkinsfile.violation.no-executor-timeout",
 "name" : "Max Global timeout in Agent",
 "rules" : [ {
   "$class" : "AgentTimeoutRule",
   "maxTime" : 30
 } ]
}, {
 "action" : "fail",
 "customMessage" : "paused actions must have timeouts",
 "description" : "paused actions must have timeouts",
 "filter" : "Jenkinsfile.violation.no-timeout-on-paused-action",
 "name" : "Paused Action Timeout",
 "rules" : [ {
   "$class" : "PausedActionTimeoutRule",
   "maxTime" : 10
 } ]
} ]▼

Troubleshooting the Pipeline Timeout rule in Declarative Pipelines

Prerequisites

Pipeline Policies plugin
Pipeline plugins

If a Declarative Pipeline has a top level agent, a Pipeline Timeout Rule will only be applied to anything that runs after the agent is done checking out.

Declarative syntax

pipeline {
 agent any (1)
  options {
   timeout(time: 50, unit: 'MINUTES') (2)
 }
 stages {
     stage('Deploy') {
         steps {
                 sh 'echo Building artifact'
         }
     }
 }
}▼

1	This is a top level agent declaration.
2	This timeout will only be applied to anything that runs after the agent is done checking out.

Scripted syntax

For comparison purposes, the same pipeline written in scripted:

node {
  timeout(time: 50, unit: 'MINUTES'){
    stage("Deploy"){
         sh 'echo Building artifact'
    }
  }
}▼

To have a Pipeline Timeout Rule applied to anything that runs before checkout, top level agents should not be used.

Declarative syntax

pipeline {
  agent none (1)
   options {
    timeout(time: 50, unit: 'MINUTES')
  }
  stages {
      stage('Deploy') {
          agent any (2)
          steps {
              timeout(time: 50, unit: 'MINUTES') {
                  sh 'echo Building artifact'
              }
          }
      }
  }
}▼

1	In this example, instead of adding a top level agent, `agent none` is specified.
2	Here an agent is specified at the stage level. This allows a top-level timeout to apply to the pipeline in this stage, not just what happens after checkout.

Scripted syntax

For comparison purposes, the same pipeline written in scripted:

timeout (50 M){
  stage ("Deploy"){
      node {
          timeout {
           sh {script='echo Building artifact'}
          }
      }
  }
}▼