Jenkins data integration

CloudBees Engineering Efficiency can integrate data from CloudBees CI and Jenkins by enabling the Jenkins app and installing the plugin for CloudBees CI and Jenkins controllers, which sends data from the Jenkins controller to CloudBees Engineering Efficiency. This data is stored in the System of Record.

The CloudBees Software Delivery Management plugin sends the following data to the CloudBees Engineering Efficiency service:

  • Controller identity and information

  • Job/Folder information

  • Build/Run events

The plugin uses a refresh token to authenticate the Jenkins controllers.

Refer to the CloudBees CI and Jenkins integration section for instructions on creating a refresh token and configuring the integration.

How the service works

The customer installs the CloudBees Software Delivery Management plugin on their Jenkins controller, activates the Jenkins app on CloudBees Engineering Efficiency, generates a refresh token, and configures the CloudBees Software Delivery Management plugin on their Jenkins controller with the refresh token.

The plugin does not activate until it is configured with a refresh token. When configured with a refresh token, the plugin batches and sends events every few seconds.

The plugin calls the CloudBees Engineering Efficiency service at least once per minute. The plugin queues events and sends events within one second of the event occurring. It is designed to minimize network bandwidth while maintaining responsiveness of information.

The plugin makes frequent short outbound HTTPS connections to our hosted service to submit relevant controller/job events and job run events.

The plugin synchronizes with the CloudBees Engineering Efficiency service to determine the specific set of data to transfer as well as to resolve any reconciliation requests for missed data. The plugin caches events so that data will not be missed if there is a disconnect between the controller and CloudBees Engineering Efficiency.

No user identifying information is specifically captured by this plugin. However, if user information is used in the naming of build jobs or folders, for example, then those job/folder names are transferred.

Data sent from Jenkins

All HTTP traffic is initiated by the plugin to CloudBees Engineering Efficiency. Inbound connections to Jenkins are never required or used. Some data might be cached on a Jenkins instance prior to transmission, but all Jenkins operations are otherwise read-only and do not interfere with Jenkins operations.

Once per hour, the plugin checks with the CloudBees Engineering Efficiency service and refreshes the list of events it subscribes to. It attaches listeners to the Jenkins eventing system for all of the appropriate events. Any time one of the events occurs, the plugin evaluates the schema against the object on which the event occurs, and prepares an event for upload.

Event schemas

The Event schemas details all data that is sent to CloudBees Engineering Efficiency from the Jenkins plugin. The schema can be viewed on the Jenkins controller that has the CloudBees SDM plugin installed. Navigate to the plugin page and select the Events schema link.

You can see on the Event details screen, for each event, the number of observations, the average number of issues. Every time we capture an event, we record how long it took to capture. The observations, average rate, average CPU time, and total CPU time show how much load the plugin places on the server.

Reading the schema information

Most of the event names should be self explanatory. The following additional conventions have been introduced by this plugin and are custom to the CloudBees Engineering Efficiency plugin.

Event ItemsDeltaRequest.onFinished

Returns a Bloom filter of all the jobs in the Jenkins instance. Used to reduce network bandwidth for synchronizing jobs.

Property stereotype

Identifies the kind of event that is being processed.

Property headSpecs

Identifies which branch.

Property scmUri

Cannonical URI of the source code repository.

Property scmFetchUrl

The actual URL used by the build to checkout the source code repository. This may be different from the scmUri if your build agents are using a mirror or proxy to access the source code. In the case of Git, you may be checking out files using a different protocol, e.g., SSH, than the cannonical protocol, e.g., HTTPS.

While the data collected is dynamic, we have implemented a guard in the plugin that ensures all data we return is subject to the standard data visibility rules of Jenkins.

CloudBees Engineering Efficiency does not have access to secrets, credentials, or similar sensitive information.

Verification of connection to the CloudBees platform for SDM service

The following properties are verified every minute.

DataDescription

Instance ID

The Jenkins instance identifier

Jenkins URL

The URL of the Jenkins instance

Jenkins version

The version of Jenkins

ItemListener.onCopied

This event listens for items, such as jobs and folders, that need to be sent to CloudBees Engineering Efficiency when an event is copied. It has the following properties:

ItemDescription

name

Event name

fullName

Item full name

displayName

Item display name

fullDisplayName

Item full display name

type

Type of item

isJob

Identifies if item is a job

isItemGroup

Identifies if the item is a container of a group of items, e.g., a folder

url

Identifies the absolute URL

displayUrl

Display URL

parentFullName

Parent of the job

scmUri

Cannonical URI of the source code repository.

scmFetchUrl

The actual URL used by the build to checkout the source code repository. This may be different from the scmUri if your build agents are using a mirror or proxy to access the source code.

headSpecs

Identifies which branch

scmBranch

Identifies the top-level branch

organization

SCM organization

repository

SCM repository name

ItemListener.onCreated

This event listens for newly created items, such as jobs and folders, and transfers the following properties.

ItemDescription

name

Item name

fullName

Item full name

displayName

Item display name

fullDisplayName

Item full display name

type

Item type

isJob

Identifies if item is a job

isItemGroup

Identifies if the item is a container of a group of items, e.g., a folder

url

Identifies the absolute URL

displayUrl

Display URL

parentFullName

Parent of the item

scmUri

Cannonical URI of the source code repository

scmFetchUrl

Actual URL used by the build to checkout the source code repository.

headSpecs

Identifies which branch

scmBranch

Identifies the top-level branch

organization

SCM organization

repository

SCM repository name

ItemListener.onDeleted

This event listens for deleted items, such as jobs and folders, and transfers the following properties.

ItemDescription

fullName

Item full name

isJob

Identifies if it is a job

isItemGroup

Identifies if the item is a container of a group of items, e.g., a folder

ItemListener.onUpdated

This event listens for newly updated items and transfers the following properties.

ItemDescription

name

Item name

fullName

Item full name

displayName

Item display name

fullDisplayName

Item full display name

type

Item type

isJob

Identifies if the item is a job

isItemGroup

Identifies if the item is a container of a group of items, e.g., a folder

url

Identifies the absolute URL

displayUrl

Display URL

parentFullName

Parent of the item

scmUri

Cannonical URI for the source repository

scmFetchUrl

Actual URL used by the build to checkout the source code repository.

headSpecs

Identifies which branch

scmBranch

Identifies the top-level branch

organization

SCM organization

repository

SCM repository name

ItemsDeltaRequest.onFinished

This event listens for changes or updates that do not exist in the record and transfers the following properties.

ItemDetails

filter

A bloom filter containing the full names, types, and display names of all items defined in Jenkins

properties

A list of the properties that were used to create the filter

displayName

The display name of the Jenkins controller

url

Jenkins URL

RestartListener.onStart

This event is sent when Jenkins starts and when the plugin is initially configured.

ItemDescription

displayName

The display name of the Jenkins controller

url

Jenkins URL

RunListener.onCompleted

This event is sent when a build completes.

ItemDescription

number

The build number

displayName

The display name of the build

type

Build type

url

The build URL

displayUrl

the display URL of the build

jobFullName

The full name of the job

startedAt

The time this build started

endedAt

The time this build finished

duration

The duration of the build

result

The result of the build

RunListener.onStarted

ItemDescription

number

The build number

displayName

The display name of the build

type

The build type

url

Jenkins URL

displayUrl

Display URL

jobFullName

Full name of the job

startedAt

The time when the Jenkins controller started, in ISO-8601 format

endedAt

null

duration

null

result

PENDING

SCMListener.onCheckout

This event is sent when a SCM is checked out during a build.

ItemDescription

jobFullName

Full name of the job

number

The build number

revisions

Lists revisions, if available

SaveableListener.onJenkinsChange

This event is sent when a change is made to Jenkins' system configuration.

Item

Description

displayName

The display name of the Jenkins controller

url

Root Jenkins URL