Introductory webhook tutorial

Introduction

This tutorial provides instructions for the process to create a functional plugin called SampleWebhook which connects to a public GitLab instance and reacts to the webhooks.

The four step plugin creation process

Create Plugin Workspace.
Define plugin spec and generate plugin.
Implement additional logic.
Build plugin and test.

Prerequisites

These are assumptions for this Tutorial.

An active CloudBees CD/RO instance.
Internet connection.
pdk installed and set up.
An active test account on the public GitLab instance.

Step 1 : Create plugin workspace

After making sure pdk is available in your PATH, you can create a plugin workspace.

Change to your working directory

cd ~/work▼

Call pdk as follows

pdk generate workspace▼

In the interactive prompt type SampleWebhook as plugin name. For the rest of the options (except the plugin language) which are all optional, choose defaults if available or provide your own values. For the plugin language you should use 'groovy'

GeneratedWorkspace

Step 2: Define plugin spec and generate plugin

The pluginspec.yaml has 2 sections one for configuration and the other for procedures.

Update webhook section of YAML

To declare a webhook we need to provide the following values:

secretRequired - does our webhook use any kind of secret (our webhook is going to use token to validate the payload source)
display name
set of parameters for the trigger object
procedure used in order to initially setup the webhook: its parameters and other procedure-level details, such as shell or hasConfig. Replace the configuration section with following:

webhook:
  secretRequired: true
  displayName: GitLab
  parameters:
    - name: repository
      label: Repository name
      required: true
    - name: eventType
      label: Event type
      required: true
      value: Push Hook
  setupProcedure:
    parameters:
      - name: projectId
        required: true
      - name: accessToken_credential
        type: credential
        credentialType: secret
        required: true▼

Generate plugin

Execute the following command from the root level directory of the plugin workspace.

pdk generate plugin▼

After execution, it should look as follows.

With this step, plugin generation is complete.

Review the auto-generated code

This section is provided to give a perspective on how the boiler plate generated code looks like and what it means.

The dsl/properties folder has the following structure:

groovy/lib
groovy/scripts

The only folder that could be modified is the groovy/lib folder.

Step 3: Implement additional logic

Add necessary library imports

In order to connect to GitLab instance we will use a java library https://github.com/gitlab4j/gitlab4j-api. Add the following lines to the build.gradle file:

implementation group: 'org.gitlab4j', name: 'gitlab4j-api', version: '4.15.7'▼

And run gradle copyDependencies to download the third-party libraries into the plugin’s assets.

Your agent folder should get content like this:

Modify setupProcedure step code

Modify the logic of the auto-generated procedure as follows:

/**
* setupProcedure - SetupProcedure/SetupProcedure
* Add your code into this method and it will be called when the step runs
* @param projectId (required: true)
* @param ec_trigger (required: true)
* @param ec_action (required: false)
* @param webhookSecret_credential (required: false)

*/
def setupProcedure(StepParameters p, StepResult sr) {
    log.info p.asMap.get('projectId')
    log.info p.asMap.get('ec_action')
    log.info p.asMap.get('webhookSecret_credential')
    log.info p.asMap.get('accessToken_credential')

    def triggerId = p.asMap.get('ec_trigger')
    def filters = [new Filter('pluginKey', 'equals', '@PLUGIN_KEY@'),
                   new Filter('triggerType', 'equals', 'webhook') ,
                   new Filter('triggerId', 'equals', triggerId)]

    def triggersResponse = FlowAPI.ec.findObjects(
        objectType: 'trigger',
        filters: filters,
        viewName: 'Details'
    )
    def trigger = triggersResponse.object?.first()?.trigger
    log.info "Trigger: $trigger"
    if(!trigger) {
        throw new RuntimeException("Failed to find trigger $triggerId")
    }
    def url = trigger.webhookUrl
    log.info "Url: $url"
    def secret = p.getRequiredCredential('webhookSecret_credential')?.secretValue
    def publicId = trigger.accessTokenPublicId

    def token = p.getRequiredCredential('accessToken_credential')?.secretValue
    GitLabApi gitLabApi = new GitLabApi("https://gitlab.com", token)
    String projectId = p.asMap.projectId
    def project =  gitLabApi.getProjectApi().getProject(projectId)
    log.info "Found project: $project"
    def hooks = gitLabApi.getProjectApi().getHooks(projectId)
    log.info "Found hooks: $hooks"
    def existing = hooks.find {
        it.url.endsWith(publicId)
    }
    if (existing) {
        gitLabApi.getProjectApi().deleteHook(existing)
        log.info "Deleted existing hook $existing.id"
    }
    def hook = new ProjectHook(pushEvents: true)
    def added = gitLabApi.getProjectApi().addHook(projectId, url as String, hook, false, secret as String)
    log.info "Added hook $added"

    log.info("step SetupProcedure has been finished")
}▼

We will use Filter object to find a trigger, so we have to add it to the imports:

import com.electriccloud.client.groovy.models.Filter▼

We will need GitLabApi client, so we will add this as well (unless it is added automatically by your IDE):

import org.gitlab4j.api.GitLabApi
import org.gitlab4j.api.models.ProjectHook▼

We will use this procedure to create a webhook automatically on the GitLab side when the webhook is getting configured on the CloudBees CD/RO side.

Webhook script

To process the payload coming from the GitLab instance, we will have to define the script.groovy webhook script. Initially, it can look like this:

import groovy.json.JsonSlurper

def retval = []

def trigger = args.trigger
Map<String, String> headers = args.headers
String method = args.method
String body = args.body
String url = args.url
def query = args.query

retval << "Headers: $headers"
retval << "Method: $method"
retval << "Body: $body"
retval << "URL: $url"
retval << "Query: $query"
retval << "Trigger: $trigger"

def pluginParameters = [:]
trigger.pluginParameters.properties.each { k, v ->
    pluginParameters[k] = v['value']
}

def repositoryParameter = pluginParameters['repository']
retval << "Parameter $repositoryParameter"

def eventTypeParameter = pluginParameters['eventType']
retval << "Parameter $eventTypeParameter"

def event = 'push'

String message = retval.join("\n")

def response = [
    eventType      : event,
    launchWebhook  : true,
    branch         : 'master',
    responseMessage: message.toString(),
    webhookData: 'some webhook data',
]

return response▼

For GitLab payload, we are going to process headers (to compare event type and the token) and the payload, to fetch repository and commit-specific data. Change the script as follows:

import groovy.json.JsonSlurper

def retval = []

//https://docs.gitlab.com/ee/user/project/integrations/webhooks.html

def trigger = args.trigger
def body = args.body
Map<String, String> headers = args.headers
def event = headers.get('x-gitlab-event')
def token = headers.get('x-gitlab-token')

if (trigger.webhookSecret != token) {
    throw new RuntimeException("Failed to validate token")
}

def decoded = new JsonSlurper().parseText(body)
def branch = decoded.ref?.replaceAll('refs/heads/', '')

def pluginParameters = [:]
trigger.pluginParameters.properties.each { k, v ->
    pluginParameters[k] = v['value']
}

retval << "Parameters: $pluginParameters"
def repositoryParameter = pluginParameters['repository']
retval << "Parameter $repositoryParameter"

def eventTypeParameter = pluginParameters['eventType']
retval << "Parameter $eventTypeParameter"

if (eventTypeParameter != event) {
    return [
        responseMessage: "Ignoring unsupported '${event}' event".toString(),
        eventType      : event,
        launchWebhook  : false
    ]
}
def repoName = decoded.repository?.name
if (repositoryParameter != repoName) {
    return [
        responseMessage: "Ignoring unsupported repository ${repoName}".toString(),
        repository     : repoName,
        launchWebhook  : false
    ]
}

def commitId = decoded.checkout_sha

def response = [
    eventType      : event,
    launchWebhook  : true, // set to false for ping events
    branch         : branch,
    responseMessage: "Launched for commit $commitId".toString(), // will be shown in the webhook response
    webhookData    : [commitId: commitId, branch: branch.toString(), user_name: decoded.user_name?.toString()] as Map<String, String>,
]

return response▼

Most of third-party systems have requirements for the maximum allowed response time of the webhook call. If you need some intricate logic for your webhook, consider using a go-between procedure by adding a trigger to a procedure that in return will perform needed actions and launch pipelines/releases/applications.

Step 4: Build, install, and test

Build the plugin from root directory of the SampleWebhook plugin workspace.

pdk build▼

Congratulations! You have now created a functional plugin.

Create a pipeline and configure the plugin

After your plugin is installed, navigate to the Pipelines page and click "New" in the top right corner of the web page.

Click Create New, enter Sample Webhook Pipeline, choose your project, click Ok.

Now create a trigger. Click on the Triggers button of the pipeline, and create a new trigger:

Now click on the Add + button:

Fill your trigger data: the repository name, the secret (token) and the event type the webhook will react to.

Choose a service account for your trigger:

The next page will provide you with the URL for your webhook, that you are supposed to provide into GitLab project settings, and, if checked, the setup procedure parameters to create the webhook automatically:

Run the setup procedure, and after it finishes, your will see the newly created webhook in your project settings:

The webhook URL should be accessible from the outside. If you are using VPN, consider using ngrok or similar services.

Running the webhook

Try sending a test payload using GitLab sandbox. If everything is correct, your should see your pipeline launched with webhook data attached to the properties: