Known issues
Release notes

Advanced reporting plugin

13 minute readReferenceExtensibilityDeveloper productivity

Introduction

This tutorial provides instructions to create a reporting plugin called Tutorial-JIRA-Reporting that has a single procedure called CollectReportingData, which sends user story data from a Jira instance to CloudBees Analytics to enable metrics in the RCC.

Note: The source code for this plugin can be found in https://github.com/electric-cloud-community/flowpdf/tree/master/perl/Tutorial-JIRA-Reporting

Prerequisites

These are assumptions for this tutorial.

  • Basic tutorial has been completed.

  • pdk is installed and setup.

  • An active CloudBees CD/RO instance.

  • An active CloudBees Analytics center that is configured and connected to CloudBees CD/RO instance.

  • Internet connection.

  • An accessible Jira instance.

  • Write access to a Git repository.

Step 1 : Generate plugin from scratch

Create a plugin workspace with Plugin name as Tutorial-JIRA-Reporting, Plugin type as reporting and Plugin description as SampleJira plugin for reporting tutorial

pdk generate workspace
▼

Once done the response from the command prompt and the workspace layout should look as follows:

image

image

Step 2 : Configuring plugin specifications and generating a plugin

Edit config/pluginspec.yaml as follows:

pluginInfo:
  # This is default sample specification
  # Feel free to change it
  # Call flowpdk showdoc pluginspec to see the list of available fields and their description
  pluginName: 'Tutorial-JIRA-Reporting'
  version: '1.0.0'
  description: 'SampleJira plugin for reporting tutorial'
  author: 'CloudBees'
  supportUrl: '<>'
  category: 'Utilities'
# The reporting configuration, will generate a procedure and a bunch of configuration scripts required for DOIS to work
devOpsInsight:
  supportedReports:
  -
    reportObjectType: 'build'
    parameters:
    -
      name: param1
      documentation: null
      required: true
      type: entry
      label: Sample Reporting Parameter
    -
      name: param2
      documentation: null
      required: false
      type: entry
      label: Sample Reporting Parameter 2
  # The name of the source as it will appear in the dashboards
  datasourceName: 'My Source Name'
▼

Make changes to pluginspec.yaml to add a jiraProjectName parameter to the procedure, a configuration section with rest to enable check connection feature and generate the plugin.

pluginInfo:
  # This is default sample specification
  # Feel free to change it
  # Call flowpdk showdoc pluginspec to see the list of available fields and their description
  pluginName: 'Tutorial-JIRA-Reporting'
  version: '1.0.0'
  description: 'SampleJira plugin for reporting tutorial'
  author: 'CloudBees'
  authorUrl: '<>'
  category: 'Utilities'


# The reporting configuration, will generate a procedure and a bunch of configuration scripts required for DOIS to work
devOpsInsight:
  supportedReports:
  -
    reportObjectType: 'feature'
    parameters:
    -
      name: jiraProjectName
      documentation: AJira project that will be used
      required: true
      type: entry
      label: Jira project name
  # The name of the source as it will appear in the dashboards
  datasourceName: 'JIRA Datasource from tutorial'

# Plugin configuration description
configuration:
  checkConnection: true
  # This is a declaration for the plugin configuration
  shell: 'cb-perl'
  # A set of fields will be added to process debug level in the configuration
  hasDebugLevel: true
  # parameters:
  restConfigInfo:
    endpointDescription: sample
    endpointLabel: Jira Server Endpoint
    checkConnectionUri: /rest/api/2/configuration
    headers: {Accept: "application/json"}
    # Auth schemes for the plugin
    authSchemes:
      basic:
        userNameLabel: Jira User Name
        passwordLabel: Jira User Password
        description: A username and password to connect to theJira instance.
        checkConnectionUri: /rest/api/2/configuration
        credentialLabel: Jira Credentials
▼

The output will look something like this:

image

Step 3 : Writing reporting logic using FlowPDF Perl reporting ecosystem

Since plugin was generated with type Reporting, flowpdk generated a module for reporting already wired into plugin procedures as follows:

image

Reporting.pm contains the following boiler plate code already generated:

package FlowPlugin::TutorialJIRAReporting::Reporting;
use Data::Dumper;
use base qw/FlowPDF::Component::EF::Reporting/;
use FlowPDF::Log;
use strict;
use warnings;

# todo more sample boilerplate
sub compareMetadata {
    my ($self, $metadata1, $metadata2) = @_;

    die 'Not implemented';

    my $value1 = $metadata1->getValue();
    my $value2 = $metadata2->getValue();
    # Implement here logic of metadata values comparison.
    # Return 1 if there are newer records than record to which metadata is pointing.
    return 1;
}

sub initialGetRecords {
    my ($self, $pluginObject, $limit) = @_;

    die 'Not implemented';
    # build records and return them
    # todo required fields
    my $records = $pluginObject->yourMethodTobuildTheRecords($limit);
    return $records;
}

sub getRecordsAfter {
    my ($self, $pluginObject, $metadata) = @_;

    die 'Not implemented';
    # build records using metadata as start point using your functions
    my $records = $pluginObject->yourMethodTobuildTheRecordsAfter($metadata);
    return $records;
}

sub getLastRecord {
    my ($self, $pluginObject) = @_;

    die 'Not implemented';
    my $lastRecord = $pluginObject->yourMethodToGetLastRecord();
    return $lastRecord;
}

sub buildDataset {
    my ($self, $pluginObject, $records) = @_;

    die 'Not implemented';

    my $dataset = $self->newDataset(['yourReportObjectType']);
    for my $row (@$records) {
        # now, data is a pointer, you need to populate it by yourself using it's methods.
        my $data = $dataset->newData({
            reportObjectType => 'yourReportObjectType',
        });
        for my $k (keys %$row) {
            $data->{values}->{$k} = $row->{$k};
        }
    }
    return $dataset;
}



1;
▼

image

Notice that at line #37, the reporting component gets loaded with 3 special parameters, out of which reportObjectTypes is already set to feature. We need to fill these two:

  • metadataUniqueKey

  • payloadKeys

In order to pass values to the above, we will first get the report object type definition for a feature report object type by calling ectool as follows:

ectool getReportObjectAttributes feature
▼

This call will return something like:

<response requestId="1" nodeId="10.211.55.19">
    <reportObjectAttribute>
        <reportObjectAttributeId>d9930108-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>baseDrilldownUrl</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Base Drilldown Url</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d965132d-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>createdOn</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Created On</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>DATETIME</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d97b3343-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>featureName</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Feature Name</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d970fa10-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>key</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Unique Feature ID</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d9697ffe-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>modifiedOn</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Modified On</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>DATETIME</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d95cd5cb-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>pluginConfiguration</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Plugin Configuration</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d9547159-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>pluginName</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Plugin Name</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d9747c81-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>releaseName</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Release Name</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d977d7e2-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>releaseProjectName</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Release Project Name</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d9983129-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>releaseUri</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Release URI</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d9898b26-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>resolution</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Resolution</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
        <enumerationValue>Cannot Reproduce</enumerationValue>
        <enumerationValue>Duplicate</enumerationValue>
        <enumerationValue>Fixed</enumerationValue>
        <enumerationValue>Incomplete</enumerationValue>
        <enumerationValue>Won't Fix</enumerationValue>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d96d779f-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>resolvedOn</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Resolved On</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>DATETIME</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d9502b98-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>source</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Source</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d960a65c-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>sourceUrl</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Source Url</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d984d035-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>status</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Status</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
        <enumerationValue>Closed</enumerationValue>
        <enumerationValue>In Progress</enumerationValue>
        <enumerationValue>Open</enumerationValue>
        <enumerationValue>Reopened</enumerationValue>
        <enumerationValue>Resolved</enumerationValue>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d97edcc4-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>storyPoints</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Story Points</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>NUMBER</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d99c76ea-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>tags</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Tags</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d958b71a-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>timestamp</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Timestamp</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>DATETIME</type>
    </reportObjectAttribute>
    <reportObjectAttribute>
        <reportObjectAttributeId>d98e6d27-e061-11e9-b47a-001c420cece5</reportObjectAttributeId>
        <reportObjectAttributeName>type</reportObjectAttributeName>
        <createTime>2019-09-26T13:30:38.710Z</createTime>
        <displayName>Feature Type</displayName>
        <lastModifiedBy>admin</lastModifiedBy>
        <modifyTime>2019-09-26T13:30:38.710Z</modifyTime>
        <owner>admin</owner>
        <reportObjectTypeName>feature</reportObjectTypeName>
        <required>0</required>
        <type>STRING</type>
        <enumerationValue>Improvement</enumerationValue>
        <enumerationValue>New Feature</enumerationValue>
        <enumerationValue>Story</enumerationValue>
    </reportObjectAttribute>
</response>
▼

This huge XML can give us a list of attributes that we will use. For now, we’re interested only in reportObjectAttributeName and type tags. Let’s create a list and note it somewhere. We will go back to it in a moment.

Metadata facilitates Change Data Capture where in the plugin is aware if there is any new change that would entail sending reporting data to the CloudBees Analytics Data Server. The fields that participate in Metadata essentially provide the mechanism to capture if that data has changed. Note that this mechanism is required as the plugin is using the PULL model, where in it is periodically looking for what has changed. The fields that participate in Metadata are "key" and "modifiedOn". These fields are part of the payload, which we will describe later. Since the key has to be unique we choose it to be the Jira project name from configuration.

Do the following and make sure to remove the die statement.

my $featureReporting = FlowPDF::ComponentManager->loadComponent(
    'FlowPlugin::TutorialJIRAReporting::Reporting', {
        reportObjectTypes     => ['feature'],
        metadataUniqueKey     => $params->{jiraProjectName},
        payloadKeys           => ['modifiedOn', 'key']
    }, $self);
$featureReporting->CollectReportingData();
▼

As we will be using JQL to get information fromJira, ensure that the following JQL returns a list of issues.

project = "TEST".
▼

For reporting we need to get entities in order from oldest to newest. SinceJira has limit of max results in a query, we need to use DESC sorting as opposed to ASC sorting (which could miss latest records if max results threshold is exceeded).

TheJira REST API is available at http://yourjira/rest/api/2/

We will use action search with jql and maxResuilts parameters. We will use Basic Auth scheme forJira and leverage flowpdf to generate the boiler plate code for it. We will implement 2 functions - One for getting request and another for getting issues.

But before that we need to import additional Perl modules and pragmas. Make sure that you have imports that are:

use strict;
use warnings;
use base qw/FlowPDF/;
use JSON;
use Data::Dumper;
use FlowPDF::Log;
### Importing exceptions that will be used for reporting.
use FlowPDF::Exception::WrongFunctionArgumentValue;
use FlowPDF::Exception::MissingFunctionArgument;
use FlowPDF::Exception::RuntimeException;
▼

Function for getting request:

### This functions is a regular function that performs get request against provided url.
sub get {
    my ($self, $url) = @_;

    if (!$url) {
        FlowPDF::Exception::MissingFunctionArgument->new('Missing url parameter for function get')->throw();
    }
    if ($url !~ m|\/|s) {
        FlowPDF::Exception::WrongFunctionArgumentValue->new('$url parameter should be started from /, got: ' . $url)->throw();
    }
    my $context = $self->getContext();
    my $rest = $context->newRESTClient();
    my $runtimeParameters = $context->getRuntimeParameters();
    my $endpoint = $runtimeParameters->{endpoint};

    $endpoint =~ s|\/$||gs;

    my $finalUrl = $endpoint . $url;
    my $request = $rest->newRequest(GET => $finalUrl);
    my $response = $rest->doRequest($request);

    ### If code is more than 399, it is an error.
    if ($response->code() && $response->code() > 399) {
        FlowPDF::Exception::RuntimeException->new("Received not ok response with code: " . $response->code())->throw();
    }
    return decode_json($response->decoded_content());
}
▼

This function accepts a URL for rest request and performs a get request.

To implement reporting we need the following:

  • Reporting procedure when run the first time, should get all issues.

  • Get last issue so we can check it against the latest issue and decide whether we need to get any more issues fromJira.

  • Get issues updated after current issue.

  • Compare metadata and if it is not the latest, return 1.

We implement getIssues with project name as 1st parameter and hash reference as the second with the following possible values:

  • asRecords (will return processed jira records instead of raw response)

  • after (if this provided, will return only issues after date)

  • getLastIssue (will return only one latest issue from jira)

  • limit (will return records count)

Using all information from above we can write the following function:

### This functions receives an issues fromJira using JQL.
### Since our reporting mechanics is primitive enough, we will be using only issue type stories.

sub getIssues {
    my ($self, $projectName, $opts) = @_;

    my $runtimeParameters = $self->getContext()->getRuntimeParameters();
    my $tempJQL = qq|project=$projectName AND issuetype=Story|;

    my $lastIssueJQL = $tempJQL . ' ORDER BY updatedDate DESC&maxResults=1';

    if ($opts->{after}) {
        if ($opts->{after} =~ m/^\d+$/s) {
            my $dt = DataTime->from_epoch($opts->{after});
            $opts->{after} = sprintf('%s %s', $dt->ymd(), $dt->hms());
        }
        $tempJQL .= qq| AND updatedDate > "$opts->{after}" ORDER BY updatedDate DESC|;
    }

    if ($opts->{limit}) {
        $tempJQL .= '&maxResults=' . $opts->{limit};
    }
    logInfo("Running JQL: $tempJQL");
    if ($opts->{getLastIssue}) {
        $tempJQL = $lastIssueJQL;
    }
    my $issues = $self->get('/rest/api/2/search?jql=' . $tempJQL);
    unless ($opts->{asRecords}) {
        return $issues;
    }
    my $records = [];
    for my $issue (@{$issues->{issues}}) {
        my $tempRecord = {
            type                => $issue->{fields}->{issuetype}->{NAME},
            source              => 'JIRA',
            createdOn           => $issue->{fields}->{created},
            defectName          => $issue->{fields}->{summary},
            key                 => $issue->{key},
            modifiedOn          => $issue->{fields}->{updated},
            resolution          => $issue->{fields}->{resolution}->{NAME},
            resolvedOn          => $issue->{fields}->{resolutionDate},
            sourceUrl           => $runtimeParameters->{endpoint},
            status              => $issue->{fields}->{status}->{statusCategory}->{NAME},
        };
        unshift @$records, $tempRecord;
    }

    return $records;
}
▼

Now we will be editing Reporting.pm file.

Make sure that it has the following imports:

package FlowPlugin::TutorialJIRAReporting::Reporting;
use Data::Dumper;
use base qw/FlowPDF::Component::EF::Reporting/;
use FlowPDF::Log;
use strict;
use warnings;
use DateTime;
▼

Before we proceed, note that CloudBees Analytics Server has date format in Zulu. For example:

2019-01-01T14:38:01.000Z
▼

ButJira has slightly different format. To convert it to Zulu format write 2 simple functions, one of them will be using DateTime module for date manipulation:

### Service function, that takes string date in jira format and returns DateTime object.
sub getDateTimeObject {
    my ($self, $date) = @_;

    if ($date =~ m/^(\d+)-(\d+)-(\d+)T(\d+):(\d+):(\d+).*$/s) {
        my $dt = DateTime->new(
            year   => $1,
            month  => $2,
            day    => $3,
            hour   => $4,
            minute => $5,
            second => $6
        );
        return $dt;
    }
    return undef;
}

### This function converts jira's internal date representation in zulu format, that being used across reporting.
sub jiraDateToZulu {
    my ($self, $date) = @_;

    my $dt = $self->getDateTimeObject($date);

    my $zulu = sprintf('%sT%s.000Z', $dt->ymd(), $dt->hms());
    return $zulu
}
▼

Now, as we decided to use key and modifiedOn as values of metadata, we need to implement compareMetadata function. It is implemented as follows:

sub compareMetadata {
    my ($self, $metadata1, $metadata2) = @_;

    my $value1 = $metadata1->getValue();
    my $value2 = $metadata2->getValue();

    # Implement here logic of metadata values comparison.
    # Return 1 if there are newer records than record to which metadata is pointing.
    my $dt1 = $self->getDateTimeObject($value1->{modifiedOn});
    my $dt2 = $self->getDateTimeObject($value2->{modifiedOn});
    return $dt1->epoch() <=> $dt2->epoch();
}
▼

It is very convenient to compare dates in epoch format. Epoch format is an integer, so it is just a comparison of 2 integers.

Now, using our getIssues() function, implement retrieval of records:

sub initialGetRecords {
    my ($self, $pluginObject, $limit) = @_;

    my $jiraProjectName = $pluginObject->getContext()->getRuntimeParameters()->{jiraProjectName};
    # build records and return them
    # todo required fields
    my $records = $pluginObject->getIssues($jiraProjectName, {asRecords => 1, after => '1970-01-01 00:00'});
    return $records;
}

sub getRecordsAfter {
    my ($self, $pluginObject, $metadata) = @_;

    my $jiraProjectName = $pluginObject->getContext()->getRuntimeParameters()->{jiraProjectName};
    my $value = $metadata->getValue()->{modifiedOn};
    my $issueKey = $metadata->getValue()->{key};
    my $dt = $self->getDateTimeObject($value);
    my $lastDate = sprintf('%s %s:%s', $dt->ymd(), $dt->hour(), $dt->minute());
    # build records using metadata as start point using your functions
    my $records = $pluginObject->getIssues($jiraProjectName, {
        asRecords => 1,
        after => $lastDate
    });

    # since the smallest unit of time that jira allows to capture modifications is a minute
    # we exclude the last reported issue from list.

    @$records = grep {
        my $zulu = $self->jiraDateToZulu($_->{modifiedOn});
        if ($_->{key} eq $issueKey && $zulu eq $_->{modifiedOn}) {
            0;
        } else {
            1;
        }
        ;
    } @$records;
    return $records;
}

sub getLastRecord {
    my ($self, $pluginObject) = @_;

    my $jiraProjectName = $pluginObject->getContext()->getRuntimeParameters()->{jiraProjectName};
    my $lastRecord = $pluginObject->getIssues($jiraProjectName, {
        asRecords    => 1,
        getLastIssue => 1
    });
    return $lastRecord->[0];
}
▼

Note that getLastRecords returns only one record on hash reference format, but not in array reference, as other getters.

Implement buildDataset as follows to ensure data sent to CloudBees Analytics Server is in the right format:

sub buildDataset {
    my ($self, $pluginObject, $records) = @_;

    my $dataset = $self->newDataset(['feature']);
    for my $row (@$records) {
        # now, data is a pointer, you need to populate it by yourself using it's methods.
        my $data = $dataset->newData({
            reportObjectType => 'feature',
        });
        for my $k (keys %$row) {
            next unless defined $row->{$k};
            if ($k eq 'modifiedOn' || $k eq 'createdOn' || $k eq 'resolvedOn') {
                $row->{$k} = $self->jiraDateToZulu($row->{$k});
            }
            if ($k eq 'status') {
                if ($row->{$k} eq 'To Do') {
                    $row->{$k} = 'Open';
                } elsif ($row->{$k} eq 'Done') {
                    $row->{$k} = 'Resolved';
                }
            }
            if ($k eq 'resolution') {
                if ($row->{$k} eq 'Done') {
                    $row->{$k} = 'Fixed';
                }
            }
            $data->{values}->{$k} = $row->{$k};
        }
    }
    return $dataset;
}
▼

Make sure that all missing imports are added.

Reporting.pm:

package FlowPlugin::TutorialJIRAReporting::Reporting;
use Data::Dumper;
use base qw/FlowPDF::Component::EF::Reporting/;
use FlowPDF::Log;
use strict;
use warnings;
use DateTime;

### Service function, that takes string date in jira format and returns DateTime object.
sub getDateTimeObject {
    my ($self, $date) = @_;

    if ($date =~ m/^(\d+)-(\d+)-(\d+)T(\d+):(\d+):(\d+).*$/s) {
        my $dt = DateTime->new(
            year   => $1,
            month  => $2,
            day    => $3,
            hour   => $4,
            minute => $5,
            second => $6
        );
        return $dt;
    }
    return undef;
}


### This function converts jira's internal date representation in zulu format, that being used across reporting.
sub jiraDateToZulu {
    my ($self, $date) = @_;

    my $dt = $self->getDateTimeObject($date);

    my $zulu = sprintf('%sT%s.000Z', $dt->ymd(), $dt->hms());
    return $zulu
}


### This function compares metadata. It works exactly as the sort function in perl.
### If 1st argument of this function is "bigger" than 2nd, it should return 1. If equal 0, else -1.
sub compareMetadata {
    my ($self, $metadata1, $metadata2) = @_;

    my $value1 = $metadata1->getValue();
    my $value2 = $metadata2->getValue();

    # Implement here logic of metadata values comparison.
    # Return 1 if there are newer records than record to which metadata is pointing.
    my $dt1 = $self->getDateTimeObject($value1->{modifiedOn});
    my $dt2 = $self->getDateTimeObject($value2->{modifiedOn});
    return $dt1->epoch() <=> $dt2->epoch();
}


sub initialGetRecords {
    my ($self, $pluginObject, $limit) = @_;

    my $jiraProjectName = $pluginObject->getContext()->getRuntimeParameters()->{jiraProjectName};
    # build records and return them
    # todo required fields
    my $records = $pluginObject->getIssues($jiraProjectName, {asRecords => 1, after => '1970-01-01 00:00'});
    return $records;
}

sub getRecordsAfter {
    my ($self, $pluginObject, $metadata) = @_;

    my $jiraProjectName = $pluginObject->getContext()->getRuntimeParameters()->{jiraProjectName};
    my $value = $metadata->getValue()->{modifiedOn};
    my $issueKey = $metadata->getValue()->{key};
    my $dt = $self->getDateTimeObject($value);
    my $lastDate = sprintf('%s %s:%s', $dt->ymd(), $dt->hour(), $dt->minute());
    # build records using metadata as start point using your functions
    my $records = $pluginObject->getIssues($jiraProjectName, {
        asRecords => 1,
        after => $lastDate
    });

    # this is required because jira does not allow us looking after date with milliseconds. We have only minutes.
    # so, we have to exclude last reported issue from list.

    @$records = grep {
        my $zulu = $self->jiraDateToZulu($_->{modifiedOn});
        if ($_->{key} eq $issueKey && $zulu eq $_->{modifiedOn}) {
            0;
        } else {
            1;
        }
        ;
    } @$records;
    return $records;
}

sub getLastRecord {
    my ($self, $pluginObject) = @_;

    my $jiraProjectName = $pluginObject->getContext()->getRuntimeParameters()->{jiraProjectName};
    my $lastRecord = $pluginObject->getIssues($jiraProjectName, {
        asRecords    => 1,
        getLastIssue => 1
    });
    return $lastRecord->[0];
}

sub buildDataset {
    my ($self, $pluginObject, $records) = @_;

    my $dataset = $self->newDataset(['feature']);
    for my $row (@$records) {
        # now, data is a pointer, you need to populate it by yourself using it's methods.
        my $data = $dataset->newData({
            reportObjectType => 'feature',
        });
        for my $k (keys %$row) {
            next unless defined $row->{$k};
            if ($k eq 'modifiedOn' || $k eq 'createdOn' || $k eq 'resolvedOn') {
                $row->{$k} = $self->jiraDateToZulu($row->{$k});
            }
            if ($k eq 'status') {
                if ($row->{$k} eq 'To Do') {
                    $row->{$k} = 'Open';
                } elsif ($row->{$k} eq 'Done') {
                    $row->{$k} = 'Resolved';
                }
            }
            $data->{values}->{$k} = $row->{$k};
        }
    }
    return $dataset;
}


1;
▼

TutorialJIRAReporting.pm:

package FlowPlugin::TutorialJIRAReporting;
use strict;
use warnings;
use base qw/FlowPDF/;
use JSON;
use Data::Dumper;
use FlowPDF::Log;

### Importing exceptions that will be used for reporting.
use FlowPDF::Exception::WrongFunctionArgumentValue;
use FlowPDF::Exception::MissingFunctionArgument;
use FlowPDF::Exception::RuntimeException;
# Feel free to use new libraries here, e.g. use File::Temp;

# Service function that is being used to set some metadata for a plugin.
sub pluginInfo {
    return {
        pluginName          => '@PLUGIN_KEY@',
        pluginVersion       => '@PLUGIN_VERSION@',
        configFields        => ['config'],
        configLocations     => ['ec_plugin_cfgs'],
        defaultConfigValues => {
            authScheme => 'basic'
        }
    };
}

# Procedure parameters:
# config
# jiraProjectName
# previewMode
# transformScript
# debug
# releaseName
# releaseProjectName

sub collectReportingData {
    my $self = shift;
    my $params = shift;
    my $stepResult = shift;

    my $featureReporting = FlowPDF::ComponentManager->loadComponent(
        'FlowPlugin::TutorialJIRAReporting::Reporting', {
            reportObjectTypes     => ['feature'],
            metadataUniqueKey     => $params->{jiraProjectName},
            payloadKeys           => ['modifiedOn', 'key']
        }, $self);
    $featureReporting->CollectReportingData();
}
## === step ends ===
# Please do not remove the marker above, it is used to place new procedures into this file.

### This functions is a regular function that performs get request against provided url.
sub get {
    my ($self, $url) = @_;

    if (!$url) {
        FlowPDF::Exception::MissingFunctionArgument->new('Missing url parameter for function get')->throw();
    }
    if ($url !~ m|\/|s) {
        FlowPDF::Exception::WrongFunctionArgumentValue->new('$url parameter should be started from /, got: ' . $url)->throw();
    }
    my $context = $self->getContext();
    my $rest = $context->newRESTClient();
    my $runtimeParameters = $context->getRuntimeParameters();
    my $endpoint = $runtimeParameters->{endpoint};

    $endpoint =~ s|\/$||gs;

    my $finalUrl = $endpoint . $url;
    my $request = $rest->newRequest(GET => $finalUrl);
    my $response = $rest->doRequest($request);

    ### If code is more than 399, it is an error.
    if ($response->code() && $response->code() > 399) {
        FlowPDF::Exception::RuntimeException->new("Received not ok response with code: " . $response->code())->throw();
    }
    return decode_json($response->decoded_content());
}


### This functions receives an issues fromJira using JQL.
### Since our reporting mechanics is primitive enough, we will be using only issue types stories.

sub getIssues {
    my ($self, $projectName, $opts) = @_;

    my $runtimeParameters = $self->getContext()->getRuntimeParameters();
    my $tempJQL = qq|project=$projectName AND issuetype=Story|;

    my $lastIssueJQL = $tempJQL . ' ORDER BY updatedDate DESC&maxResults=1';

    if ($opts->{after}) {
        if ($opts->{after} =~ m/^\d+$/s) {
            my $dt = DataTime->from_epoch($opts->{after});
            $opts->{after} = sprintf('%s %s', $dt->ymd(), $dt->hms());
        }
        $tempJQL .= qq| AND updatedDate > "$opts->{after}" ORDER BY updatedDate DESC|;
    }

    if ($opts->{limit}) {
        $tempJQL .= '&maxResults=' . $opts->{limit};
    }
    logInfo("Running JQL: $tempJQL");
    if ($opts->{getLastIssue}) {
        $tempJQL = $lastIssueJQL;
    }
    my $issues = $self->get('/rest/api/2/search?jql=' . $tempJQL);
    unless ($opts->{asRecords}) {
        return $issues;
    }
    my $records = [];
    for my $issue (@{$issues->{issues}}) {
        my $tempRecord = {
            type                => $issue->{fields}->{issuetype}->{NAME},
            pluginConfiguration => $runtimeParameters->{config},
            pluginName          => $self->getPluginName(),
            releaseName         => $runtimeParameters->{releaseName},
            releaseProjectName  => $runtimeParameters->{releaseProjectName},
            source              => 'JIRA',
            createdOn           => $issue->{fields}->{created},
            defectName          => $issue->{fields}->{summary},
            key                 => $issue->{key},
            modifiedOn          => $issue->{fields}->{updated},
            resolution          => $issue->{fields}->{resolution},
            resolvedOn          => $issue->{fields}->{resolutionDate},
            sourceUrl           => $runtimeParameters->{endpoint},
            status              => $issue->{fields}->{status}->{statusCategory}->{NAME},
            # sourceUrl           => $runtimeParameters->{endpoint} . '/browse/' . $issue->{key},
            # tags                => '',
            # timestamp           => '',
        };
        unshift @$records, $tempRecord;
    }

    return $records;
}

1;
▼

Step 4: DOIS setup and reporting check

Install and promote the plugin. Then navigate to releases and create "Jira Test Project" release as follows:

image

image

Navigate to CloudBees Analytics Center:

image

Navigate to Release Command Center:

image

Click on Dashboard editor:

image

Click on Setup:

image

Click "Add" button for User stories section:

image

Select our tutorial plugin and click for new configuration creation:

image

Enter valid config values:

image

After a small period you can see your procedure running in the Jobs Tab as follows:

image

Navigate to Release Command Center again to see your Release data being populated. Note that your reporting numbers may be different.

image

Advanced REST