Git

11 minute readExtensibilityDeveloper productivity

Git is a distributed revision control system with an emphasis on speed. Every Git working directory is a full-fledged repository with complete history and full revision tracking capabilities, not dependent on network access or a central server. Git is free software distributed under the terms of the GNU General Public License version 2 The Git plugin allows you to connect the Git source control server to ElectricFlow. The EC-Git plugin enables you to perform:

  • Code checkouts.

  • Preflight build.

  • Continuous integration.

Two ways of integration are available:

  • JGit Java library: The Java library is used to operate the Git tree. Git cli is not needed on the agent machine.

  • Git Cli: CLI is used to operate the Git tree. Git CLI must be installed on the agent machine in order for the plugin to work.

Plugin version 1.9.0.2021111737

Revised on November 17, 2021

The difference between Git and other services around Git

Git

Provides the main functionality for managing git-repositories and their data, which can be deployed anywhere.

GitHub

Provides additional GitHub REST API functionality.

Bitbucket

Provides additional Bitbucket REST v2 API functionality.

Prerequisites

Plugin has two separate ways to perform git repository operations

JGit Java Library

JGit Java library: Java library is used to operate the Git tree.

Some operations like a shallow clone (with depth specified) and the newer OpenSSH private keys are not supported by the library.

Tested on version: 5.7.0.202003110725-r

Git CLI client

  • The 'git' CLI utility is used to operate the Git tree. Git CLI must be installed on the agent machine in order for the plugin to work with this option.

Tested on version: 2.25.1

Plugin configurations

Plugin configurations are sets of parameters that apply across some or all of the plugin procedures. They reduce repetition of common values, create predefined parameter sets for end users, and securely store credentials where needed. Each configuration is given a unique name that is entered in designated parameters on procedures that use them.

Creating plugin configurations

To create plugin configurations in CloudBees CD, do these steps:

  • Go to Administration Plugins to open the Plugin Manager.

  • Find the EC-Git-1.9.0.2021111737 row.

  • Click Configure to open the Configurations page.

  • Click Create Configuration as per the description of parameters below.

Configuration procedure parameters

ParameterDescription

Configuration name

The name for the created configuration.

Description

Description of the configuration.

Auth type

Authentication type: a username with a password, access token, or a private key.

NOTE: if your account uses 2-factor authentication, please consult the documentation of your third-party Git provider regarding the use of passwords.

If you are using GitHub as your Git provider, consider using personal access tokens instead of a password.

Username and password

Username and password.

NOTE: Please consult the third-party Git provider for the security details of password usage in checkout calls.

For example, for GitHub please use Personal Access Token instead of the password.

Token

The personal access token (PAT) that has proper access scope.

Key

The private key that allows access to Git repositories. Please note that the JGit library does not support keys in OpenSSH format. In this case, you should use the git CLI or regenerate the key to a PEM format.

This parameter can only be used with Git or SSH protocol, and is not applicable to the HTTP(S) protocol.

Passphrase

Passphrase which is used to encrypt the protected key.

Library to Use

Use either git CLI or JGit lib to work with Git tree. Some commands are not available in the JGit library but independent of resources. Git CLI requires git cli to be installed on the resource.

Repository URL for check configuration

Plugin will check if this repository is accessible before saving the configuration values.

Ignore SSL errors

Turn SSL verification off for instances with self-signed certificates.

Check configuration resource

A resource that is used to check the configuration. If you are using git CLI it has to be installed on the resource.

Check Connection?

If checked, the connection endpoint and credentials entered as part of the configuration will be tested. If this option is checked, configuration will not be saved if the test fails.

Debug Level

This option sets debug level for logs. If info is selected, only summary information will be shown, for debug, there will be some debug information and for trace the whole requests and responses will be shown.

Plugin procedures

Clone

This procedure performs clones a remote repository into a path on the local filesystem.

Clone parameters

ParameterDescription

Configuration Name

Previously defined configuration for the plugin

Destination directory

Indicate the path where the source tree will be created (can be relative to the job’s workspace).

Git repository

A repository URL, e.g. git://server/repo.git .

Clone mirror?

Set up a mirror of the source repository. It allows for the mirroring of repositories to and from external sources. You can use it to mirror branches, tags, and commits between repositories.

Supported only by GIT CLI.

Path of the reference repo to use during clone

Specify a folder containing a repository that will be used by git as a reference during clone operations. This option will be ignored if the folder is not available on the agent.

Supported only by GIT CLI.

Remote branch

The name of the Git branch to use. ie: experimental . The default will be master .

Tag

Provide the name of a specific tag to checkout after the clone command.

Commit hash

The commit hash to update the index.

NOTE: This will make a checkout with a detached HEAD.

Clone submodules?

After the clone is created, initialize and clone submodules within based on the provided pathspec. If no pathspec is provided, all submodules are initialized and cloned.

Submodules pathspecs

The list of the submodules' pathspecs delimited by newlines.

Supported only by GIT CLI.

Shallow submodules

All submodules which are cloned will be shallow with a depth of 1.

Supported only by GIT CLI.

Overwrite?

This option will delete the already existing previously cloned folder with the repository and will run the clone anew.

Depth

Create a shallow clone with a history truncated to the specified number of revisions.

Supported only by GIT CLI.

Result property sheet

The results will be saved into this property/property sheet.

Output parameters

ParameterDescription

cloneId

SHA Id of the top cloned commit.

cloneBranches

Branches of the top cloned commit in JSON format.

cloneData

JSON representation of the cloned data: commit, commit message etc.

Clone Form

Commit

This procedure performs Git commit and optionally pushes changes to the remote repository.

Commit parameters

ParameterDescription

Configuration Name

Previously defined configuration for the plugin

Folder

The folder where to take files from. Must be under a git tree (this means that either in this folder or in any of its parents there must be a .git folder).

Files

Files pattern (GLOB) to use to add files to the commit. Each pattern per line, e.g. README\*.

Commit message

Commit message for the newly created commit. If not provided, date and CloudBees CD Job ID will be used to identify the commit.

Author name

Commit author name, e.g. John Wick.

Author email

Commit author’s email, e.g. sample@gmail.com.

Committer name

Name of the committer. This can be the name of the project or a schedule, e.g. $[/myProject/name].

Committer email

Email of the committer, e.g. build-user@company.com.

Remove missing?

If checked, the files that are missing from the repository will be removed from SCM.

Push?

If checked, the commit will be pushed to the remote repository.

Remote

If the push is checked, provide the remote name here. Defaults to the origin.

Fail on empty commit?

If checked, the procedure will fail if there is nothing to commit.

Result property sheet

The results will be saved into this property/property sheet.

Output parameters

ParameterDescription

commitId

SHA Id of the created commit.

commitData

JSON representation of the pulled data: commit, commit message etc.

Commit Form

Pull

This procedure fetches changes from the remote repository into the local one. If the local repository does not exist, it will be cloned at the first run.

Pull parameters

ParameterDescription

Configuration Name

Previously defined configuration for the plugin

Git repository

A repository URL, e.g. git://server/repo.git .

Destination directory

Indicate the path where the source tree will be created (can be relative to the job’s workspace).

Remote branch

The name of the Git branch to use. ie: experimental . The default will be master .

Fetch submodules?

This option controls if new commits of populated submodules should be fetched, and if the working trees of active submodules should be updated, too. It can be used as a boolean option to completely disable recursion when set to No or to unconditionally recurse into all populated submodules when set to Yes , which is the default when this option is used without any value. Use On demand to only recurse into a populated submodule when the superproject retrieves a commit that updates the submodule’s reference to a commit that isn’t already in the local submodule clone.

For JGit it is no difference between Yes and On demand .

Clean The Workspace

Clean the workspace before or after every checkout by deleting all untracked files and directories, including those which are specified in .gitignore . Resets all tracked files to their versioned state. Ensures that the workspace is in the same state as if clone and checkout were performed in a new workspace. Reduces the risk that the current build will be affected by files generated by prior builds. Do not remove files outside the workspace (like temporary files or cache files). Do not remove files in the .git repository of the workspace.

Files to clean

What the kind of the files will be cleaned: untracked only or changed and untracked

Metadata property path

Property sheet where the last commit will be stored for Git log report. Optional field. If omitted, generate a report for the last commit. If present generate a report from the specified commit to the last one. e.g: /myProject/RepoName_branch/lastCommit

Result property sheet

The results will be saved into this property/property sheet.

Output parameters

ParameterDescription

pullId

SHA Id of the last pulled commit.

pullBranches

Branches of the top cloned commit in JSON format.

pullData

JSON representation of the committed data: commit, commit message etc.

Pull Form

Collect Reporting Data parameters

ParameterDescription

Configuration name

Previously defined configuration for the plugin

Git repository URL

A repository URL, e.g. 'git://server/repo.git' or 'https://github.com/github/testrepo.git'.

Remote branch

The name of the Git branch to use. ie: 'experimental'. The default will be master.

Starting commit

The hash of the earliest commit from which data needs to be collected. Needs to be set up only for the first run.

File prefix

If provided, the matching string will be removed from the file path before sending the report. For example if file path is /opt/repo/file1, file prefix = /opt/repo will resolve it to /file1. Similarly, file prefix = /opt/repo/ will resolve it to file1.

Metadata property path

Property sheet where run metadata will be stored. If omitted, /mySchedule/EC-Git-%JobName%-%ReportObjectType% will be used for schedule contest. For all other contexts root is /myProject.

Commit URL template

Enter the Template for the URL to be resolved. For example the template for a URL https://github.com/somerepo/commit/a12b3 would be ${repoUrl}/commit/${commitId}.

Transform script

Allows a user to provide Groovy closure for payload customization. This script will be invoked by the plugin with 1 or 2 parameters. * if one parameter: it is a payload object. * if two parameters: 1st parameter is context object (FlowPlugin), 2nd is a payload object. In this example commitMessage field will be added to payload object:

[source,txt] ---- { Object pluginObject, Map commit -> if (!commit) { return commit }

commit["commitMessage"] = commit["commitMessage"] + System.lineSeparator() + "CUSTOM TEXT"

return commit }

or

{ Map commit -> if (!commit) { return commit }

commit["commitMessage"] = commit["commitMessage"] + System.lineSeparator() + "CUSTOM TEXT"

return commit } ----

Preview

If checked, no data will be sent to the reporting system. Use this option to preview gathered data.

Debug

If checked, the log level will be set to "Debug" for the job.

Release name

Name of the CloudBees Flow release collected data relates to.

Release project name

Name of the CloudBees Flow release project collected data relates to.

Triggers Support

Polling Trigger is designed to work instead of ECSCM-SentryMonitor and it’s not related to the last one at all.

The plugin allows configuring the CloudBees CD to poll the specified source control repository for updates with a preconfigured schedule. If an update is detected, the pipeline or release object will run.

The Polling schedule is managed by the PollingTriggers located in the Electric Cloud project.

After the Trigger with a type 'Polling' is created, the schedule will check the specified ref commitId and run the trigger.

All successful jobs, except the last one, are removed.

For configuring use the documentation Event-based triggers - Polling triggers.

Schedule PollingTriggers should be enabled.
  1. Go to DevOps Essentials Platform Home page.

  2. Select Electric Cloud from the Projects tab.

  3. Select the Schedules tab and enable the PollingTriggers.

Also see section: Known Issues.

Result

As a result of running a trigger will have written down a pipeline property parsedWebhookData.

From the documentation:

Name: parsedWebhookData; Type: String; Description: JSON with parsed webhook data to be set on a pipeline/release/procedure run.

The property contains such fields:

  • repository — a repository URL

  • monitor — what was monitored: branch or tags

  • commitId — a commit identifier

  • branch — a branch name if monitor == branch

  • tags — tags names if monitor == tags

  • fullMessage — full commit message

  • commitAuthorName — commit author name

  • commitAuthorEmail — commit author email

Known Issues

GitHub

If you are using GitHub as a remote repository hosting, please do not use password and use Personal Access Token instead.

JGit library

JGit does not support shallow clone, i.e. depth parameter of the clone procedure. If you intend to use it, consider switching to cli library. JGit does not support private keys in a new OpenSSH format, please regenerate key with a '-m PEM'. Issue for the JSch library used by jGit: #129 "JSchException: invalid privatekey" on OpenSSH 7.8 and above JGit does not support the mirror of a repo during cloning. JGit does not support the use of a reference repo during cloning. JGit nameRev can return tags instead of branches.

Git

While using submodules make sure to apply one type of the authentication for access to the main repo and submodules. Don’t mix keys and passwords/tokens simultaneously.

For passwords or tokens use URL like: https://github.com…​;
For keys use URL like: git@github.com…​

Git versions before 2.13 don’t support the option 'exclude' in command "name-rev".

A change in schedules/triggers

There is a change in schedules/triggers after upgrading the CD server from 10.0 to 10.1.

The ECSCM-SentryMonitor scheduler does not check anymore the schedules/triggers related to Git repositories while in version 10.0, it checks and triggers jobs/pipelines.

The new scheduler named PollingTriggers calls a procedure in this plugin. It’s not enabled by default.

How to configure it described in Triggers Support.

There is no automatic migration path for this.

Old triggers should be removed and new triggers should be created manually.

For instance. There is a project: myProject; pipeline: myPipeline; trigger: myTrigger.

ectool deleteTrigger myProject myTrigger --pipelineName myPipeline

If uses Git server with a self-signed certificate occurs SSL-error like these:

  • javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

  • sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

  • sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

To avoid the problem add certificates to the JRE Keystore:

(Linux) cd /opt/cloudbees/sda/jre/lib/security
(Windows) cd C:\ProgramData\CloudBees\Software Delivery Automation\jre\lib\security

keytool -importcert -file /tmp/CA_crt.pem -keystore cacerts -alias myRepository

Release notes

EC-Git 1.9.0

  • Added generation of git log report.

EC-Git 1.8.0

  • Added an "Ignore SSL errors" flag.

  • Added resetting uncommitted changes also.

EC-Git 1.7.0

  • Extra functionality added to support DSL Round-Trip feature

  • [Fixed case:avoid use option 'exclude' for Git before v2.13]

EC-Git 1.6.1

  • Fixed issue with the setup script.

EC-Git 1.6.0

  • Added support for the plugins configurations as objects.

  • Added support SSH keys with passphrases

EC-Git 1.5.0

  • Added support submodules for Clone procedure

  • Added support submodules for Pull procedure

  • Fixed an issue with overwriting sources during a multi-microservice application deploy.

EC-Git 1.4.0

  • Added integration with CloudBees Analytics server.

EC-Git 1.3.0

  • Polling Triggers. Add possibility to monitor tags for the specified repository.

  • Polling Triggers. Added support for "parsedWebhookData".

  • Added mirroring of a source repository during a clone.

  • Added "Path of the reference repo to use during clone" option to use a cached copy of the repository.

  • Added "Clean The Workspace" option to use during a pull.

  • Added branch name to the clone/pull output data.

EC-Git 1.2.1

  • Upgraded third-party dependencies to address security issues.

EC-Git 1.2.0

  • Added the Source Provider procedure for new Microservices model

EC-Git 1.1.1

  • Fixed an issue when on some old setups EditConfiguration did not work properly.

EC-Git 1.1.0

  • Added Polling Triggers support.

  • Enhanced message and documentation on the unsupported private key types.

EC-Git 1.0.0

  • First release.