Creating projects based on GitHub repository structure

The GitHub Branch Source plugin allows you to create a new project based on the repository structure from one or more GitHub users or organizations. You can either:

  • Import all or a subset of repositories as jobs into the workspace from a GitHub user or organization

  • Import a single repository’s branches as jobs from a GitHub user or organization

The GitHub Organization project scans the repository, importing the Pipeline jobs it identifies according to your criteria. To define a job, you create a Pipeline script in a Jenkinsfile in the root directory of the project or branch. It is not necessary to use the exact same Pipeline script in each branch; instead you can have different Pipeline scripts defined in project branches, or add and remove `Jenkinsfile`s depending on your use cases. After a project is imported, Jenkins immediately runs a job. Removing a Pipeline script from a branch or from the entire project, removes the associated jobs from the view. You can restrict imports using a regular expression on the GitHub Organization configuration page. This plugin fits into the larger "Pipeline as Code" story; for more information, see Understanding and implementing Pipeline as Code.

The GitHub Branch Source plugin creates a folder for each repository. It then creates a separate job for each branch which contains a Jenkinsfile in the top level directory. The Jenkinsfile contains a Pipeline script outlining the steps of the job. Scripts can be different for each branch.

Importing GitHub pull requests

The GitHub Branch Source plugin allows the project to import externally originated pull requests containing a Pipeline script. The following features are included:

  • GitHub user or organization repository’s pull requests are treated as jobs

  • Pull request build result status reported back to GitHub

Pull requests will be added to Jenkins as long as the pull request originates from a remote repository, contains a Pipeline script in a Jenkinsfile, and is mergeable.

When the proper webhooks are configured, Jenkins will report the status of the pull request’s job to GitHub. The status is reported as follows:

  • Success - the job was successful

  • Failure - the job failed and the pull request is not ready to be merged

  • Error - something unexpected happened; example: the job was aborted in Jenkins

  • Pending - the job is waiting in the build queue

Creating a GitHub Organization

This plugin creates a new project type, a GitHub Organization. To use this plugin, you must have GitHub credentials with the appropriate access level to your project (see the section on credentials below) installed in Jenkins. Set up your GitHub Organization as follows:

From the Jenkins dashboard select New Item > GitHub Organization. Enter an item name and select OK.

jenkins github org

On the Projects tab, enter the details for GitHub Organization.

github org projects tab

Select or add new Credentials.

These credentials should allow you to at least see the contents of your targeted repository.

In the Owner field, specify a username or an organization name.

Save, and wait for the Scan Organization to run. Job progress is displayed to the left hand side.

scan org log

You may need to refresh the page after it runs to see your repositories.

A this point, you are finished with basic project configuration. You can now explore your imported repositories, configuring different settings on each of those folders if needed. You can also investigate the results of the jobs run as part of the Scan Organization. Each of your imported repositories is a folder containing a project for each branch with a Pipeline script defined in a Jenkinsfile in the root directory. You cannot modify the settings on these folders.

Selecting GitHub Organization credentials

There are two different types of credentials for the GitHub organization:

  • scan credentials: used to access the GitHub API

  • checkout credentials: used to clone the repository from GitHub

You can use different credentials for each as long as they have the appropriate permissions. There is an anonymous option for checkout. Regardless, you should create a GitHub access token to use to avoid storing your password in Jenkins and prevent any issues when using the GitHub API. When using a GitHub access token, you must use standard Username with password credentials, where the username is the same as your GitHub username and the password is your access token.

Using build triggers and webhooks in a GitHub Organization

You can change the build configuration to perform the Scan Organization at different times. The default setting will scan your GitHub repository for changes if it has not received any change notifications from GitHub within the past day. This scan will catch the repositories you’ve added or removed from the managed GitHub repository. You can always force the Scan Organization to run from the GitHub Organization page.

scan now

While the build triggers are often enough, you can set up webhooks to automatically trigger builds when changes are pushed to your GitHub repositories.

To do this you must have a GitHub login with a token.
To set up a webhook:
  1. Navigate to the main configuration settings page by selecting Manage Jenkins > Configure System.

  2. In the GitHub section, add a GitHub Server with your credentials.

  3. If you need a token, generate one by selecting the Advanced button next to the notebook and pencil, then selecting the Manage additional GitHub actions button next to Additional actions and select Convert login and password to token.

You can also configure this manually through GitHub by registering the URL provided in the help section of the server config.

Controlling what is built in a GitHub Organization

By default, Jenkins will build any branches it finds in the origin repository (the actual repository found in your organization). Also, any pull requests that have been filed from forks (clones in user accounts) will also be built; Jenkins will try to merge the pull request with the base (target) branch on every commit, to check for possible integration conflicts. Pull requests filed from the origin repository are skipped as these are already getting built as plain branches.

To customize this behavior, as of version 1.8 of the plugin, you can add Behaviors to the GitHub Organization configuration.

In particular, you can choose to build origin-repository pull requests in addition to, or instead of, building the plain branches; and you can choose to build pull requests without merging with the base branch, or in addition to building the merge product.