Getting started
IntroductionInstalled packagesEnvironment variablesDebugging buildsCachingParallel testingDeployment pipelinesRunning commands as rootIgnoring commandsRun a command in case of an errorSetting timezoneAccess to other repositoriesUsing scriptsGit LFSUbuntu 20.04 Focal
IntroductionFTP/SSH/SFTP/RSYNCCustom scriptHerokuHeroku MigrationAWS Elastic BeanstalkAWS CodeDeployAWS LambdaAWS S3Google App EngineAWS CLIgcloud CLIAzure CLIDigital OceanEngine YardCapistranoGit PushCloudFoundryCloud66anynines
IntroductionBrowser testingBackground commandsGit submodulesCustom DNSArtifacts
IntroductionCassandraCouchDBDynamoDBMongoDBMySQLPostgreSQLSQLite
IntroductionDartElixirGoJava and JVMMeteorNode.jsPHPPythonRubyRust
IntroductionBeanstalkdRabbitMQRedis
IntroductionElasticsearchMemcachedSphinx
IntroductionCloudBees CodeShip Pro introduction guideCloudBees CodeShip Pro introduction guide 2CloudBees CodeShip Pro introduction guide 3CloudBees CodeShip Pro introduction guide 4CloudBees CodeShip Pro introduction guide 5
Migrating to CloudBees CodeShip Pro
Docker for CI/CD
Getting started with CloudBees CodeShip Pro overview and resources
IntroductionServices configSteps configEnvironment variablesImage registriesImage cachingBuild argumentsVolumesHandling secretsMultiple reposServices and databasesSetting an SSH private keyPreinstalled images
IntroductionInstallationUsing the Jet CLIjet cleanupjet decryptjet encryptjet generatejet loadjet runjet stepsjet updatejet validatejet version
IntroductionAWSAWS S3AWS EBAWS ECSAWS EKSAWS CodeDeployAzure Container ServiceHerokuGoogle CloudGoogle App EngineGoogle Cloud FunctionsGoogle Container EngineGoogle Compute EngineKubernetesTerraformDeploy Via SSHDocker SwarmIBM Cloud FoundryIBM Cloud Container Service
IntroductionBrowser testingParallel testing
IntroductionElixirGoJava and JVMNode.jsPHPPythonRuby
IntroductionCaching multi-stage DockerfileContainer is slow to exitDiffering results with Jet CLIDocker Hub rate limitingManaging multiple environments for a serviceReducing log output from dependent containersStep log not appearingUsing env vars in command steps

Using Multiple Repositories

3 minute read

This task requires the following: - Local machine install of our CLI tool

Multi-Repo Build Pipelines

When using microservices, you may have several repos you want to build together or in sequences. There are two primary setups to accomplish this, and for each setup there are a couple of methods you can use. * You can create separate CI/CD projects for each repository, allowing them to deploy independently of one another. Then, you can either have each project bring in all of your individual component repos for full-stack testing, or you can create a separate, agnostic project just for the purposes of combining them for full-stack testing. * You can create a single project that pulls from all of your repos, deploying all components together whenever the master project is updated.

Once you decide which workflow you like, there are two main ways to combine your repos for full-stack testing.

Using Image Pushes

If you opt to create separate CI/CD projects for each individual component, you can have each component’s pipeline end with an image push to an image registry.

Then, either with a separate project or as a part of the build process for each component, you can pull in the latest images for each component to build and test your combined application.

Cloning Another Repository Into Your Docker Image

In addition to the above method using images in a registry, you can also clone the repos themselves during the build process - either with a separate project or as part of the build for each individual component.

The OAuth Option

Before delving into the SSH approach, another option to cloning from GitHub is to use [OAuth instead of an SSH key]:

git clone https://<token>:x-oauth-basic@github.com/owner/repo.git

Github and Bitbucket provide additional information on this.

The SSH Key Option

The process for securely setting your private SSH key is already covered in our setting SSH private key documentation. But the outlined approach has one drawback – it does not allow for SSH based operations during the image build. That’s a concern if you are trying to bake a cloned repo right into the image (and not wanting to pull the repo into a volume for every CloudBees CodeShip Pro build).

The suggested deviation from the outlined approach is to:

  1. Designate the encrypted file in codeship-services.yml as the encrypted_args_file for a build.

app:
  build:
    image: project-image
    encrypted_args_file: codeship.env.encrypted
    dockerfile: Dockerfile
▼

  1. Ensure that a git clone can be performed with the generated public key (either setting as a deploy key with one repo, or setting up a machine user with access to multiple repos)

  2. Configure your Dockerfile with the following as a guide:

FROM ubuntu:16.04
ARG PRIVATE_SSH_KEY

RUN apt-get update \
  && apt-get install -y --no-install-recommends\
    ca-certificates  \
    git-core \
    ssh

# Prevent the PRIVATE_SSH_KEY from persisting in an image layer by forcefully removing at end of multi-line command
RUN mkdir -p $HOME/.ssh \
  && echo $PRIVATE_SSH_KEY >> $HOME/.ssh/id_rsa \
  && chmod 600 $HOME/.ssh/id_rsa \
  && ssh-keyscan -H github.com >> $HOME/.ssh/known_hosts \
  && git clone repo:repo.git \
  && rm -rf $HOME/.ssh

COPY . ./
▼

Note that this is a high-level, directional example and will require additional work to use in a live build.

Building With Repos Via API

You can also use the API v2 to programmatically trigger builds that depend on each other, or potentially an external event or system.

To get more details on how to do this, head over to the API Documentation page

Handling secrets
Services and databases