Getting started
IntroductionInstalled packagesEnvironment variablesDebugging buildsCachingParallel testingDeployment pipelinesRunning commands as rootIgnoring commandsRun a command in case of an errorSetting timezoneAccess to other repositoriesUsing scriptsGit LFSUbuntu 20.04 Focal
IntroductionFTP/SSH/SFTP/RSYNCCustom scriptHerokuHeroku MigrationAWS Elastic BeanstalkAWS CodeDeployAWS LambdaAWS S3Google App EngineAWS CLIgcloud CLIAzure CLIDigital OceanEngine YardCapistranoGit PushCloudFoundryCloud66anynines
IntroductionBrowser testingBackground commandsGit submodulesCustom DNSArtifacts
IntroductionCassandraCouchDBDynamoDBMongoDBMySQLPostgreSQLSQLite
IntroductionDartElixirGoJava and JVMMeteorNode.jsPHPPythonRubyRust
IntroductionBeanstalkdRabbitMQRedis
IntroductionElasticsearchMemcachedSphinx
IntroductionCloudBees CodeShip Pro introduction guideCloudBees CodeShip Pro introduction guide 2CloudBees CodeShip Pro introduction guide 3CloudBees CodeShip Pro introduction guide 4CloudBees CodeShip Pro introduction guide 5
Migrating to CloudBees CodeShip Pro
Docker for CI/CD
Getting started with CloudBees CodeShip Pro overview and resources
IntroductionServices configSteps configEnvironment variablesImage registriesImage cachingBuild argumentsVolumesHandling secretsMultiple reposServices and databasesSetting an SSH private keyPreinstalled images
IntroductionInstallationUsing the Jet CLIjet cleanupjet decryptjet encryptjet generatejet loadjet runjet stepsjet updatejet validatejet version
IntroductionAWSAWS S3AWS EBAWS ECSAWS EKSAWS CodeDeployAzure Container ServiceHerokuGoogle CloudGoogle App EngineGoogle Cloud FunctionsGoogle Container EngineGoogle Compute EngineKubernetesTerraformDeploy Via SSHDocker SwarmIBM Cloud FoundryIBM Cloud Container Service
IntroductionBrowser testingParallel testing
IntroductionElixirGoJava and JVMNode.jsPHPPythonRuby
IntroductionCaching multi-stage DockerfileContainer is slow to exitDiffering results with Jet CLIDocker Hub rate limitingManaging multiple environments for a serviceReducing log output from dependent containersStep log not appearingUsing env vars in command steps

Parallel testing

2 minute read

What are parallel test pipelines?

Parallel test pipelines allow you to split your test commands across multiple test pipelines to speed up your build time.

Note that all test pipelines run on separate build VMs to avoid the possibility of race conditions or resource limitations.

Using Parallel Test Pipelines

Once parallel test pipelines are enabled, each project can have multiple test pipelines that will be run in parallel. CodeShip will first run your setup commands and then any arbitrary test commands you defined for this specific pipeline via the interface.

To ease distinguishing different pipelines you can provide a name for each one.

Configuration of test pipelines

Restarting Individual Pipeline

If a pipeline fails, you can restart only that individual pipeline by hovering over the failed pipeline’s status indicator.

Restarting test pipeline

Disabling Parallel Test Pipelines

You can either delete additional test pipelines, or comment out any commands by prepending a # symbol to each line. A test pipeline is only active if it contains at least one command.

Deployment Pipeline

Using parallel test pipelines is unrelated to creating your deployment pipelines, which are specific to deploying your application at the end of your tests and can not run in parallel.

You can learn more about deployment pipelines here.

When using a single test pipeline, the same build container is used for the entire build and deploy so any configuration done during the setup and testing phase will also be present for the deployment phase. When using multiple test pipelines, a new container is used for the deploy and setup commands do not run again. If your deployment has any specific dependencies you will want to include those commands with your deployment steps.

Artifacts Support

As your build and deployment commands are run on multiple virtual machines, artifacts created during the test steps will not be available during the deployment. If you need artifacts from the previous steps, make sure to regenerate them during the deployment using a script deployment added before the actual deployment.

Account Downgrade Behavior

If you downgrade to a subscription with fewer parallel pipelines, any additional pipelines will be automatically merged into the first pipeline on the project.

If this is not desirable for your project make sure to manually move the steps to the appropriate test pipelines before downgrading.

Parallel Modules

In addition to parallelizing explicitly with parallel pipelines, most popular frameworks offer modules that you can install to parallelize within the codebase itself.

While we do not officially support or integrate with any of these modules, many CodeShip users find success speeding their tests up by using them. Note that in many cases these modules create additional strain on your machine resource usage, so you will want to keep an eye on this as misconfiguration can result in a resource max out that ultimately slows your builds down or causes failures.

Rails - https://github.com/grosser/parallel_tests - https://github.com/ArturT/knapsack

Node - https://www.npmjs.com/package/mocha-parallel-tests - https://www.npmjs.com/package/mocha-pipelines

PHPUnit - https://github.com/paratestphp/paratest

Caching
Deployment pipelines