CLI release notes

Support for the injecting of the new environment variable CI_PR_NUMBER and CI_PULL_REQUEST into builds. Given a PR number is present, these variables will be populated in the CloudBees CodeShip Pro system.

Slight change to step naming. Duplicate naming of steps is supported. The output of duplicate step names will lead to a bracketed counter. i.e. Two steps name foo will be presented as foo and foo (2). This kind of counter based de-duplication is only enforced between sibling steps. Two steps name foo which are not siblings will remain named foo.

CI_BUILD_APPROVED boolean environment variable, indicating whether or not manual approval has been given.

Support for leveraging identitytoken field in docker config. Docker config generators which produce config containing an identity token rather than a username and password will authenticate successfully with the target repository. Prior to this change such authentication flows would result in an authentication failure. This reinstates our ability to support platforms such as IBM Cloud (see https://github.com/codeship-library/ibm-bluemix-utilities). === Changed

Building Jet with Go 1.11 == 2.7.0 - 2018-07-18

jet completion command to generate bash/zsh completions for Jet == 2.6.1 - 2018-06-07

Nil pointer panic if decrypting auth was attempted and no AES key was present.

TLS verify disabled when mounting unix domain socket via add_docker directive.

Sign all new files encrypted with jet encrypt with a checksum to verify both file and key integrity.

Ensure containers with failing healthchecks are cleaned up when Jet exits.

Support for manual approval step types.

Do NOT html escape build arguments when interpolating CI arguments in to them.

Retry up to 3 times to connect to Docker at startup.

Ability to use template variables such as `` as service build arguments == 2.3.1 - 2018-01-30

Image build/pull behavior if more than one service reference the same image.

Caching of Docker login authentication between steps. Logins provided in the form of either an encrypted_dockercfg_path or dockercfg_service directive, will be cached for up to 10 minutes. This will ensure we don’t exhaustively make calls to Docker login authentication providers. It also improves our ability to collapse concurrent builds of services to a single build call, when creating services which rely on these directives. Print warning for services that use links. For more information please see Deprecated keys for more info.

healthchecks directive for healthchecks built into a Docker image. For images that contain a healthcheck, we will poll for availability every 1 second for up to 60 minutes before proceeding. You can find the health polling status in your logs.

Print warning for service names that include underscores as it can cause problems with Docker networks.

Stop printing build arguments when pretty printing image types. Don’t start containers which are purely referenced by another service using the volumes_from directive.

depends_on service directive. Now with support for the depends_on key in the codeship-services.yml. This behaves similarly to links, in the sense that it describes the dependency of one service to another. The difference is that it does not support any kind of additional networking changes. The legacy links directive supported network aliasing and “service discovery” environment variables. These are now not necessary as each service is discoverable via DNS using it’s service name as the address.

The above example is all that is required from a dependency perspective, for a service app to boot and talk to a dependent service server over the network via DNS.

jet update command to allow Jet to update itself to the most recent version == 2.1.0 - 2018-01-16

jet cleanup command for removing resources left behind by previous jet commands. For example, when the --no-remove flag is used it leaves containers and networks around. The cleanup subcommand will remove these Docker resources. jet validate command to validate codeship-services.yml and codeship-steps.yml files == 2.0.1 - 2018-01-12

Deprecation warnings now always show as yellow in CLI === Fixed

Require both version and services top-level keys to be present for services files to be seen as versioned. Ensure environment variables describing --link port mappings are ordered. Such that the mapping <alias>_PORT always contains the lowest port mapping described. As per Docker’s legacy links spec. Skip push steps by default in CLI == 2.0.0 - 2018-01-08

Isolated networks for steps.

Major rewrite.

You can now define a set of steps to be executed when any run, push or group step fails. This will allow users to define recovery options when steps do fail. Any time a regular step fails, the build will fail regardless of whether the resulting failure steps fail or succeed. You can also define recursive failure pipelines for steps that are part of a failure pipeline themselves. For more information see the CloudBees CodeShip Documentation on Failure Pipelines.

Fix for error when running jet run against Docker 1.13. Containers now wont get created with a trailing slash in their name when using jet run (does not affect jet steps).

In codeship-services.yml, you can now declare build arguments to be consumed at image build time. Check out Build arguments.

Jet can now handle parsing all services indented in a services section of the yml/json CodeShip services file. Configurations with services indented under a services section will be treated the same as configurations where all services are defined on the top level.

In your codeship-services.yml file, you can now specify a context key that contains a directory with a Dockerfile. This is identical to the existing path key but more in line with Docker’s naming conventions. The path key will still be supported. Note: context must contain a directory, not a URL to a git repository.

If a step is skipped due to a tag or exclude field on the step, the step status will now be shown as type:STEP_FINISHED_TYPE_SKIPPED.

This release updates how Jet catches circular dependecy issues to account for aliased links and volume_from entries. With this in place Jet will correctly error immediately after detecting a circular service dependency through links or volumes_from when an alias us used for the link or volume reference.

In a recent change we added validation requiring an “=” in all environment variables. We are removing this check but keeping new checks that error if non UTF8 characters are present in environment variables. This should help users detect issues with encrypted environment variables not decrypting as expected.

This change fixes a bug where the environment parsing for CodeShip values was duplicating keys.

This change fixes some bugs around how we parse and validate environment. With this change we ensure we correctly parse empy values, strip out comments or blank entries, and error actively on invalid ones.

Stripping blank strings out of Environment Jet will now ignore all blank strings added to environment instead of passing them into the container config. This includes those present if env files and those encrypted.

Jet will now validate environment variables before attempting to start containers. This helps protect against triggering errors in the underlying OCI implementation. Environment variables must contain only valid UTF8 characters and at least one ‘=’.

This release changes how we compile Jet to update from Go 1.6 to Go 1.7. Go 1.6 has some known issues on OSX Sierra, this changes fixes some issues with running Jet on OSX Sierra.

This flag has been marked as deprecated since July. For local builds, rely on the local Docker image cache. CodeShip no longer provides a remote cache during local build runs.

Jet log fetching has been updated to retry several times when the docker logs request returns a non error response. This allows us to capture logs for more than ~2 hours.

When using cached services, you can now specify which branch to use as a fallback if no cached image is found for your service. The default fallback branch is master. To override this, add default_cache_branch to your service in the codeship-services.yml file. If no value is present, the cache will fall back to master. Note that caching only works for builds run on our infrastructure and not with the Jet CLI.

If two services use an identical cached image and attempt to load the service in parallel, the first service is responsible for loading it, and other services will wait. Services will also check to see if the image ID is available on the build machine before downloading.

A bug in the AWS SDK for golang did not return the correct error code for S3 HeadObject request errors, which prevented new cached services from falling back to the master cache because an empty error was returned. We’ve updated the SDK to include the bugfix, and updated our internal handling of AWS errors.

With 1.9.0 we introduced a bug around how services were built. This version fixes that bug and allows images to be built in parallel.

This change updates how Jet pulls and builds images to parallelize it where possible. This should mean your jet steps and jet run commands are faster, as well as when you run jet load with multiple services specified

This change updates the way cached images are stored and retrieved. Cached images are now imported and exported using Docker’s save and load mechanisms, instead of being pushed and pulled from a registry. For local builds using the Jet CLI, remote caching has been removed. Instead, rely on the local Docker image cache. For more information, see our documentation on Image caching.

bugs fixed: If the user specifies a tag for a cached image, CodeShip will append our cache tag to the existing image tag using a hyphen as a delimiter.

This change ensures that images which are tagged and pushed for push steps or for caching are provided with a name which Docker considers to be valid.

This change updates the StringTime environment field to use RFC3339 format in order to remove spaces from the env variable. Values in this field will now be in the format “2006-01-02T15:04:05Z07:00”

This release changes the version of Go used to build the jet binary from 1.5.2 to 1.6.0.

This change adds a –remote-cache flag to allow users to enable the remote cache when using the jet tool. The remote cache will now be off by default.

This change disables the cache tag pushing and pulling with –no-cache=true. This means that if that flag is present, no cache docker images will be pulled or pushed.

This change ensures that all GCR and ECR dockercfg are refreshed before they expire.

This change updates what remote docker api we use to manage build containers from 1.18 to 1.22.

This release updates Jet to v1.0. No other changes are included.

Updating docker parameters to pass memory limits and CPU shares correctly for build containers. This will enforce CPU shares and memory limits defined in the services file when steps are being executed

Fixes a bug where links used alongside volumes would cause a race condition. Links can now be used alongside containers.

This change fixes an intermittent issues where jet would fail sometimes when cleaning up containers on exit. Jet will no longer panic and exit when an issue cleaning up occurs, but will handle the issue gracefully and continue.

This change fixes the regex used to parse image tag names and determine authentication using the dockercfg file.

This change updates the fork of fsouza/go-dockerclient CodeShip uses to be based off of 299d728486342c894e7fafd68e3a4b89623bef1d, pulling in some recent bug fixes from upstream.

Updating method used to transfer credentials to use relative volumes mounted between containers. This makes dockercfg credential transmission valid on the hosted CI service

Switching directories used to store generated dockercfg which was causing an error due to the directory not existing by default. Jet will now use a standard base directory. This should not be a user-visible change

The method jet uses to parse urls from image names is now more standardized, allowing all valid url characters to be used in the registry url. “foo-bar.com/myrepo/myimage” will now reference the registry “foo-bar.com”.

Adding support for using services to generate dockercfgs Jet users can now use a service from their codeship-services file to write a custom dockercfg, or use a secondary authentication method to unlock docker credentials.

You can now specify an encrypted dockercfg to use for a specific service in the services file, rather than in all relevant steps of the steps file. This should be particularly useful when using caching, or when using private base images.

With this change when caching pulls error, the build fails. This change turns those pulls into warnings and allow the build to continue.

If a user specifies an image tag template, but the generated image tag string is empty, Jet will now error. If a user does not specify an image_tag, or specifies an empty string Jet will use the default tag, “latest”.

This only affects non-ci jet runs. If the ci-branch flag is not specified by the user, then “master” is assumed as the cache source. This change ensures that if the branch is not provided then the blank tag is not pulled, or pushed once the image has been built.

This change ensures that configured volumes for containers are evaluated as full paths in the volume definition. This allows volumes to be mounted to remote or virtualized docker instances.

This change fixes several bugs around add_docker. You can now use a local Docker socket and it will be mounted into the container. This change also stops converting “tcp” host URLs to “https” when TLS is being used as this is no longer supported by Docker.

This change fixes a bug where jet diverged from how docker-compose handled local relative directory mounts. With this change in place, you can mount folders within your build directory.

Jet will now tag images built for services based on the image name specified in the push step. This allows different services to build the images with different names, push to the same repository.

With this change jet will remove link and volume containers upon error, unless the --no-remove flag is specified.

In order to use regex matching, the regex must include one of the regex line matchers $ and ^. This reduces the chances of an intended strict tag match matching a substring of a longer tag.

This change sanitizes the cache tag generated since branches may contain characters which are not valid for tags

Jet will now detect circular dependencies in the services file. This matches the behaviour of docker-compose, which errors when reading a compose.yml which contains a circular loop.

This change ensures that jet honors termination signals during builds and pulls. The relevant build and pull will still continue, however jet will not build, pull or run any further services, and exit.

Fixing cleanup bugs This change fixes some issues with how jet cleans up containers when shutting down and ignores container not found errors when trying to destroy containers.

Cache pulls which have registry errors will be ignored since they are triggered in cases by a not found response. Since a cache may not exist for a given build, this can cause false negatives.

Removing error on missing dockercfg credentials This change removes the error raised when no credentials are found for a specific registry. This may not actually be an configuration issue.

When using cached jet builds, the cached image will now be pushed regardless of whether a push step was run.

This update synchronizes duplicate image pulls to avoid race conditions. Multiple services with the same image will result in a single image pull. Those services will wait for the initial pull to complete.

Fixing duplicate build bug

This update fixes a bug with the synchronization of duplicate image builds. Previously services which depended on an image already being built would immediately continue as soon as a previous, in-progress build was detected

Steps files can now use regex to determine what steps to run e.g. tag: “test-\w+” matches test-feature1, test-123. Exact string matches take precedent, and invalid regex will not error, only fail to match.

This change extends the pull errors we discard during a cache pull. All pull errors relating to the image not being found are discarded. This change makes that selection process more compatible with different registries.

Updating jet to pull image cache before building

Jet will now determine an image cache to pull and use as a build cache during docker image builds. If an image exists with a tag “codeship-cache-$BRANCH”, it will be pulled and provide faster builds. Jet will also automatically push to this cache when an image is pushed.

Jet will now error when no AES key is found, but an encrypted dockercfg file is used

This change adds support for Dockerfile image bases from private registries

We look for a codeship.aes in your project directory instead of $HOME

This change resolve an issue where links and volumes were not loading correctly.

This change fixes a bug where jet would give false positives when detecting cases where images are being rebuilt

This change fixes a bug where a single image was being build multiple times for a specific set of steps. All steps now use the same image which is built once

This fixes a bug where image uniqueness checks were causing builds to error

Jet run will now exit if an error occurs while starting services

Jet will now return an error if multiple services try to build the same image name

This change renames Commit to CommitID to bring jet in line with other CI systems. The Commit ID/SHA can not be accessed via for push tags and via the CI_COMMIT_ID environment variable within containers

The same variables used for Docker image tagging will now be available as environment variables in any containers running steps. We have also extended what variables are available. Here is a complete list:

For tagging within push steps the following variables will also be available, and the resulting template will be stripped of any invalid characters.

We have enabled a simple retry mechanism when pushing docker images. Often simultaneous image pushes cause problems, so when an image push step receives one of several listed errors from the registry, the push will be attempted again. You may see some duplicate logging where push steps ran several times due to registry errors.

Jet will also pick up link container creation errors

Jet reads services/compose file more in line with docker-compose.

Previously in relation to service Dockerfile paths and the build declaration jet and docker-compose would differ. docker-compose would allow you to specify a context directory for the entire build, not just the location of the Dockerfile, while jet would enforce using the top level directory as the context directory for the entire build.

Now you can specify a context directory for building an image by using the build: custom_directory tag in your services file, which should match docker-compose in behavior.