Mask ephemeral secrets in Pipeline build logs

5 minute readSecurity
CloudBees Beta

This feature is available as a beta release and is subject to change without notice. CloudBees recommends stringent testing in a development environment and a complete review of the documentation and architecture before using it in production.

Protect ephemeral credentials from accidental exposure by automatically masking them as they appear in Pipeline build logs and step arguments. Unlike stored credentials, ephemeral credentials are generated at runtime and are never stored in the Jenkins credentials store.

Mask credentials without Pipeline changes

Once masking patterns are configured, they apply globally to all Pipeline builds on the controller. Pipeline authors do not need to modify their Pipelines to benefit from this protection.

Configure patterns for your environment

Choose from stock patterns for common ephemeral credential formats or define custom regular expression patterns to match token formats specific to your environment.

Enforce secrets hygiene

Optionally configure patterns to fail the build if a secret is detected in the build log, signaling to Pipeline authors to fix their Pipelines rather than relying on masking as a permanent solution.

CloudBees CI version 2.528.1.29783 or later is required.

Masking behavior

Patterns define the credential formats to detect and mask. You can configure any pattern to fail the build if matched, to discourage Pipeline authors from accidentally printing secrets to logs. CloudBees CI always logs a warning to the controller system log when masking occurs.

Stock patterns

Stock patterns are predefined patterns for ephemeral credential formats, such as AWS Security Token Service (STS) access key IDs, AWS secret access keys, AWS session tokens, and JWTs. For the full list of available patterns and their descriptions, refer to Manage Jenkins  Security in the CloudBees CI UI.

  • The AWS STS access key ID pattern does not match long-term access key IDs stored in the Jenkins credentials store.

  • The AWS secret access key pattern can produce false positives (for example, from strings containing both mixed case and digits).

Custom patterns

Custom patterns allow you to define your own regular expression patterns for token formats specific to your environment. Each custom pattern requires a name to identify it in system log warnings and failure messages.

Replacement text

The replacement text is a string used to replace matched secrets in Pipeline build logs and step arguments. The default value uses a pattern that is visually distinctive in log output. You can specify any static text, or include the NAME placeholder to substitute the pattern name at runtime. For stock patterns, this is the predefined pattern name. For custom patterns, this is the name you specify when configuring the pattern. The replacement text applies to all patterns, both stock and custom.

Step argument masking

Masking also applies to step arguments (the parameters passed to Pipeline steps), which are visible in Pipeline visualization tools, such as CloudBees Pipeline Explorer. When double quotes are used for the script argument of sh, bat, powershell, and pwsh steps, the secret value is embedded directly in the step argument; the plugin detects and masks these, but matched step arguments are not displayed in the Pipeline Steps view.

The script argument is the default argument for sh, bat, powershell, and pwsh steps. For example, sh 'make world' is equivalent to sh script: 'make world'.

As a best practice, use single quotes for the script argument so step arguments are not exposed in Pipeline visualization tools.

Configure masking patterns

Choose one of the following configuration methods:

Configure masking patterns in the UI

  1. Install the CloudBees Mask Secrets plugin to the controller.

  2. Navigate to Manage Jenkins  Security.

  3. Scroll down to Automatic pattern masking.

  4. To specify a stock pattern:

    1. Select + Add.

    2. Select the appropriate pattern.

    3. (Optional) To fail the build if the pattern is matched, select Fail build on match.

  5. To add a custom pattern:

    1. Select + Add custom pattern.

    2. Enter a Regular expression for the pattern to match.

    3. Specify a Name for the pattern, which is used in system log warnings and failure messages.

    4. (Optional) To fail the build if the pattern is matched, select Fail build on match.

  6. If needed, update the default Replacement text for matched secrets.

  7. Select Save.

Configure masking patterns using Configuration as Code

You can use Configuration as Code (CasC) to define stock and custom masking patterns.

CloudBees recommends using CasC Plugin Management 2.0 to automatically manage plugin dependencies. CasC Plugin Management 2.0 requires apiVersion:2 in the bundle.yaml file.

To configure masking patterns using CasC:

  1. Add the CloudBees Mask Secrets plugin to the controller plugins.yaml file:

    plugins: - id: cloudbees-mask-secrets
  2. Update the controller jenkins.yaml file to configure masking patterns:

    security: cloudbeesMaskSecrets: stockPatterns: - awsStsAccessKeyId - awsSecretAccessKey - awsSessionToken: failBuild: true(1) - jwt customPatterns: - name: internal-api-key(2) regexp: key_[A-Za-z0-9]+(3) failBuild: true(1) replacementText: '****NAME****'(4)
    1 Optional. Set to true to fail the build if this pattern is matched, to discourage Pipeline authors from accidentally printing secrets to logs; applies to stock and custom patterns.
    2 A name for the pattern, which is used in system log warnings and failure messages.
    3 A Java regular expression for the pattern to match.
    4 The replacement text for matched secrets. Include NAME to substitute the pattern name at runtime.
  3. Apply the bundle to your controller. For more information, refer to Update a CasC bundle.

Security considerations

Masking ephemeral secrets in build logs is a secondary, best-effort defense against accidental disclosure. Malicious Pipeline authors can bypass masking trivially, or steal secrets without involving the build log at all. Pipeline authors should avoid printing secrets to logs in the first place.

Encoding and escaping limitations

Patterns match against log text as it appears; if any process transforms a secret before it appears in the log, the match fails.

Shell metacharacter escaping

When a shell prints a variable, it may escape characters such as "'[{(*&$` that are present in the secret value. For hexadecimal and Base64 patterns, this is unlikely to be an issue, since these formats do not contain shell metacharacters.

Secret encoding

CloudBees CI does not mask secrets embedded in Base64-encoded output. For example, credentials returned by aws sts assume-role are masked when they appear as-is in a kubectl create secret command. However, retrieving those credentials later with kubectl get secret -o yaml returns them Base64-encoded, and the encoded form is not matched.

Agent temp file exposure

For any form of masking in CloudBees CI, including injecting secrets into builds, log text from sh, bat, powershell, and pwsh Pipeline steps is streamed to a temporary file on the agent before being forwarded to the controller or external log storage. Masking occurs during this forwarding process, so secrets exist in plaintext in the temporary file on the agent.

On High Availability (HA) controllers, masking occurs on the agent rather than the controller. A message is sent to the controller to support failing build behavior and system log warnings.

On permanent agents or cloud agents with multiple executors, all builds run under the same OS account. Builds belonging to other jobs can therefore read files from concurrently running builds, inspect process arguments, and in some environments read process memory. Even if the agent only has a single executor, a malicious build can set up a long-running process to steal secrets from future builds running on the same agent.

Do not reuse agents across teams that must be security-isolated from one another. This applies to any situation in which an agent has access to privileged information, including regular usage of Jenkins credentials.