Frequently asked questions

3 minute readTroubleshooting

How to deploy monitoring support

Use Grafana to visualize the Elasticsearch metrics. Issue the following commands to deploy it at your site:

helm install --name prometheus stable/prometheus helm install --name grafana stable/grafana # Note that username is admin and password is the result of following command kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | \ base64 --decode ; echo RandomPassword

How to deploy logging support

In general, each object in a Kubernetes cluster produces its own logs. And usually the user has their own mechanism in place to manage logs from different services in the cluster. Logs from CloudBees CD/RO services and pods can be captured with standard log shipper tools. A sample configuration file for the FileBeat log shipper is provided here.

How to configure agents to share a workspace

Once the first agent is deployed with ReadWriteMany access mode, subsequent agents deployed for the same workspace with storage.volumes.agentWorkspace.existingClaim to true share the first agent’s workspace. The following example shows how to set up flow-agent-1 and flow-agent-2 to share the same workspace, MyWorkspace.

  • Deploy the first agent with storage.volumes.agentWorkspace.accessMode set to ReadWriteMany. This creates the persistent volume claim, setting up the scenario where agents can use the flow-agent-workspace shared workspace.

    helm install flow-agent-1 cloudbees-flow-agent -f <valuesFile> \ --set storage.volumes.agentWorkspace.accessMode=ReadWriteMany \ --set storage.volumes.agentWorkspace.name=MyWorkspace \ --namespace <nameSpace> --timeout 10000
  • Deploy subsequent agents to the same workspace with storage.volumes.agentWorkspace.existingClaim to true.

    helm install flow-agent-2 cloudbees-flow-agent -f <valuesFile>\ --set storage.volumes.agentWorkspace.existingClaim=true \ --set storage.volumes.agentWorkspace.name=MyWorkspace \ --namespace <nameSpace> --timeout 10000

The following table summarizes parameters used to configure a shared agent workspace. Also see Persistent storage.

Parameter Description

storage.volumes.agentWorkspace.accessMode

Define the workspace access mode. Possible values include ReadWriteMany and ReadWriteOnce.

For shared workspaces use ReadWriteMany.

storage.volumes.agentWorkspace.name

The agent workspace name. Use the same name across all agents sharing the same workspace. If not specified, flow-agent-workspace is used.

Specify the same name across all agents that share the workspace.

storage.volumes.agentWorkspace.storage

The amount of storage to allocate.

For shared workspaces, allocate approximately 5 GiB per agent. Increase based on the agent’s requirements.

storage.volumes.agentWorkspace.existingClaim

Whether to use the existing claim for a previously deployed agent to share its workspace.

Set to true to share the existing claim for storage.volumes.agentWorkspace.name.

How to increase memory limits for CloudBees CD/RO components

During periods of high work load, a server component could run out of memory if it asks for more memory than is allocated to the JVM. In order to increase the memory for a component, we have to allocate more memory to the component’s container. Then, depending on the component, the memory allocation for the component, itself, running in the container needs to be increased accordingly. See Cluster capacity for default container memory settings.

The following configurations can be used to change the memory allocation for each container and component.

Component Container memory limit Component memory setting Example

CloudBees CD/RO server

server.resources.limits.memory

server.ecconfigure

server.ecconfigure: "--serverInitMemoryMB=4096 --serverMaxMemoryMB=4096"

CloudBees CD/RO web server

web.resources.limits.memory

N/A

Repository server

repository.resources.limits.memory

repository.ecconfigure

ecconfigure: " --repositoryInitMemoryMB=256 --repositoryMaxMemoryMB=512"

CloudBees Analytics server

dois.resources.limits.memory

dois.esRam (heap size in MB for Elasticsearch)

dois.lsInitRam , dois.lsMaxRam (heap size in MB for Logstash)

Bound agent

boundAgent.resources.limits.memory

boundAgent.ecconfigure

ecconfigure: "--repositoryInitMemoryMB=256 --repositoryMaxMemoryMB=512"

Inject new memory limits using helm . Update your local values file (here it is called myvalues.yaml ) with the new values and issue the Helm `upgrade ` command.

helm upgrade <chartName> --name <releaseName> \ -f <valuesFile> --namespace <nameSpace> --timeout 10000

How to resolve volume note affinity conflicts

Sometimes pods can hang in the `pending ` stage with the following error:

x/y nodes are available: y node(s) had volume node affinity conflict.

This can happen when the availability zone for the persistent volume claim is different from the availability zone of the node on which the pod gets scheduled.

A cluster administrator can address this issue by specifying the WaitForFirstConsumer mode, which delays the binding and provisioning of a PersistentVolume ` until a pod using the `PersistentVolumeClaim is created. PersistentVolume s are selected or provisioned conforming to the topology that is specified by the pod’s scheduling constraints.

For more information see this article: https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode

Data backups and disaster recovery

Kubernetes backup tools such as Heptio’s Velero (previously known as Ark) are used to backup and restore Kubernetes clusters. CloudBees CD/RO services and pods deployed in the cluster can be backed using those standard backup tools. Application data maintained in persistent volumes needs to be backed up at the same time. Tools such as Velero support persistent volume backups.

Here is the list of volumes associated with the CloudBees CD/RO pods:

Container Volumes

flow-server

efs-deployment, logback-config,init-scripts

flow-agent

flow-agent-workspace, logback-config

flow-devopsinsight

elasticsearch-data

flow-repository

efs-repository, logback-config

flow-web

efs-deployment