How to deploy monitoring support
Use Grafana to visualize the Elasticsearch metrics. Issue the following commands to deploy it at your site:
helm install --name prometheus stable/prometheus helm install --name grafana stable/grafana # Note that username is admin and password is the result of following command kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | \ base64 --decode ; echo RandomPassword
How to deploy logging support
In general, each object in a Kubernetes cluster produces its own logs. And usually the user has their own mechanism in place to manage logs from different services in the cluster. Logs from CloudBees CD/RO services and pods can be captured with standard log shipper tools. A sample configuration file for the FileBeat log shipper is provided here.
How to configure agents to share a workspace
Once the first agent is deployed with ReadWriteMany access mode, subsequent agents deployed for the same workspace with storage.volumes.agentWorkspace.existingClaim to true share the first agent’s workspace. The following example shows how to set up flow-agent-1 and flow-agent-2 to share the same workspace, MyWorkspace.
-
Deploy the first agent with
storage.volumes.agentWorkspace.accessModeset toReadWriteMany. This creates the persistent volume claim, setting up the scenario where agents can use theflow-agent-workspaceshared workspace.helm install flow-agent-1 cloudbees-flow-agent -f <valuesFile> \ --set storage.volumes.agentWorkspace.accessMode=ReadWriteMany \ --set storage.volumes.agentWorkspace.name=MyWorkspace \ --namespace <nameSpace> --timeout 10000 -
Deploy subsequent agents to the same workspace with
storage.volumes.agentWorkspace.existingClaimtotrue.helm install flow-agent-2 cloudbees-flow-agent -f <valuesFile>\ --set storage.volumes.agentWorkspace.existingClaim=true \ --set storage.volumes.agentWorkspace.name=MyWorkspace \ --namespace <nameSpace> --timeout 10000
The following table summarizes parameters used to configure a shared agent workspace. Also see Persistent storage.
| Parameter | Description |
|---|---|
|
Define the workspace access mode. Possible values include For shared workspaces use |
|
The agent workspace name. Use the same name across all agents sharing the same workspace. If not specified, Specify the same name across all agents that share the workspace. |
|
The amount of storage to allocate. For shared workspaces, allocate approximately 5 GiB per agent. Increase based on the agent’s requirements. |
|
Whether to use the existing claim for a previously deployed agent to share its workspace. Set to |
How to increase memory limits for CloudBees CD/RO components
During periods of high work load, a server component could run out of memory if it asks for more memory than is allocated to the JVM. In order to increase the memory for a component, we have to allocate more memory to the component’s container. Then, depending on the component, the memory allocation for the component, itself, running in the container needs to be increased accordingly. See Cluster capacity for default container memory settings.
The following configurations can be used to change the memory allocation for each container and component.
| Component | Container memory limit | Component memory setting | Example |
|---|---|---|---|
CloudBees CD/RO server |
|
server.ecconfigure |
|
CloudBees CD/RO web server |
|
N/A |
|
Repository server |
|
repository.ecconfigure |
ecconfigure: " --repositoryInitMemoryMB=256 --repositoryMaxMemoryMB=512" |
CloudBees Analytics server |
|
|
|
Bound agent |
|
|
|
Inject new memory limits using helm . Update your local values file (here it is called myvalues.yaml ) with the new values and issue the Helm `upgrade ` command.
helm upgrade <chartName> --name <releaseName> \ -f <valuesFile> --namespace <nameSpace> --timeout 10000
How to resolve volume note affinity conflicts
Sometimes pods can hang in the `pending ` stage with the following error:
x/y nodes are available: y node(s) had volume node affinity conflict.
This can happen when the availability zone for the persistent volume claim is different from the availability zone of the node on which the pod gets scheduled.
A cluster administrator can address this issue by specifying the WaitForFirstConsumer mode, which delays the binding and provisioning of a PersistentVolume ` until a pod using the `PersistentVolumeClaim is created. PersistentVolume s are selected or provisioned conforming to the topology that is specified by the pod’s scheduling constraints.
For more information see this article: https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode
Data backups and disaster recovery
Kubernetes backup tools such as Heptio’s Velero (previously known as Ark) are used to backup and restore Kubernetes clusters. CloudBees CD/RO services and pods deployed in the cluster can be backed using those standard backup tools. Application data maintained in persistent volumes needs to be backed up at the same time. Tools such as Velero support persistent volume backups.
Here is the list of volumes associated with the CloudBees CD/RO pods:
| Container | Volumes |
|---|---|
flow-server |
efs-deployment, logback-config,init-scripts |
flow-agent |
flow-agent-workspace, logback-config |
flow-devopsinsight |
elasticsearch-data |
flow-repository |
efs-repository, logback-config |
flow-web |
efs-deployment |