Use Apache ZooKeeper v3.9.3 or later to maintain and synchronize group services in a clustered CloudBees CD/RO configuration. For more information, refer to the Apache ZooKeeper website.
For your convenience, ZooKeeper v3.9.3 is bundled in your CloudBees CD/RO installation here: <install dir>/utils
.
To install ZooKeeper:
-
Use either the bundled ZooKeeper or download ZooKeeper from the ZooKeeper website.
-
Extract and install the files into an appropriate location. For example,
/opt/zookeeper-3.9.3
You must install ZooKeeper on an odd number of machines. The number of machines will determine if you install ZooKeeper in standalone mode (for one machine) or in replicated mode (for three or five machines). -
Create a
zoo.cfg
configuration file for each machine with an instance of ZooKeeper. For example,zookeeper-3.9.3/conf/zoo.cfg
.-
For standalone mode on a single ZooKeeper machine, the file has these values:
tickTime=2000 dataDir=/var/lib/zookeeper clientPort=2181
-
For replicated mode across multiple ZooKeeper machines, the
zoo.cfg
file on each server has these values:tickTime=2000 dataDir=/var/lib/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1=<ZooKeeper_hostname_1>:2888:3888 server.2=<ZooKeeper_hostname_2>:2888:3888 server.3=<ZooKeeper_hostname_3>:2888:3888 server.4=<ZooKeeper_hostname_4>:2888:3888 server.5=<ZooKeeper_hostname_5>:2888:3888
Where < ZooKeeper_hostname_1 > through < ZooKeeper_hostname_5 > are the hostnames of the servers for the ZooKeeper service.
The file has only three server value lines for a three-ZooKeeper configuration.
-
-
Create
/var/lib/zookeeper/myid
files on each ZooKeeper server, with each containing a single ASCII digit: 1 for the first server, 2 for the second server, and so on, corresponding to their server.<digit>
values in thezoo.cfg
files -
Create
/var/lib/zookeeper/myid
files on each ZooKeeper server, with each containing a single ASCII digit as follows:-
1 for the first server
-
2 for the second server
-
Up to 5, corresponding to the appropriate server.
<digit>
value in thezoo.cfg
file.
-
-
Start ZooKeeper by entering the following command:
Missing command
Running ZooKeeper as a service on Linux
Install ZooKeeper on machines other than those running CloudBees CD/RO. ZooKeeper must not be run on the same machines as those running the CloudBees CD/RO servers.
To run ZooKeeper as a service (running as the root
user), follow these steps:
-
For each machine on which you want to install ZooKeeper, navigate to the
zookeeper-wrapper.zip
file in the CloudBees CD/RO<install dir>/utils
directory. -
Copy or move the
zookeeper-wrapper.zip
file to each of the machines that you want to use for ZooKeeper. -
Extract
zookeeper-wrapper.zip
to a directory of your choice.For example,
/opt/zookeeper-3.9.3
. -
Using a text editor, open the
wrapper.conf
file in the extractedzookeeper-wrapper/conf
directory.For example,
/opt/zookeeper-3.9.3/zookeeper-wrapper/conf
. -
Add the path to the ZooKeeper home directory.
# Path to unpacked zookeeper set.default.ZOOKEEPER_HOME=/opt/zookeeper-<ZK-VERSION>
For example:
# Path to unpacked zookeeper set.default.ZOOKEEPER_HOME=/opt/zookeeper-3.9.3
-
Add the path to the JRE. For example:
set.default.JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/jre
ZooKeeper has specific OS limitations and JDK version requirements. For more information, refer to Apache ZooKeeper. -
Edit
/opt/zookeeper-3.9.3/zookeeper-wrapper/bin/zookeeper
to change:WRAPPER_ROOT="../"
to
WRAPPER_ROOT="/opt/zookeeper-<ZK-VERSION>/zookeeper-wrapper"
For example:
WRAPPER_ROOT="/opt/zookeeper-3.9.3/zookeeper-wrapper"
-
Verify that you can start the ZooKeeper service by running:
sudo /opt/zookeeper-<ZK-VERSION>/zookeeper-wrapper/bin/zookeeper start
For example:
sudo /opt/zookeeper-3.9.3/zookeeper-wrapper/bin/zookeeper start
-
If the ZooKeeper service starts successfully, you can stop the service using:
sudo /opt/zookeeper-<ZK-VERSION>/zookeeper-wrapper/bin/zookeeper stop
For example:
sudo /opt/zookeeper-3.9.3/zookeeper-wrapper/bin/zookeeper stop
(Optional) Configuring the maximum heap size for ZooKeeper
Configure the max heap size ( Xmx
) for ZooKeeper by uncommenting the following line in /opt/zookeeper-<ZK-VERSION>/zookeeper-wrapper/conf/wrapper.conf
:
#wrapper.java.maxmemory=64
For example to set a 2 GB heap size, use:
wrapper.java.maxmemory=2048
Otherwise, by default as per ZooKeeper Administrator’s Guide, it uses a maximum of 256 MB (if the physical RAM exceeds 1 GB). Refer to the memory recommendation in ZooKeeper Administrator’s Guide.
Configuring the ZooKeeper service auto-restart on Linux
Perform the following procedures to auto-restart the ZooKeeper process upon server reboot or when the process exits abnormally.
Installing daemontools
You can untar daemontools-0.76.tar.gz
into any directory you want, but do not rename that directory after you have done your first package/install
command. If you were to rename it after that first package/install
command, daemontools would silently fail. The build actually queries for the current directory and writes it into various compile files.
-
Log in as
root
. -
Enter the following commands:
mkdir -p /package chmod 1755 /package cd /package
-
Download daemontools by entering:
wget https://cr.yp.to/daemontools/daemontools-0.76.tar.gz
or
curl -o daemontools-0.76.tar.gz https://cr.yp.to/daemontools/daemontools-0.76.tar.gz
-
Install daemontools by entering:
tar -xpf daemontools-0.76.tar.gz rm -f daemontools-0.76.tar.gz yum -y install gcc cd admin/daemontools-0.76
-
In
/package/admin/daemontools-0.76/src/conf-cc
, change:gcc -O2 -Wimplicit -Wunused -Wcomment -Wchar-subscripts -Wuninitialized -Wshadow -Wcast-qual -Wcast-align -Wwrite-strings
to:
gcc -O2 -Wimplicit -Wunused -Wcomment -Wchar-subscripts -Wuninitialized -Wshadow -Wcast-qual -Wcast-align -Wwrite-strings -include errno.h
-
Run:
package/install
The end of the output should look simuliar to:
... Coping commands into ./command... Creating symlink daemontools -> daemontools-0.76... Making command links in /command... Making compatibility links in /usr/local/bin... Creating /service... Adding svscanboot to inittab... inti should start svscan now.
The command tools are in
/package/admin/daemontools-0.76/command
.
Configuring auto-restart using daemontools
You can use daemontools to monitor and start ZooKeeper.
-
Run the following commands to create a ZooKeeper service directory and navigate to it:
mkdir -p /opt/zookeeper-<ZK-VERSION>/service/zookeeper cd /opt/zookeeper-<ZK-VERSION>/service/zookeeper
For example:
mkdir -p /opt/zookeeper-3.9.3/service/zookeeper cd /opt/zookeeper-3.9.3/service/zookeeper
-
Create a file named
/opt/zookeeper-<ZK-VERSION>/service/zookeeper/run
with the following contents:#!/bin/sh echo Starting hello exec /opt/zookeeper-<ZK-VERSION>/zookeeper-wrapper/bin/zookeeper start
For example:
#!/bin/sh echo Starting hello exec /opt/zookeeper-3.9.3/zookeeper-wrapper/bin/zookeeper start
-
Grant execute permissions by entering:
sudo chmod u+x run
-
Create a soft link under the
/service
folder that will be monitored by daemontools by entering:ln -s /opt/zookeeper-<ZK-VERSION>/service/zookeeper /service/zookeeper
For example:
ln -s /opt/zookeeper-3.9.3/service/zookeeper /service/zookeeper
This command attempts to “install” the zookeeper
service but fails, because it adds the following entry to/etc/inittab
:SV:12345:respawn:/command/svscanboot
-
To resolve this issue, open
/etc/inittab
:Search for:
SV:12345:respawn:/command/svscanboot
Comment it out:
#SV:12345:respawn:/command/svscanboot
Configuring auto-restart using daemontools (RHEL 7 and CentOS 7)
RHEL 7 and Centos 7 use systemd
for managing services.
-
Create a file named
/etc/systemd/system/daemontools.service
with the following startup code in it:[Unit] Description=daemontools Start supervise After=getty.target [Service] Type=simple User=root Group=root Restart=always ExecStart=/command/svscanboot /dev/ttyS0 TimeoutSec=0 [Install] WantedBy=multi-user.target
-
Start the daemontools service by entering:
systemctl start daemontools.service
-
Test that the daemontools service is running by entering:
systemctl status daemontools.service
From the output, you can confirm that the ZooKeeper service has started.
-
Enable the daemontools service to start at boot time by entering:
systemctl enable daemontools.service
Processes started by daemontools
Daemontools starts two processes:
-
The
zookeeper-wrapper
process -
The child
zookeeper
Java process started byzookeeper-wrapper
If you terminate the Java process, the zookeeper-wrapper
process restarts it. If you terminate the zookeeper-wrapper
process, the daemontools.service
starts it. In this way, the zookeeper
process will auto-start on server reboot and also when the process exits abnormally.
Running ZooKeeper as a service on Windows
Install ZooKeeper on machines other than those running CloudBees CD/RO. ZooKeeper must not be run on the same machines as those running the CloudBees CD/RO servers.
To run ZooKeeper as a service, follow these steps:
-
For each machine on which you want to install ZooKeeper, navigate to the
zookeeper-wrapper.zip
file in the CloudBees CD/RO<install dir>\utils
directory. -
Copy or move the
zookeeper-wrapper.zip
file to each of the machines that you want to use for ZooKeeper. -
Extract
zookeeper-wrapper.zip
to a directory.For example,
C:\Users\Administrator\zooservice
-
Using a text editor, open the
wrapper.conf
file located in the extractedzookeeper-wrapper\conf
directory.For example:
C:\Users\Administrator\zooservice\zookeeper-wrapper\zookeeper-wrapper\conf
-
Add the path to the ZooKeeper home directory.
# Path to unpacked zookeeper set.default.ZOOKEEPER_HOME=C:\Users\Administrator\zooservice\zookeeper-<ZK-VERSION>
For example:
# Path to unpacked zookeeper set.default.ZOOKEEPER_HOME=C:\Users\Administrator\zooservice\zookeeper-3.9.3
Now you are ready to install and start ZooKeeper as a service.
-
Navigate to and click
InstallZooKeeper-NT.bat
to install ZooKeeper as a service.The file is in the
zookeeper-wrapper\bin
directory.For example:
C:\Users\Administrator\zooservice\zookeeper-wrapper\zookeeper-wrapper\conf
-
Navigate to and click
StartZooKeeper-NT.bat
to start ZooKeeper as a service.The file is located in the
zookeeper-wrapper\bin
directory.For example:
C:\Users\Administrator\zooservice\zookeeper-wrapper\zookeeper-wrapper\conf
If you choose to use the command-line interface or a script to start the service, enter:
ZooKeeperCommand.bat start
Ensuring that ZooKeeper can locate Java
Because ZooKeeper is a Java application, ensure Java is installed and ZooKeeper can locate it. The default value for the JAVA_HOME
setting (in the zookeeper-wrapper/conf/wrapper.conf
file) is:
set.default.JAVA_HOME=/opt/cloudbees/sda/jre
If CloudBees CD/RO is not installed or is not installed in its default directory, set JAVA_HOME
in wrapper.conf
to the location for Java. For example:
set.default.JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
Verifying that ZooKeeper is in standalone mode
The following example shows how to verify that the ZooKeeper service is running in standalone mode:
You can find the ZooKeeper bin directory by running ps -ef | grep "zoo.cfg" . This command will display the location of the zoo.cfg file. For example, zookeeper-3.9.3/conf/zoo.cfg .
|
Get the ZooKeeper status:
/opt/zookeeper-<ZK-VERSION>/bin$sudo ./zkServer.sh status
For example:
/opt/zookeeper-3.9.3/bin$sudo ./zkServer.sh status
The response should be similar to:
JMX enabled by default Using config: /opt/zookeeper-3.9.3/bin/../conf/zoo.cfg Mode: standalone
If you receive the following error:
JMX enabled by default Using config: /opt/zookeeper-3.9.3/bin/../conf/zoo.cfg Error contacting service. It is probably not running.
ZooKeeper might be configured in replication mode and therefore cannot connect to the other nodes in its ensemble.
The status when ZooKeeper is in replication mode will look similar to the following:
ZooKeeper version: 3.9.3-1392090, built on 03/30/2023 17:52 GMT Clients: 10.168.33.13.35821[0](queued=0,recved=1, sent=0) 10.68.33.13.35748[1](queued=0,recved=2189,sent=2189) Latency min/avg/max: 0/0/86 Received: 2198 Sent: 2197 Connections: 2 Outstanding: 0 Zxid: Ox27758 Mode: standalone Node count: 29
In this case, you must configure ZooKeeper in standalone mode and then restart the ZooKeeper service:
-
To stop ZooKeeper, run:
/opt/zookeeper-<ZK-VERSION>/bin$sudo./zkServer.sh stop
For example:
/opt/zookeeper-3.9.3/bin$sudo./zkServer.sh stop
-
The response should be similar to:
JMX enabled by default Using config: /opt/zookeeper-3.9.3/bin/../conf/zoo.cfg Stopping zookeeper ..../zkServer.sh: line 143: kill: (1776) - No such process STOPPED
-
-
To restart ZooKeeper, run:
/opt/zookeeper-<ZK-VERSION>/bin$sudo./zkServer.sh start
For example:
/opt/zookeeper-3.9.3/bin$sudo./zkServer.sh start
-
The response should be similar to:
JMX enabled by default Using config: /opt/zookeeper-3.9.3/bin/../conf/zoo.cfg Starting zookeeper ... STARTED
-
Verifying that ZooKeeper is running
For testing or troubleshooting purposes, it may be necessary to verify the ZooKeeper server is running. To verify ZooKeeper is running:
-
Configure ZooKeeper to recognize four-character test commands:
-
Open the
<zookeeper>/conf/zoo.cfg
file. -
Modify the configuration file one of the following ways:
-
To allow test commands only enter:
4lw.commands.whitelist=stat, ruok, conf, isro
-
To authorize all four-character commands enter:
4lw.commands.whitelist=*
-
-
-
Log in to each ZooKeeper machine and run:
echo ruok | nc 127.0.0.1 2181
-
Confirm that you get a response from each ZooKeeper instance by running:
imok
-
If no response appears or a
broken pipe
error appears, then ZooKeeper is not running.
-
-
Obtain more information about the status of ZooKeeper by logging into each ZooKeeper machine and running:
echo status | nc 127.0.0.1 2181