Email notifications can be very useful when your highly available Jenkins controller(1) fails-over from one node to the other. Such notification can help to review the list of aborted builds or to look at the failed node.
To implement email notification on Jenkins controller failover, please follow the steps:
-
Read the "CloudBees Jenkins Enterprise HA monitor tool" section of the "[CloudBees Jenkins Enterprise » High Availability][CloudBees Jenkins Enterprise _ High Availability]`" user guide
-
On each Jenkins controller of the highly available cluster
-
Download the "jenkins-ha-monitor-1.2-jar-with-dependencies.jar" jar from the [Jenkins HA monitor (active/passive) download section][] (where “1.2” is the version of the tool, you should pick the latest one)
-
Install a shell notification script "send-notification-email-on-jenkins-controller-promotion.sh" similar tomail -s "[Jenkins HA failover] controller
hostname
has been promoted`"
[jenkins-admin@example.com][jenkins-admin_example.com] «< "Jenkins controllerhostname
has been promoted at `date`" -
Launch the process
java -jar jenkins-ha-monitor-4.3-jar-with-dependencies.jar \ -home /mnt/nfs/jenkins-home/ \ -host-promotion /path/to/send-notification-email-on-jenkins-controller-promotion.sh
-
The linux user used to start this process must have read & write access to the shared
JENKINS_HOME
-
-home
is the path to the JENKINS_HOME -
-host-promotion
is the path of the script invoked when the node is promoted
-
-
Sample of logs of the HA Monitor Tool on the Jenkins controller node that gets promoted:
Aug 05, 2014 6:22:19 PM com.cloudbees.jenkins.ha.singleton.HASingleton$3 viewAccepted INFO: Cluster membership has changed to: \[ip-10-0-0-12-45063 (309 bytes)|7\] \[ip-10-0-0-12-45063 (309 bytes), ip-10-0-0-11-33579 (272 bytes), ip-10-0-0-11-46466 (272 bytes), ip-10-0-0-12-54079 (272 bytes)\] Aug 05, 2014 6:22:19 PM com.cloudbees.jenkins.ha.singleton.HASingleton$3 viewAccepted INFO: New primary node is JenkinsClusterMemberIdentity\[member=ip-10-0-0-12-45063 (309 bytes),weight=0,min=0\] Aug 05, 2014 6:22:19 PM com.cloudbees.jenkins.ha.monitor.GraveDigger$1 run INFO: Processing the fail over from JenkinsClusterMemberIdentity\[member=ip-10-0-0-11-8467 (309 bytes),weight=0,min=0\] to JenkinsClusterMemberIdentity\[member=ip-10-0-0-12-45063 (309 bytes),weight=0,min=0\] Aug 05, 2014 6:22:19 PM com.cloudbees.jenkins.ha.monitor.GraveDigger promoteHost WARNING: This host is now hosting the primary node Aug 05, 2014 6:22:19 PM com.cloudbees.jenkins.ha.monitor.GraveDigger run INFO: Executing /home/ec2-user/send-notification-email-on-jenkins-controller-promotion.sh Tue Aug 5 18:22:20 UTC 2014 host promotion Aug 05, 2014 6:22:20 PM com.cloudbees.jenkins.ha.monitor.GraveDigger run INFO: Executed /home/ec2-user/send-notification-email-on-jenkins-controller-promotion.sh: exit code=0
Notes:
-
The jenkins-ha-monitor allows to attach scripts to two events: “on-promotion” when the controller gets promoted as the active controller and “on-demotion” when the controller becomes passive
-
In very specific cases, the “on-promotion” and “on-demotion” events can be invoked on a jenkins controller that is respectively already active or already standing-by. For example, the on-promotion event is triggered on the active node if the monitor-tool restarts
(1) High Availability (active/passive) is a feature of CloudBees Jenkins Enterprise by CloudBees.
CloudBees Jenkins Enterprise _ High Availability (active/passive)