EC-FlowLogCollector is used to collect CloudBees CD/RO and user defined logs from CloudBees CD/RO servers and agents. The plugin can be used in both CloudBees CD/RO standalone servers as well as clusters.
Plugin version 1.2.0.2023050905
Revised on May 09, 2023
Prerequisites
-
This plugin uses the STOMP server for sending and receiving files. Please ensure that STOMP port (61613 by default) on the CloudBees CD Server on which the plugin is installed and executed, is accessible from the CloudBees CD resources from which logs are collected.
To check STOMP server URI, run the command:
ectool getServerInfo
The example of the output:
<response requestId="1" nodeId="10.200.1.167"> <serverInfo> <cluster>0</cluster> <httpPort>8000</httpPort> <httpsPort>8443</httpsPort> <jobEventsDestination>/topic/events.job</jobEventsDestination> <jobEventsSelector>path = '/jobs/$[jobId]'</jobEventsSelector> <putFileDestination>/queue/jobs.$[jobId]</putFileDestination> <putFileSelector>command='PUT_FILE' AND channel='$[channel]'</putFileSelector> <stompClientUri>stomp+ssl://my.server:61613</stompClientUri> </serverInfo> </response>To check connectivity, please run the following command from the resource:
telnet my.server 61613If the telnet command has finished with success, the STOMP server is accessible.
STOMP uri can be configured under server properties:
-
Go to the Administration > Server
-
Click on "Settings"
-
Find the setting "Stomp Client URI"
-
Set it to
stomp+ssl://<accessible hostname>:<STOMP Port>
STOMP server port is configured in commander.properties, under a property called COMMANDER_STOMP_PORT.
-
-
When using the plugin to collect logs from all the nodes of a Cluster, make sure that if the default pool does not include the local agent resources from each of the nodes, the local agent resources are included explicitly in the parameter list for the "Log Target Resources".
-
The parameter "Filter CD Server Logs by Job Info" requires Linux with zgrep utility installed.
Limitations
-
The plugin uses command line execution to retrieve logs from each CloudBees CD resource. In a Cluster topology where Nodes only have remote agents, CloudBees CD Server logs cannot be collected. In other words any node from which server logs needs to be collected should have a functioning local agent resource.
-
The ability to collect CloudBees CD Server Logs based on filtering by Job Info is not implemented for Windows.
Plugin procedures
Collect Logs
Collect Logs parameters
| Parameter | Description |
|---|---|
Log Collector Resource |
Required. Enter the name of the CloudBees CD Resource that performs the work of log collection. Zip archive containing the collected logs will be created in this resource. |
Log Target Resources |
Required. Enter Name of comma separated list of CloudBees CD Resources or Resource Pools from which logs need to be collected. |
Collect CloudBees CD Server Logs |
Collect the commander.log and commander-service.log from the applicable Log Target Resource. |
Collect CloudBees CD Agent Logs |
Collect the agent.log and jagent.log from the applicable Log Target Resource. |
Collect CloudBees CD Repository Logs |
Collect the repository.log from the applicable Log Target Resource. |
Collect Job Logs |
Collect the Job Logs matching the Job IDs provided in the Job ID text area, from the applicable Log Target Resource. |
CloudBees CD Job IDs |
Provide newline separated job IDs to collect job logs. |
Collect Installer Logs |
Collect the corresponding CloudBees CD server/agent installer logs viz., installer.log, uninstaller.log, upgrade.log, setupScripts.log from the applicable Log Target Resource. |
Collect Apache Logs |
Collect the corresponding Apache server logs: access.log, error.log and stdout.log. |
Collect User Defined Logs |
Collect the user specified log names provided in the "User Defined Logs" text area, from the applicable Log Target Resource. User defined log names need to provided with absolute paths. Wildcards are allowed. For example /var/opt/mydir/x.log or /var/opt/mydir/*.log |
User Defined Logs |
User defined log names need to provided with absolute paths. Wildcards are allowed. For example /var/opt/mydir/x.log or /var/opt/mydir/*.log |
Start Timestamp |
Enter either an absolute time stamp as in YYYY-MM-DD HH:MM:SS or a relative time stamp as the number of days before. Valid examples are 2017-09-03, 2017-09-03 22:15:00 and 3. 3 would be interpreted as 3 days before current time. Default will be 1 day before. |
End Timestamp |
Enter either an absolute time stamp as in YYYY-MM-DD HH:MM:SS or a relative time stamp as the number of days before. Valid examples are 2017-09-03, 2017-09-03 22:15:00 and 3. 3 would be interpreted as 3 days before current time. Default will be system time at the time of invocation. |
Filter CloudBees CD Server Logs by Job Info |
Enter job id or job name to fetch only CloudBees CD server logs that contains them. |
Obfuscate Sensitive Data |
Choose this option to obfuscate known type of sensitive data such as email, URL and IP. |
User Defined Sensitive Data |
Provide a REGEX pattern to identify sensitive data other than what is obfuscated out of the box. For example, Username: \w+ or [\w_=].+SERVICENAME=myhost.example.com\). Several regular expressions are separated by a newline. |
Collected Logs Identifier |
Procedure will return all collected Logs in a zip file named as YYYY-MM-DD.zip. Add an identifier such as DEV or PROD for further categorization. For example the categorization of DEV would result in a file named as DEV-CloudBees FlowLogs-2018-09-03.zip. Default for the zip file name is CloudBees FlowLogs-YYYY-MM-DD.zip. |
Result property |
Provide a property path to store run results. Results are generated to the path provided in the archivePath property. |
After the procedure has completed, the zip archive containing the collected logs can be downloaded from Job Summary. The zip archive will be located under the workspace in the agent resource where the log collector is run. The path to the zip archive will be <job workspace>/artifacts/<log archive name>.zip.