Issue
Build directories continue to exist on the filesystem even though the builds do not appear in the UI.
Log rotation appears to have deleted the builds from the controller, but the corresponding directories in $JENKINS_HOME/jobs/$JOB_NAME/builds remain on disk.
This causes disk space exhaustion or inode exhaustion despite builds being removed from the UI.
Explanation
Build directories that are missing the build.xml file cannot be fully deleted by the log rotator.
When log rotation runs, the controller removes the build from its internal records and attempts to delete the directory from the filesystem.
If the build.xml file is missing or inaccessible during the deletion process, the filesystem cleanup fails while the logical deletion succeeds.
This leaves orphaned build directories on disk that are no longer visible in the UI.
Workaround
Manually identify and delete orphaned build directories that are missing the build.xml file.
-
Navigate to
$JENKINS_HOME/jobs/$JOB_NAME/builds -
Identify build directories that are missing
build.xml:for dir in */; do if [ ! -f "${dir}build.xml" ]; then echo "Missing build.xml in: ${dir}" fi done -
After verifying the builds are safe to delete, remove the directories
-
Reload configuration from disk using:
Investigate the root cause of why build.xml files are being removed to prevent this issue from recurring.
|