Pipeline - Files manipulation

Article ID:230922508
2 minute readKnowledge base

Issue

  • I am trying to port some legacy Jenkins build jobs that used Linux shell scripts (to Jenkins Workflow scripts) to download and stage/deploy to multiple server nodes. I need to be able to save and manipulate files to the Workflow workspace (or to somewhere under /tmp) directory structure.

  • When we use groovy library classes (AntBuilder, File) it is working fine on Client-controller but it is not working on Agent machines.

Environment

Resolution

The operations with File class are run on the controller, so only works if the build is run on controller. In this example, I create a file and check if I can access it on a node using the method exists. The file can’t be found because the new File(file) code is executed on the controller. To double-check this, I did the same for the folder Users, which exists on my controller but not in the node.

stage 'file move wrong way'

//it only works on master
node('agent') {

    def ws = pwd()
    def context  = ws + "/testArtifact"
    def file = ws + '/file'
    sh 'touch ' + file
    sh 'ls ' + ws

    echo 'File on node : ' + new File(file).exists()
    echo 'Users : ' + new File('/Users').exists()

    sh 'mv ' + file + ' ' + context
    sh 'ls ' + ws
}

To execute file manipulation command we recommend to use native commands.

This is a simple example of operations in shell

stage 'Init'
node {
    stage 'Environment'
    sh 'export'

    stage 'Create file'
    sh 'touch test.txt'

    stage 'List relative Directory'
    sh 'ls ../../../../../'

    stage 'List / dir'
    sh 'ls /'

    stage 'List Home of user'
    sh 'ls $HOME'

    stage 'List Home of agent'
    sh 'ls $JENKINS_HOME'

    stage 'List temp'
    sh 'ls $TMPDIR'

    stage 'List job dir'
    sh '''
        echo $(pwd)
        ls .
        ls $(pwd)
        '''

    stage 'download file'
    def out='$(pwd)/download/maven.tgz'
    sh 'mkdir -p ./download'
    sh 'curl -L https://ftp.cixug.es/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz -o ' + out

    stage 'move/rename'
    def newName = 'mvn.tgz'
    sh 'mkdir -p $(pwd)/other'
    sh 'mv ' + out + ' ' + newName
    sh 'cp ' + newName + ' ' + out
}

Reference