How to Overcome Template Clobbering Issues?

Issue

When updating the description in a Job created from template, the updated text is overwritten when the job’s configuration is changed (using the "Configure" option).
When updating the description in a Folder created from a Template, all Views, Groups, Controlled agents, Credentials and Properties are overwritten when the template is saved.

Environment

CloudBees Jenkins Enterprise
CloudBees Template plugin

Background

By design, all content of the instance (templatized job) is defined by the Transformer. The Templates plugin overrides the Configure link, which covers most of the ways in which config.xml might be modified.

However there are some cases not covered, such as the "Edit Description" link, RBAC Groups, Views etc. In those cases any customizations made outside the knowledge of Templates will be “clobbered” the next time the instance is reconfigured (attributes changed, or template changed).

Resolution

There is a solution to overcome the clobbering of existing values of a Template instances. It is possible to access the instance item from within the Groovy Transformer via the attribute ${instance}. Hence we can force the Template to read an instance attribute during the transformation.

The solution is quite straight-forward for simple/raw values like the description (see this article). The manipulation of Objects in the Template transformers requires a deeper understanding of Groovy and the Jenkins API.

Objects

While a description attribute points to a String value, other attributes like views and nectar.plugins.rbac.groups.Group are Objects and need to be serialized.

How to write a Groovy Template transformation so that changes to Credentials / Controlled nodes / Groups are persisted when the template is being updated?

It can be done in the Groovy Template transformation using the exact same technique as above. You need to capture the current Attributes and pass the XML representation through the transformation.

Access the instance attributes in the transformer
Serialize the object to XML

It is also important to check that the instance and the properties being manipulated are not null.

1) Access Instance Objects

Properties, Views and other components can be accessed via the attribute ${instance.item}, even if this is not a best practice.

For example, in a Folder I can access the Properties using ${instance.item.properties}. This becomes obvious when looking at the structure of the config.xml of a Folder :

<com.cloudbees.hudson.plugins.folder.Folder>
    <actions/>
    <description/>
    <properties/>
    <folderViews>
       <views/>
       <tabBar/>
    </folderViews>
    <primaryView/>
    <healthMetrics/>
    <icon/>
</com.cloudbees.hudson.plugins.folder.Folder>

Important Notice: You should not need to access any items using the attribute ${instance.item}, if that’s the case, our recommendation is that you review your template architecture so that you will not need to do it. Failing to do so, might cause the template engine to throw an stack overflow exception that could bring your instance down.

2) Serialize Objects

The Serialization of such Objects to XML can be done using either of these functions:

${xml(hudson.model.Items.XSTREAM.toXML(object))} for a single Object
${serialize(object)} for a single Object or ${serializeAll(object [])} for collections of object. More information about this helpers can be found in the documentation

Examples

1) Persistence of All Properties

For example, if you would like to keep all the properties of your folder instance (created from a Template), you can specify something like this in the Groovy Transformer:

<com.cloudbees.hudson.plugins.folder.Folder plugin="cloudbees-folder@5.1">
    ...
    <% if (instance != null
            && instance.item != null
            && instance.item.getProperties() != null) { %>
        ${xml(hudson.model.Items.XSTREAM.toXML(instance.item.getProperties()))}
    <% } else { %>
        <properties>
           ...
           //Define the properties by default on first creation
           ...
        </properties>
    <% } %>
    ...
</com.cloudbees.hudson.plugins.folder.Folder>

2) Persistence of Specific Property

You can also use this technique for a specific property. Following is an example of how to keep Controlled Agents only:

<com.cloudbees.hudson.plugins.folder.Folder plugin="cloudbees-folder@5.1">
    ...
    <properties>
        <% if (instance != null
                && instance.item != null)
                && instance.item.getProperties().get(com.cloudbees.jenkins.plugins.foldersplus.SecurityGrantsFolderProperty.class) != null) { %>
            //Rewrite the existing controlled agents
            ${xml(hudson.model.Items.XSTREAM.toXML(instance.item.getProperties().get(com.cloudbees.jenkins.plugins.foldersplus.SecurityGrantsFolderProperty.class)))}
        <% } else { %>

            //Otherwise just write default
            <com.cloudbees.jenkins.plugins.foldersplus.SecurityGrantsFolderProperty plugin="cloudbees-folders-plus@3.0">
                <securityGrants/>
            </com.cloudbees.jenkins.plugins.foldersplus.SecurityGrantsFolderProperty>
        <% } %>
    </properties>
    ...
</com.cloudbees.hudson.plugins.folder.Folder>

(Note: I know that a controlled agent corresponds to the class com.cloudbees.jenkins.plugins.foldersplus.SecurityGrantsFolderProperty.class by looking into the config.xml of a Folder with controlled agents)

3) Mixed Persistence / Attributes

Now if you want to mix behavior and use a combination of values specified in instances and Template attributes, it is also possible but it gets a bit more complicated as it requires some kind of merge mechanism of existing values and templatized values.

In the next example, I want to keep all Folder Credentials that have been specified in my Folder instances but I want to add mine as well. Again, looking into the config.xml of a sample folder, we can see that the property for folder credentials is com.cloudbees.hudson.plugins.folder.properties.FolderCredentialsProvider$FolderCredentialsProperty. Therefore, I need to capture all such property and pass it to the instance when it is created:

<com.cloudbees.hudson.plugins.folder.Folder plugin="cloudbees-folder@5.1">
    ...
    <properties>
        <% if (instance != null
            && instance.item != null
            && instance.item.getProperties().get(com.cloudbees.hudson.plugins.folder.properties.FolderCredentialsProvider$FolderCredentialsProperty.class) != null) { %>
            <com.cloudbees.hudson.plugins.folder.properties.FolderCredentialsProvider_-FolderCredentialsProperty>
                <domainCredentialsMap class="hudson.util.CopyOnWriteMap\$Hash">
                    <% if (instance.item.getProperties().get(com.cloudbees.hudson.plugins.folder.properties.FolderCredentialsProvider$FolderCredentialsProperty.class).domainCredentialsMap != null) { %>
                        <% instance.item.getProperties().get(com.cloudbees.hudson.plugins.folder.properties.FolderCredentialsProvider$FolderCredentialsProperty.class).domainCredentialsMap.each { %>
                    <entry>
                        ${xml(hudson.model.Items.XSTREAM.toXML(it.key))}
                        ${xml(hudson.model.Items.XSTREAM.toXML(it.value))}
                    </entry>
                        <% } %>
                    <% } %>

                    <entry>
                        //I can add my own credentials here (based on template attributes for example)
                    </entry>
                </domainCredentialsMap>
            </com.cloudbees.hudson.plugins.folder.properties.FolderCredentialsProvider_-FolderCredentialsProperty>
        <% } %>
        ...
        </properties>
    ...
</com.cloudbees.hudson.plugins.folder.Folder>

(Note: This FolderCredentialsProperty contains a Map and that is why we need to iterate on entries and write each key/value pair)

4) Persistence of All Views

<com.cloudbees.hudson.plugins.folder.Folder>
    <% if (instance != null
    && instance.item != null
    && instance.item.getFolderViews() != null) { %>
    <folderViews class="com.cloudbees.hudson.plugins.folder.views.DefaultFolderViewHolder">
        <% if (instance.item.getFolderViews().getViews() != null) { %>
        ${xml(hudson.model.Items.XSTREAM.toXML(instance.item.getFolderViews().getViews()))}
        <% } else { %>
        <views>
            <hudson.model.AllView>
                <owner class="com.cloudbees.hudson.plugins.folder.Folder" reference="../../../.."/>
                <name>All</name>
                <filterExecutors>false</filterExecutors>
                <filterQueue>false</filterQueue>
                <properties class="hudson.model.View\$PropertyList"/>
            </hudson.model.AllView>
        </views>
        <% } %>
        <tabBar class="hudson.views.DefaultViewsTabBar"/>
    </folderViews>
    <% } %>
</com.cloudbees.hudson.plugins.folder.Folder>

A common cause of template reconfiguration failure is script security. Ensure that there are no pending script approvals / signature due to the transformer changes.