Solution
Consider a "utility" project as one where some common collection of procedures are placed for shared use among many teams.
There are pros and cons with creating utility projects in CloudBees CD.
Generally speaking, a shared utility project is a good best-practice to implement. The pros in favour of creating utility projects are many, here are some key advantages:
1) ACLs can be defined to detemine which groups or users should have access to these utilities, although most utility projects will be setup with global access for all.
2) The procedures inside this project get consistently used by a variety of projects.
3) A utility project ensures that code re-use is being applied to your CD utilities to encourage overall efficiencies for adoption of CD. Scripting often suffers from "copy and modify" patterns which allow you to move fast, but ultimately results in bugs having to be fixed multiple times. Setting up a shared utility project can avoid that pattern.
4) When early adopters create utility procedures, subsequent teams on-boarding to CD can leverage these early investments to get their own designs implemented faster. This might include utilities that handle detailed IT nuances, or complicated logic.
5) Use of common utilities make CD practices across teams more similar, thus making developers on one team adaptable to working with other Devops teams in your organization.
6) A utility project can be similar to having an open-source initiative across your DevOps teams, or possibly for sharing in the wider CD user community.
Cons
1) Existing utilities can sometimes be in a state of being close to your needs, yet still requring modifications to work effectively. Coding such special patterns will require further time investment for updating and testing the modifications.
2) Modifying utility procedures shared across teams demands a wider set of testing be applied to your scripts. Essentially this means using a "Devops for your Devops" approach to how you make modifications within CD. So while it’s generally true that the sooner you put a process in place, the better, it’s also true that too much process for new initiatives can bog you down from achieving important early wins. Fnding the balance point for the best time to formalize processes will depend on your organization’s priorities, budget, and available resources.
3) Managing your utilities requires oversight and decision making. You don’t want to create 1 monster utility project holding 100s of procedures. So determining how to group such procedures into different projects, and which groups should have access to which utilities will require someone or some team to own that responsibility..
4) Within CD, project-to-project usage does come with a small performance impact due to a "context-switch" cost happening within the Database. If your code is rarely calling utiilty procedures from a different project, such costs are minimal and of little concern.
More details about the cost of context-switching
However, if you are referencing such utilities repeatedly within the same process, then the time costs could become noticeable. Some examples of when the time cost can be observed are:
a) Launching processes that rely on a utility sub-procedures in large blocks of parallel steps
b) Calling the utility sub-procedure in some form of loop
c) Calling utility sub-procedures which rely on a 1-step or exclusive resource (encouraging bottle-necking)
The above example patterns are rare, but have been observed by a few customers. These patterns can be mitigated as follows:
a) Splitting your large block of steps. Keeping parallel step blocks below a few-hundred steps per grouping by adding "separator" steps (non-parellel) in between your larger block (these can be no-op or do-nothing steps) can limit some of the overhead costs that arise in this scenario. eg; a 600-step parallel test block was split into 3 blocks of 200-parallel steps each, and the time needed to process this work came down considerably.
b) Copying the procedure to be present locally within the caller’s project to avoid the "project-context-switching" cost.
c) Repeated use of 1-step resources, or exclusive resources will create a bottle-neck, but the times involved are lessened if the context-switchching cost between projects is removed. So if the number of cross-project calls is high, then the resolution is to again look at copying the procedure into your local project instead of relying on the utility project.
It’s true that the copy technique stated in (b) and (c) undermines a number of of the pros behind using a utility project. But recognize that such cases are rare, so having to remove the occassional procedure for run-time benefit is more of an edge-case than a major deviation from the best-practice recommendation.