KBEA-00172 - Using the eMake Registry Virtualization Feature

This page describes the registry virtualization feature in emake and the agent.

In addition to the file system, builds on Windows sometimes depend on information stored in the system registry. Just as jobs running on a node need access to files on disk, sometimes a job will need to read or create registry information.

Registry information loosely groups into three types:

The first category pertains to the configuration of the hardware of the node. This information should not be interfered with in any way or the node may break. Luckily this information is kept in specific areas of the registry that can easily be avoided.

The second category pertains to the installation information of the various development tools installed on the nodes. The most obvious of these is Visual Studio. This information is spread throughout the registry. Much of it lives in HKEY_LOCAL_MACHINE\Software, but some of it also lives in HKEY_CLASSES_ROOT.

The third category pertains to keys owned by the software being built. Windows builds sometimes produce .dll files that contain a DllRegisterServer entry point. This function is responsible for creating various registry keys that describe any COM object interfaces provided by the .dll. This function is typically invoked during installation of the .dll on a new system. However, in some builds this registration step is performed in the middle of the build so the given COM interfaces can be used by later build steps. Most of these values live inside HKEY_CLASSES_ROOT.

The first and second categories pose no issues for distributed builds, because the installation of the os and tools on each node guarantees the appropriate values are present. The third category is more problematic. In the course of the build, dll registration happens on whatever node runs a job. The changes need to be mirrored to all of the nodes in the cluster in case the use of the COM object happens on a different node. Additionally, the changes need to be propagated back to the build machine in order for the user to see the expected side effects of the build.

Emake provides the ability to virtualize parts of the registry. The --emake-reg-roots command line option specifies a registry key that emake will use as the root of a tree in the registry that it will monitor. It watches for requests to read and/or change keys or values below the specified root key and ensures that the values are fetched from the build machine and propagated to all of the nodes in the build. If a job makes changes to the registry, those changes are sent back to emake in the usage report at the end of the job. Those changes are then applied to the registry on the build machine as well as any other nodes that request the values.

Unfortunately a simple top-down mirroring scheme is not sufficient because of the overlap between installed software and build-specific keys. Given that the build machine may have a different installation configuration than the nodes, not everything can be mirrored. Emake handles this by giving priority to values that are already present on the node. If a value is present on both the node and the emake machine, the node value is used. The idea is that if the value of a key specifies the location of installed files, the value on the emake machine may not work.

Additionally, some tools (notably Visual Studio 2003) look in several places in the registry for information about installed modules. Visual Studio decides which addins to load based on a list of known add ins as well as a list of installed modules on the system. Some of this data is inside the HKEY_CLASSES_ROOT hierarchy. This causes problems for emake because we want HKEY_CLASSES_ROOT to be the starting point for virtualization in order to catch changes made by dll registration. Unfortunately this picks up information about modules that are installed on the emake machine, but not the nodes, causing Visual Studio to crash.

To address this issue, emake provides the ability to remove subtrees from the virtualized area with negative path entries in the registry root list. Any subkeys of a negative path entry are not sent to the nodes by emake. This allows parts of the tree to be pruned when problems are discovered.