KBEA-00162 Integrating Coverity Analysis for C/C++ into an ElectricAccelerator Build

You can integrate Coverity Analysis for C/C++ into a makefile-based ElectricAccelerator® build on Linux or Windows if the build meets the prerequisites that follow.

Prerequisites

Your build must meet the following criteria before you integrate Coverity Analysis for C/C++ into an ElectricAccelerator build:

Gather required information

Before you begin the integration, make sure that you have:

Initial setup

Before you can integrate Coverity Analysis for C/C++ into an ElectricAccelerator build, there are a few set-up steps that you need to perform on the build machine and on each ElectricAccelerator agent.

Installing Coverity Analysis for C/C++

Before your ElectricAccelerator build can call Coverity Analysis, you must make Coverity Analysis available and accessible to the build machine and to each agent. You can install Coverity Analysis on each machine, or share a common installation between them. For example, you can use an ElectricAccelerator Electric File System, or a shared directory.

IMPORTANT:

Configuring your compiler

The compiler configuration must be available on the build machine as well as on each agent. If you have separate installations on each machine, the compiler must be configured on each machine. If you have a shared installation, you only need to run the configuration once. To configure you compiler, use the cov-configure command.

On Linux systems, if you use the GCC compiler, you can use the following example:

% cov-configure --compiler gcc

On Windows systems, if you use the cl.exe compiler, you can use the following example:

C:\> cov-configure --compiler cl

Note :
You can use the ElectricAccelerator utility named clusterexec to run this command on each of your agents. For example:

See the "Configuring compilers for Coverity Analysis" section (section 2.6) in the "Coverity Analysis 8.0 User and Administrator Guide" for more information about configuring your compiler.

Coverity integration files

ElectricAccelerator comes with the files that you need to integrate Coverity Analysis with an ElectricAccelerator build. These files are in the ElectricAccelerator install directory under /unsupported/coverity or may be obtained from Electric Cloud support. Copy them to the Coverity Analysis /bin directory. On Linux, the files are shell scripts. On Windows, the files are batch files.

Linux

To integrate with an ElectricAccelerator build on a Linux system, use the following shell scripts:

Before you can use the scripts, you must customize them with values from your site configuration. You only need to customize the script cov-ec-vars.sh because the other scripts get their customized values from it. Edit the cov-ec-vars.sh script using the following values:

Windows

To integrate with a Windows ElectricAccelerator build, use the following batch files:

Before you can use the batch files, you must customize them using the following values from your site configuration. You only need to customize the batch file cov-ec-vars.bat because the other batch files get their customized values from it. Edit the batch file cov-ec-vars.bat using the following values:

Modifying the build

After the initial setup is complete, you must modify the build so that:

To change your build, modify your makefiles.

Retaining existing functionality

When you modify your makefiles, it is important to control whether or not Coverity Analysis will run. To accomplish this, use the if functionality of make. The following example uses GNU make syntax:

ifeq ($(COVERITY),1)
     ... makefile contents for a Static Analysis build ...
else
     ... makefile contents for a non-Static Analysis build ...
endif

With this in your makefile, your existing functionality is kept intact, and you can perform a Coverity Analysis build with the emake COVERITY=1 command.

The syntax for a makefile that uses nmake is different than the syntax in the preceding example. See the "Differences using nmake makefiles" section in this document for details.

Injecting cov-translate calls into the build on Linux

To make sure that the cov-translate command is run alongside each compile, each native compile must be run through a wrapper batch file or script. To make sure that the batch file is called, the compiler macro, typically $(CC), needs to be overridden so that it calls the Coverity wrapper. For example, an original makefile line is:

CC=gcc
CXX=g++

On Linux systems, the modified makefile is:

ifeq ($(COVERITY), 1)
     CC=cov-ec-wrapper.sh gcc
     CXX=cov-ec-wrapper.sh g++
else
     CC=gcc
     CXX=g++
endif

Injecting cov-translate calls into the build on Windows with the Visual Studio add-in.

On Windows systems, the new version of Visual Studio plugin (4.2.5/5.0 or greater) lets you set an environment variable which enables you to add something to the front of the cl.exe compile lines in the generated makefiles. This is much easier than having to edit the makefiles by hand.

The environment variable ECADDIN_COVERITY_BATCH_FILE should be set to cov-ec-wrapper.bat.

The environment variable ECADDIN_COVERITY_DATABASE_LOCATION should be set to the location of the Coverity database set by coverity_agent_local-dir in cov-ec-vars.bat. (e.g. c:\cov-agent).

When using precompiled headers, the emit database for the PCH creator is copied and used by the PCH user. Copying the PCH database is necessary to ensure subsequent compiles can be parallelized. The Visual Studio add-in will create a maximum of ECADDIN_MAX_PDB_FILES databases.

When not using precompiled headers, a unique emit database is created for each compilation.

Add --emake-exclude-env=COMPUTERNAME to ensure teh agent hostname is used when creating the emit database.

Set ECADDIN_USE_INLINE_FILES=true if using a PATH containing Chinese characters.

Injecting cov-translate calls into the build on Windows by hand

On Windows systems, the modified makefile is:

ifeq ($(COVERITY), 1)
     CC=cov-ec-wrapper.bat gcc
     CXX=cov-ec-wrapper.bat g++
else
     CC=gcc
     CXX=g++
endif

Alternative to modifying the makefiles

Note : An alternative to modifying each makefile is to override $(CC) on the command line. The drawback to this method is that you must remember to override the $(CC) macro each time you want to perform a Coverity Analysis build. If you use the nmake command, you might need to modify your makefiles before you can override a macro from the command line. See the "Macros set on the command line" section in this document for details.

Gathering the build results

By injecting the cov-translate command calls into an ElectricAccelerator build, the cov-translate command creates Coverity Analysis emit directories on each of the ElectricAccelerator agents used. Before the analysis, you must gather and merge together all of the individual emits with the cov-ec-finalize-agents.sh script or cov-ec-finalize-agents.bat batch file; you must modify the build to call the script or batch file.

Note : The cov-ec-finalize-agents.sh script or cov-ec-finalize-agents.bat batch
file uses the Electric Cloud tools clusterdownload and clusterexec to gather and cleanup
the emits from each of the agents. Both of these tools require that the eRunner daemon is running on the machine. You can modify the cov-ec-finalize-agents.sh script or
cov-ec-finalize-agents.bat batch file to use other tools, such as the ssh or scp command, to perform these steps.

Before you call the cov-ec-finalize-agents.sh script or cov-ec-finalize-agents.bat batch file, it is required that:

Gathering the results inside of make

To modify your makefiles to call the cov-ec-finalize-agents.sh script or
cov-ec-finalize-agents.bat batch file, add a new target to your makefile.

On Linux systems:

# If this is a Static Analysis build:
ifeq ($(COVERITY), 1)

# Define the finalize target to gather all of the emits:
#pragma runlocal
cov-finalize:
      @cov-ec-finalize-agents.sh $(ECLOUD_BUILD_ID)
else

# Not a Static Analysis build; Define an empty target:
cov-finalize:
endif

On Windows systems:

# If this is a Static Analysis build:
ifeq ($(COVERITY), 1)

# Define the finalize target to gather all of the emits:
#pragma runlocal
cov-finalize:
     @cov-ec-finalize-agents.bat $(ECLOUD_BUILD_ID)
else

# Not a Static Analysis build; Define an empty target:
cov-finalize:
endif

Gathering the results outside of make

Rather than the build calling the cov-ec-finalize-agents.sh script or the cov-ec-finalize-agents.bat batch file, the script or batch file can be run after the build is complete. However, the script or batch file must be run as the same user that the build was run as.

Important : Until the cov-ec-finalize-agents.sh script or cov-ec-finalize-agents.bat batch file is called, all of the temporary emit directories will remain on the agents.

Performing a build

After your build is modified, it is time to try a Coverity Analysis build with ElectricAccelerator. This is as easy as running an emake build.

The actual command you run might vary slightly, so there are a few important points regarding the sample commands that follow:

The following examples perform a Coverity Analysis build using ElectricAccelerator.

If you previously started your build with:

You could now run:

The resulting emit directory is now in the current directory and named coverity, or the name of the directory that you customized to be, in the cov-ec-vars.sh script or cov-ec-vars.bat batch file.

Using a different intermediate directory name

The scripts or batch files use a default emit directory named or the name of the directory that you customized to be, in the cov-ec-vars.sh script or cov-ec-vars.bat batch file. If you want a different emit directory name, you can change it on the command line by setting the COVERITY_INTERMEDIATE macro:

% emake COVERITY=1 COVERITY_INTERMEDIATE=coverity.myname all cov-finalize

On the preceding command line, the final emit directory is named coverity.myname.

Running the analysis

After you have your emit directory, you can run the analysis as usual.

Validating the results

Because the wrapper script or batch file was injected by overriding the compiler $(CC) makefile macro, it is possible that some compiles were missed. For example, a Makefile might have directly called gcc.
Therefore, it is a good idea to compare the results from a serial cov-build to those that were obtained from an ElectricAccelerator build.

Linux:

# build with cov-build
% cov-build --dir coverity.covbuild make all

# build with ElectricAccelerator
% emake COVERITY=1 COVERITY_INTERMEDIATE=coverity.emake all cov-finalize

Windows:

# build with cov-build
C:\> cov-build --dir coverity.covbuild make all

# build with ElectricAccelerator
C:\> emake COVERITY=1 COVERITY_INTERMEDIATE=coverity.emake all cov-finalize

Validation 1: Number of translation units

You can check if any translation units are missing, by using the cov-manage-emit command (and a few shell commands if you are on Linux). You can compare the emit that is generated by the ElectricAccelerator build with the emit that is generated by the cov-build command.

Linux:

# Generate the list of translation units for the cov-build emit:
% cov-manage-emit --dir coverity.covbuild list \
| grep -v "Translation unit" | cut -d ">" -f 2 | sort > covbuild.lst

# Generate the list of translation units for the ElectricAccelerator emit:
% cov-manage-emit --dir coverity.emake list \
| grep -v "Translation unit" | cut -d ">" -f 2 | sort > emake.lst

# Compare the results:
% diff covbuild.lst emake.lst

Windows:

# Generate the list of translation units for the cov-build emit:
C:\> cov-manage-emit --dir coverity.covbuild list

# Generate the list of translation units for the ElectricAccelerator emit:
C:\> cov-manage-emit --dir coverity.emake list

If there are any differences, find the location in your makefiles where that source file is compiled. There are a couple of likely causes why a source file might be missed:

After you have modified the build, perform a new ElectricAccelerator build and repeat the validation step.

Validation 2: Analysis results

Although you can perform the previous validation quickly, a more thorough validation is to compare the analysis results. To do this, run an analysis on each emit and compare the results.

Linux:

# analyze the cov-build emit
% cov-analyze --dir coverity.covbuild

# analyze the ElectricAccelerator emit
% cov-analyze --dir coverity.emake

Windows:

# analyze the cov-build emit
C:\> cov-analyze --dir coverity.covbuild

# analyze the ElectricAccelerator emit
C:\> cov-analyze --dir coverity.emake

Both runs should have similar results.

Differences using nmake makefiles

There are a few important differences between GNU make and nmake that relate to the instructions previously described.

Syntax for if

One of the most noticeable differences between GNU make and nmake is the syntax used for the if functionality. An if in nmake is similar to:

!if "$(COVERITY)" == "1"
    ... makefile contents for a Static Analysis build ...
!else
    ... makefile contents for a non-Static Analysis build ...
!endif

The makefile fragment required for this integration is similar to:

# if this is a Static Analysis build
!if "$(COVERITY)" == "1"

# Define the finalize target to gather all of the emits:
#pragma runlocal
cov-finalize:
     @cov-ec-finalize-agents.bat $(ECLOUD_BUILD_ID)
!else
# Not a Static Analysis build, define an empty target
cov-finalize:
!endif

Macros set on the command line

The GNU make command allows you to set on the command line a macro that is passed down to all of the sub-makes, and that overrides any value that might be specified within the sub-makefiles. For example:

C:\> make NUMBER=1

In the preceding line, the NUMBER macro is set to 1 in all of the sub-makes regardless of what it is set to in the sub-makefiles. This behavior makes it easy to override the compiler macro, such as CC, in all sub-makefiles by specifying it on the command line.

Unfortunately, nmake has different behavior; if a macro is redefined within a sub-makefile the recursive make does not override the value by default. Therefore, if the compiler macro is defined in all of your makefiles, you cannot override the compiler macro on the command line. However, you can modify the nmake makefiles to have it behave similarly to the GNU make command.

There are two procedures that you can use to modify your nmake makefiles to behave similarly to GNU make:

To explicitly set the macro when recursing calling make

To explicitly pass down the MAKEFLAGS macro