Gathering Test Code Coverage for Unreal Engine Games the Easy Way Using OpenCppCoverage

Being relatively new to Unreal Engine, I was surprised to discover that there was no way to measure test code coverage out-of-the-box. Google search was only revealing very infrastructure-intensive solutions – mainly setting up a Cadillac solution using Google Test, or customizing Unreal Build Tool. I thought that there had to be an easier way.

I reasoned to myself, “The Unreal Editor is just like any other app on the planet. I already attach a debugger to it all of the time. I should be able to use standard coverage tools that act like debuggers against it.” So I looked at what was available for measuring code coverage on C++ applications, and quickly came across OpenCppCoverage.

It turned out to be really easy. Much easier than the other methods I found available. In its current state, it is not 100% automated to run tests, but there are minimal steps involved and I understand that there is a way to fully automate this.

My work on this actually came up in a job interview and seemed to be quite helpful, so I’m writing it up here in hopes that it can help others. I did this on Windows 10.

Download and Setup OpenCppCoverage

Simple enough. Get it from the official source here. https://github.com/OpenCppCoverage/OpenCppCoverage/releases

Add OpenCppCoverage to your PATH

Add the path to which you installed OpenCppCoverage to your PATH environment variable. See this SO question if you are unsure how to do this.

Figure out what Modules and Sources to Test

You DEFINITELY want to filter modules down to the ones you are attempting to measure. Unreal loads a LOT of modules. You should just be able to use the names of your modules from Unreal.

Sources will likely be the same as modules unless you have some kind of exotic setup.

We’ll set some default exclusions and the like in the next step to squelch the noise like generated files from measurement.

Build Your OpenCppCoverage Batch File

If you want to use Powershell, thisshell or thatshell or whatevershell, the steps are similar.

In this batch file, you will want to navigate to your Unreal project directory, and then launch OpenCppCoverage, instructing it to in turn open Unreal Editor with your project.

The “navigate to your Unreal project directory” step will vary based on your project folder setup. This is what works for me as a template batch file:

pushd %~dp0
pushd ..\unreal\{project_name}

OpenCppCoverage --modules {module} --sources {source} --excluded_sources .gen. --excluded_sources \Tests\ --export_type html:..\..\codecoverage\Report -- "C:\Program Files\Epic Games\UE_5.1\Engine\Binaries\Win64\UnrealEditor.exe" "C:/{Path_to_your_Unreal_project}/{project_name}.uproject"

popd
popd

Repeat the --modules parameter once for each module, and the --sources parameter once for each file.

You’ll definitely want to keep --excluded_sources .gen.

Be sure to adjust --excluded_sources \Tests\ to the correct subpath to exclude your test code itself from coverage measurement.

Also adjust the path to where you want the output. I’m using html export but you can use any OpenCppCoverage export type.

Of course, adjust the path to your UE installation if necessary (or be less lazy than me and use an environment variable).

Refer to the full reference of OpenCppCoverage command line arguments for more information

Configure Your Project for Code Coverage Measurement

Compiler optimizations and inlining can get in the way of code coverage measurement, so it is highly recommended that you turn these things off before you measure code coverage (and remember to turn them back on when done!)

Add the following lines to each module’s {module}.build.cs file:

    OptimizeCode = CodeOptimization.Never;
    bUseInlining = false;

Again, remember to comment these back out when done! (Or figure out how to make multiple build configurations – which is beyond the scope of this post)

Aside – Controlling Inlining

Note that bUseInlining = false will not necessarily disable all instances of inlining – specifically, if you are in the habit of putting your function definitions inside your .h file and not separate from your declaration, your code will have an annoying tendency to be inlined when you don’t want it to be for code coverage purposes. Sometimes OpenCppCoverage can pick up the coverage anyway. Sometimes it can’t. And sometimes can change depending on the phase of the binary moon (the compiler’s mood and whether it decides it wants to inline something based on other proximate influences within the compilation unit).

To more fully control when something is inlined, I resorted to the “.inl file” pattern, where all functions that may be inlined are defined in a separate .inl file, which is conditionally included in either your .h file (when inlining is supposed to be on) or your .cpp file (when inlining is supposed to be off), usually depending on a #define value. Further elaboration is beyond the scope of this post, though one place this pattern can be referenced in the KismetMathLibrary.h/.cpp/.inl files in the Unreal Engine source code.

It can feel a bit kludgy, but honestly I found the pattern cleans things up quite a bit, with all of the forwarders and trivial functions sectioned off in their own file, leaving the .h file free to focus on the interface, and leaving the .cpp file free to focus on the important code details. (C++’s forced separation of an entity across files is unfortunate IMO, but when life gives you lemons, make lemonade.)

Run Your Batch File

That’s it. Nothing else special should be required. We’re ready to gather coverage on tests.

If your batch file is in order, you should be able to double-click it, opening OpenCppCoverage, which in turn will launch your project in the Unreal Editor.

Run Your Test Payload

Navigate to the Test Automation window as usual, and run your tests.

It’s worth noting that if you have a test that fails at this stage, you’ll need to shut down Unreal Engine and relaunch before attaching a debugger. This is because OpenCppCoverage attaches itself to Unreal Engine as a debugger (in order to do its job of measuring code coverage), and Windows does not allow two debuggers to attach to the same process (probably for the best).

Shut Down Unreal

Once your tests complete, you need to shut down Unreal. This will cause OpenCppCoverage to wrap up proceedings and generate the report.

View Your Results

Tweaking & Next Steps

You’ll likely want to customize things, exclude some files, etc. But this is all elementary now that you have the minimum viable system working. Remember the full reference of OpenCppCoverage command line arguments.

If you are just working on a small project independently, this is likely “good enough right there™”. But for a larger project with a whole team, you’ll want to make paths system-independent in the batch file, automate the launch of the tests, and so forth. But we’ve met our objective here, which is to be able to measure code coverage in an Unreal project without a ton of infrastructure setup, custom Unreal Build Tool builds, or cloud dependencies.

I’ve tried to plant some seeds regarding the direction this could go; it’s up to you to run with it from here.

Conclusion

Thanks for reading my mini-instructional on how to set up Code Coverage Measurement easily in an Unreal project. It was definitely a lot easier than other sources would suggest! Let me know if somehow I’ve missed a step or otherwise flubbed in my recounting of how I set this up.

My motivation for setting this up was to help thoroughly test the first Unreal Plugin that I will be announcing and releasing on Unreal Marketplace in just a couple of days plus however long it takes to get the plugin approved. I’ll give just one hint as to what it will be. Actually, I’ll give 10³⁰⁹ or more hints 🤓.

I’m currently looking for work, so if you like what you see here and elsewhere on my site, and are a hiring decision maker or have influence over a hiring decision maker and want to put me to good use, please reach out.

Also, be sure to check out my Games, especially Prepare For Warp.