MSBuild explained: Understanding Microsoft’s build platform

Visual Studio is a lot more than an editor and debugger; it’s the host for Microsoft’s cross-platform build tools. MSBuild is one of those forgotten heroes, a tool that’s at the heart of all your development processes, but one that’s hidden behind the façade of your development environments.

MSBuild is where your code, libraries, and SDKs are marshaled and delivered to .NET and other compilers. If you’re coding in C#, C++, or any other Microsoft language, and working in Visual Studio or Visual Studio Code—or using the standalone build tools with any other code editor or IDE, or as part of a GitHub build pipeline—you will be using MSBuild without even thinking about it. Even tools like the Windows App Development CLI depend on MSBuild.

In short, MSBuild is easy to overlook. But that’s unfair to the team that’s been delivering Microsoft’s build tooling consistently for 18 different major versions (and many minor releases in between). It’s not many groups inside Microsoft that can say they’ve taken development from proprietary to open, from single platform to cross platform, and extended it to multiple processor architectures along the way. I’m as guilty as any of us here, tending to focus on the languages, the editors, and the compilers but not on the plumbing that holds it all together and delivers ready-to-run applications from our code.

MSBuild — the Microsoft Build Engine

If you’re coming from a Unix background, then you’re familiar with tools like make and cmake that use scripts to build binaries from source code. In the Microsoft world, although you can still use tools like make with gcc and other compilers, you’re more likely using MSBuild, as the Microsoft Build Engine is usually called.

MSBuild is designed to use the instructions in a project file, produced by Visual Studio or another development tool, to orchestrate the processes that build source code into ready-to-run binaries. You can do this inside your IDE, or alternatively, download the command line build tools. These are covered by the same licenses as Visual Studio, though you can use them freely when working with open-source software. (As with any application that works with both proprietary and open-source software, it’s important to read the license to ensure you’re compliant.)

Microsoft develops MSBuild in the open, on GitHub, as part of the .NET project, though it is designed to work with much more than .NET. Naturally, you can compile MSBuild yourself, including on any Unix platform that supports the core .NET runtimes and libraries. And you can make your own pull requests to contribute new features or fix bugs. The available code includes helper classes that can be used to add your own tasks or logging to a build pipeline, which can be useful if you’re developing CI/CD tooling and you want to add support for MSBuild.

Getting started with MSBuild

You can install MSBuild as part of the Visual Studio Build Tools, or from the .NET CLI, where the dotnet build command launches the .NET version of the tooling. It’s a relatively small download, which makes it suitable for launching by scripts or building into your own pipeline as part of a custom build system.

Custom build systems do require some work but can take advantage of advanced capabilities such as compiling parts of large applications in parallel, as well as adding your own pre-processing and post-processing steps or saving outputs in a custom location. This approach is how tools like Azure Pipelines and GitHub Actions use MSBuild, taking code from one repository and delivering test builds and releases to specific locations.

If you’ve got source code and project files, you can start to use MSBuild from the command line. As the .NET CLI has various aliases for MSBuild features, the same command-line switches can be used with tools like dotnet build and dotnet publish (though not with dotnet run).

Using MSBuild command-line settings

MSBuild’s command-line switches allow you to tune operations. They also allow you to generate useful outputs that can help you understand how a build has run or what components make up an application. This latter option allows you to build a dependency graph for your code, giving you a starting point for building a software bill of materials (SBOM) for any open-source libraries or other components you’re using in your application. With more and more regulatory bodies requiring SBOMs to help ensure security, this feature will undoubtedly become increasingly useful.

Other features include the ability to tune MSBuild for the hardware you’re using, for example by controlling the number of processes used. By default MSBuild works as a single process, but by using the right switch you can either choose a specific number of processes or force MSBuild to use all the available resources. Locking it down and controlling its priority can help manage costs when using cloud-hosted systems for builds, by managing the resources for virtual machines used for GitHub runners and the like.

Inside MSBuild project files

Microsoft has been transitioning its project file syntax to XML, with a solution file that references multiple projects. Project files include all the input and target information MSBuild needs to build the project from source. This process includes defining the build environment and building an in-memory structure from the various project files.

This helps define the target build order, which processes dependencies and executes the build appropriately, so that code with dependencies on another project in the solution will wait until those dependencies are built. Getting the order right is key, which is why MSBuild evaluates the build structure and puts together a stack before doing anything else. The move to using a graph to manage this dependency map helps speed up builds, scheduling the various tasks and orchestrating them across parallel nodes running on separate cores.

Project files will hold lists of dependencies, as well as language-specific requirements. The latter allow MSBuild to manage the processes necessary for both C# and C++ builds, where one is building byte code to run on the .NET runtime and the other is compiling to native binaries.

Because the files are XML, they’re available for customization. However, this needs to be done carefully. Microsoft provides instructions for building your own project files from scratch, a process that can be useful if you’re porting code from one environment to another and you don’t have access to the original project files. You can use this approach if you’re working without an SDK and you don’t want the overhead that comes with a platform like .NET.

Using MSBuild to get the most from your code

More common is using MSBuild to add custom tasks to a build process. One possibility is to add code generation features to a build, taking text files and using templates to turn them into code. This approach can be useful when you need to generate code to work with specific input data, dynamically modifying a template based on data that needs to be hard-coded into, say, a control application for IoT hardware.

Understanding (and customizing) MSBuild becomes critical when you’re building CI/CD pipelines. Here you need to run the build engine outside Visual Studio, embedding it in containers or virtual machines that can be triggered on demand, installing it alongside the necessary compilers and using command line tools to tune performance. One option here is to deliver resource limits in a configuration file so that the same build script can run in different environments without causing issues.

By scripting the build service, you can then target and prioritize different build types appropriately, putting artifacts in one location for automated tests and another for acceptance testing, automating deployment for staging, and delivering to production in the required format. Test builds might be configured to use less CPU than production, as they’re likely to be run more often and costs will need more control.

Hitting F5 in Visual Studio to build an executable hides what turns out to be a flexible and powerful too that can help make large-scale software development both faster and cheaper—if you’re prepared to get down into the weeds and tweak the default build settings. For large projects using cloud-based CI/CD tooling, digging into MSBuild may well be worth the effort.