Thursday, May 7, 2020

Cross-Platform Build Events in .NET Core using MSBuild

In the last article, we looked at post-build events and .NET Core -- specifically how the '$(SolutionDir)' and other macros based on the solution folder do not work when we do project-based builds. Instead, we should use folders relative to the project.

    xcopy "$(ProjectDir)AdditionalFiles\*.*" "$(TargetDir)" /Y

This works fine for Windows-based projects (like the WPF project used in the previous article), but it fails when we move to a cross-platform environment. "xcopy" doesn't exist on macOS or Linux.

Fortunately, we can dig into MSBuild a little deeper and create tasks that copy files in a platform-agnostic manner. Here's a preview of the build task that does the same things as above:

Let's dig into some code.

MSBuild Tasks
MSBuild (the build tool in the .NET world) has a number of Tasks that we can use as part of the build process. These go right into our project files.

In fact, when we add a post-build event through the Visual Studio project settings editor, we end up with an MSBuild task in the project file. For example, our "xcopy" command that we saw above:

Ends up in the project file looking like this:

  <Target Name="PostBuild" AfterTargets="PostBuildEvent">
    <Exec Command="xcopy &quot;$(SolutionDir)AdditionalFiles\*.*&quot; &quot;$(TargetDir)&quot; /Y" />

This uses the "Exec" task to run a command at the command prompt.

So we're already using MSBuild tasks. Fortunately for us, there are other tasks to choose from besides "Exec".

The Copy Task

On the Microsoft Docs site, we can see the tasks that are built in: MSBuild task reference. One of those is the Copy task (docs reference for Copy task).

For today's examples, I'll be showing the cross-platform examples from my C# Interfaces talk: There are 2 specific project files that I'll link to as we go.

Let's look at our new copy task and see what's going on (this code is available in the "PeopleViewer.csproj" file at this link: PeopleViewer.csproj -- specifically lines 17-26):

First, let's look at the "Target". This is a grouping that tells when this task is going to run. Notice that "AfterTargets" parameter is set to "Build". So this will run after the build. If the build fails, then this will not run.

Inside, we have the "Copy" task itself. We are using 3 parameters: (1) SourceFiles, (2) DestinationFolder, and (3) SkipUnchangedFiles.

"SourceFiles" is a required parameter. This is an array that specifies the files that we want to copy. Since this is an array, we create our own element called "DataFiles". The "Include" parameter has a list of the files that we want to include. We can still use the build macros, so "$(ProjectDir)" works just fine here. In this case, we only have one item in our array (our one item happens to include a wildcard "*.*").

The value for the "SourceFiles" parameter is the item created in the ItemGroup above. We reference it with using "@(DataFiles)".

As an alternative to using wildcards, we can specify individual files separated by semi-colons. An example of this is shown on the Microsoft Docs site: Example (with multiple files) -- screenshot below (but be sure to follow the link):

"DestinationFolder" is a kind-of-required parameter. This has the destination for the files to be copied. If the folder does not exist, then it is created automatically. Our value for this folder is the "$(TargetDir)" that we had in the "xcopy" version.

"DestinationFiles" is a kind-of-required parameter. You'll notice that we do not have a "DestinationFiles" parameter for our copy task. This is because we can use either "DestinationFolder" or "DestinationFiles", but not both (and we need at to have at least one of them). The "DestinationFiles" parameter is also an array, and it expects a one-to-one mapping of items from the "SourceFiles" array.

"SkipUnchangedFiles" is an optional parameter. This will not copy files that are unchanged between the source and the destination. It determines this based on the file size and the timestamp.

There are a number of other interesting parameters, so you should take a look at the docs. For example, "UseHardlinksIfPossible" will create links to the files rather than copying them.

Recursive Copying
Many times we need to copy entire folder structures (including subfolders). The good news is that we can do that with the copy task as well. For more information, take a look at the example on the Microsoft Docs site: Example (recursive copy) -- screenshot below (but be sure to follow the link):

A Lot of Effort?
This seems like a lot of effort compared to our previous post-build event. With the post-build event, we can use the Visual Studio project editor to build our commands and insert macros. Here, we need to edit our project files by hand. But it is totally worth it. This command works on Windows, macOS, and Linux -- and yes, I have tried it in all of those environments.

Before I found the MSBuild tasks, I was thinking about how I would get around this. Could I put in different commands (like using "xcopy" for Windows and "cp" for macOS), and then add some sort of conditional? That is technically possible, but it is a lot more brittle when all I need to do is copy some files.

I definitely prefer to use the build task.

OS-Specific Tasks
I ran into an issue where I needed to copy different files based on the OS, so I ended up digging into conditional MSBuild tasks.

For my specific issue, I have a library that uses SQLite to access a local database file. I need to do runtime binding to this library, so I need all of the required files in a shared location. The problem is that SQLite has different runtime files for each environment. For Windows, it uses "e_sqlite3.dll", for macOS it uses "libe_sqlite3.dylib", and for Linux it uses "". I needed to figure out how to get the right file to the output folder.

The MSBuild tasks for this are in the "PersonReader.SQL.csproj" file which is on GitHub: PersonReader.SQL.csproj.

Windows-Specific Task
To get started with conditional tasks, let's look at the task that copies over the Windows-specific SQLite file (this is on lines 31-41 of the PersonReader.SQL.csproj file):

Overall, this task looks similar to what we saw above. One difference is that the Target has a "Condition" parameter.

Condition=" '$(OS)' == 'Windows_NT' "
"Condition" lets us put in a restriction on when this task will run. For this example, we can look at the '$(OS)' value to see if we are running under Windows. If so, then it runs the copy task.

For more details on the types of conditions that we can use here, check out the MSBuild conditions article on Microsoft Docs.

macOS-Specific Task
It seems like doing the same thing for macOS should be straight-forward: just change the condition to use "macOS". Unfortunately, that does not work here. The '$(OS)' values are not specific enough for what we need. If we run on macOS, we get the value of "Unix" (since macOS is a Unix-based system). The problem is that for Linux, the value is also "Unix". So the "$(OS)" value is not helpful.

Fortunately, there is another option. Here is the code (from lines 43-53 of the same PersonReader.SQL.csproj file):

This condition is a bit harder to understand because things are fully-qualified. So before we look at this condition, let's look at this another way: as C# code.

Checking the OS in C#
If we wanted to check for the OS in C#, we can use the "RuntimeTypeInformation" class that is part of System.Runtime.InteropServices. Here's the code:

    if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX)) {}

The "IsOSPlatform" method takes an enum value (OSPlatform) as a parameter. So we can use this method to check whether something is running under Windows, macOS (i.e. OSX), or Linux.

For those of you who are new to macOS, the operating system was renamed from "OSX" to "macOS" a few years back. Since it is generally a bad idea to change enums that are already in use, we'll see "OSX" in various places when referencing the Apple OS.

Back to the MSBuild Condition
The condition that we have in the MSBuild task above uses this same "IsOSPlatform" method, but the syntax is a little different because it is in the MSBuild format and also because the names are fully-qualified.

If we pull out the fully-qualified parts, we get something a bit more readable:

The syntax is still different, but it's easier to see that we are calling the "IsOSPlatform" method on the "RuntimeInformation" class, and that we are passing in the equivalent of "OSPlatform.OSX" as a parameter.

Now this will not work unless it is fully-qualified, so we need to use the long version in our MSBuild task.

Linux-Specific Build Task
The task for Linux is similar to the macOS task. If you're interested, you can look at the code in the PersonReader.SQL.csproj file (lines 55-65).

Wrap Up

Now we have a post-build copy command that works cross-platform. And we also have some conditional commands that run based on what operating system we are building on. This gives us quite a bit of flexibility in building truly cross-platform applications.

MSBuild is quite an extensive system, and there are tons of options. The MSBuild task reference has a list of built-in tasks. In addition, you can create custom build tasks if needed (although that is something I am not likely to do myself).

One of the most interesting things to me about .NET Core is the cross-platform capabilities. As I move my applications forward, I think about how I can make them easier to run cross-platform. This gives a bigger audience for the applications and also provides different options for deployment.

Copying files in a way that is friendly to multiple operating systems is just one of the things we need to think about when creating cross-platform applications. Be sure to check back; I'll continue to document things I come across as I move projects over.

Happy Coding!

Tuesday, May 5, 2020

Post-Build Events and .NET Core

I have used post-build events in Visual Studio for a long time, primarily to copy files from one location to another. But due to changes in how we work with .NET Core, I have changed the way I write build events.

Short version:
The '$(SolutionDir)' macro doesn't work well from the command line. Replace references to the Solution with references relative to the current Project.
If you'd like an overview on using build events (and the macros that are available), take a look at this article: Using Build Events in Visual Studio to Make Life Easier.

Using a Post-Build Event
Let's look at an example of how I use post-build events. Here's the folder structure for a solution:

This solution has 4 projects: (1) "Common" is a shared library with cross-cutting classes defined; (2) "PeopleViewer" is a desktop application, (3) "PeopleViewer.Presentation" is a view-model class library, and (4) "PersonReader.CSV" is a data reader.

The top folder, "AdditionalFiles", contains a text file with the data. The "PeopleViewer" needs access to this file in order for the application to work. So, in this scenario, I want to copy the contents of the "AdditionalFiles" folder into the output folder for the "PeopleViewer" project.

Here is a post-build event that does this:

    xcopy "$(SolutionDir)AdditionalFiles\*.*" "$(TargetDir)" /Y

The "$(SolutionDir)" macro expands into the fully qualified path for the solution. This includes the trailing "\". The "$(TargetDir)" macro expands to the fully-qualified location where the .exe file ends up -- something like "[fullpath]\PeopleViewer\bin\Debug\" for .NET Framework and "[fullpath]\PeopleViewer\bin\Debug\netcoreapp3.1" for .NET Core.

This works fine when we build the application from Visual Studio:

    4>1 File(s) copied

The build succeeds and the file is copied.

The problem arises when we leave Visual Studio.

.NET Core and the Lack of Solutions
In the .NET Core world, the concept of the "Solution" is less important. We often open folders (not solutions) in Visual Studio Code. And when we use the command-line tools, we build projects (not solutions). The ".sln" file is not important in this scenario, and it is generally not used at all.

This works because the projects have all of the information that they need to build, including dependencies on other projects and on NuGet packages.

So to build the "PeopleViewer" project, I can open the command line to the project location and type "dotnet build".

But, the post-build event fails:

  0 File(s) copied
  File not found - *.*
C:\Development\Sessions\DI\Scratch\PeopleViewer\PeopleViewer.csproj(15,5): error MSB3073: The command "xcopy "*Undefined*AdditionalFiles\*.*" "C:\Development\Sessions\DI\Scratch\PeopleViewer\bin\Debug\netcoreapp3.0\" /Y" exited with code 4.

The message tells us that the copy step failed. And if we look closer at the message, we can see why:

    "xcopy "*Undefined*AdditionalFiles\*.*"

When trying to expand the macros, the '$(SolutionDir)' is undefined. There is no concept of a solution at this level, just the project. So the build fails.

Relative to the Project
The fix for this is to not use any of the "Solution" macros in the post-build events. Instead, set things relative to the current project.

Here's an updated post-build event:

    xcopy "$(ProjectDir)..\AdditionalFiles\*.*" "$(TargetDir)" /Y

To get to the "AdditionalFiles" folder, we start with the current project folder, then go up a level (with the "..").

When we build this from Visual Studio, it still works. But more importantly, it works when we build from the command line:

  1 File(s) copied


Post-build events are really useful, and I'm often using them to copy data files or late-bound assemblies to the output folder of my projects. But I've changed how I use them.
Instead of using post-build events relative to the Solution, I use events relative to the Project.
It's always exciting to come up with a fix to an issue. But this isn't completely solved.

Cross-Platform Concerns
One of the really awesome things about .NET Core is that it is cross platform. I can create applications that will run on Windows, macOS, and Linux.

But "xcopy" is a Windows command. How do we copy files for other OSes?

For that we need to dig into MSBuild. That will wait for the next article, but here's a preview:

This block in the project file will copy the files from the "AdditionalFiles" folder to the output folder. And it works on macOS and Linux.

We'll explore that in the next article.

Update: That article is now available: Cross-Platform Build Events in .NET Core using MSBuild.

Happy Coding!