Integrating NUnit into Visual Studio Test Explorer

So, I've been inspired to explore NUnit a bit more as a testing framework. In a previous article, we saw how NUnit allows us to easily parameterize our tests. But there are a couple of things that have kept me leaning toward MSTest, primarily the integration with Visual Studio.

It turns out that NUnit *can* integrate with the Visual Studio Test Explorer. It's not quite as complete as MSTest, but it's definitely a lot better than using the stand-alone test runner.

The reason I'm getting back into this is because there are some compelling features of NUnit that I want to start using regularly. In an upcoming article, I'll talk about testing for exceptions, which is much better in NUnit than the current version of MSTest. Check for other testing articles here: Coding Practice with Conway's Game of Life.

[Update 11/23/2015: The steps shown in this article are temporarily broken due to a version mismatch between the NUnit framework and the NUnit test adapter. This will be fixed once the 3.0 version of the test adapter is released. In the meantime, you can check out this article for a workaround: Fixing an NUnit Version Mismatch.]

[Update 3/4/2016: A demonstration of installing NUnit in Visual Studio 2015 (including dealing with the current mismatch) is available by watching a few minutes of this TDD video: TDD Basics @ 3:30.]

[Update 04/20/2016: The 3.0 version of the Test Adapter has just been released. When using NUnit 3.0, be sure to use the "NUnit3TestAdapater" package from NuGet. When using NUnit 2.0, use the "NUnitTestAdapter" package from NuGet. More information here: Integrating NUnit into Visual Studio - Update for NUnit 3]

Integrating NUnit into Test Explorer
One of the things that I like about MSTest is the integration with Visual Studio. When I initially looked at NUnit, I was using the separate test runner application.

This shows all of our parameterized tests and the results. (As a reminder of how to install the NUnit test runner, see the prior article: Parameterized Tests with NUnit.)

We got the test runner from NuGet. But it turns out that there is another NuGet package that we can use to integrate NUnit with the Visual Studio Test Explorer.

For this, we add a package to the test project (you'll want to add this to each of your test projects that use NUnit):

The "NUnit Test Adapter for VS2012, VS2013 and VS2015" will add the Test Explorer support that we're looking for.

After rebuilding our solution, we see the tests show up in the Test Explorer:

That's pretty cool.

Note: For this example, I unloaded the MSTest project so that we're only seeing the NUnit tests. If we have MSTest projects loaded as well, then we would see all of the tests mixed together.

Parameterized Test Results
The results of our parameterized tests are shown here. We only have 5 actual test methods, but we have 18 passing tests.

If we look at one of our parameterized tests:

We'll see that we have a test result for each of the test cases:

We don't get the same grouping as we get with the custom test runner, but the important thing is that we get all of the results.

A Few Shortcomings
Everything isn't quite perfect here. And that's okay once we understand it. (And I'll be looking into this further to see if there are some ways around it.)

One shortcoming has to do with the way that CodeLens works with these tests. Here's what we see when we turn on CodeLens and take a look at our production method:

Notice where it says "1/1 passing". All 5 of our tests reference this method. But CodeLens only sees one of the tests. And if we look at the details, we can see which test it is.

Let's look at that test:

Notice that this is *not* a parameterized test. This has the "[Test]" attribute rather than "[TestCase]". And if we see the green circle/check mark. This tells us that this test currently passes.

But what if we look at one of the other tests:

Here we have 2 test cases (the same that we saw above). And notice that there is no green circle/check mark here. So, CodeLens is not recognizing this as a test.

This isn't a huge deal to me because I don't normally use CodeLens (I find it more distracting than helpful). But it has another implication, though: my favorite button:

The "Run Tests After Build" button is a toggle button in the Test Explorer (sorry, this is not in all editions of Visual Studio; I'm using Visual Studio 2013 Ultimate). When it is clicked down, affected unit tests are automatically re-run after every build.

When I say "affected unit tests", these are tests that are impacted by changes to the code or changes to the tests. The great thing about this is that I get immediate feedback each time I build my application.

The problem is that the tests are not getting associated with the methods that they are testing. As we saw above, only 1 of the tests is actually associated with our GetNewState method. So if that method is changed, only 1 test is affected (and the other tests are not re-run).

I've noticed this behavior as I've been working with NUnit in some other areas. This means that I've had to manually re-run tests after I've made code changes. It's not a huge deal, but it takes away a piece of functionality that I really liked.

I'm looking to see if there is a solution to this (such as adding another attribute to my tests). And hopefully this will be addressed in future versions of the NUnit Test Adapter (I'm using version 2.0.0 from April 2015) -- and it may not even be a problem with the Test Adapter, I'm just guessing.

Wrap Up
Big plus: NUnit tests can be integrated in the Visual Studio Test Explorer. Bit of a minus: there are a few quirks with CodeLens and "Run Tests After Build".

But I'm willing to deal with the shortcomings as I dig into NUnit a bit more. Next time, I'll show how NUnit lets us easily test for specific exceptions in specific places in our code. Testing for expected exceptions is a big part of my testing process, so I'm very interested in this. And what I've seen so far is really cool.

Happy Coding!