Mixed Feelings About FsUnit

A few days ago, I did an initial experiment using FsUnit to test C# code. My initial tests were pretty positive. Since then, I've done a bit more experimentation, and now my feelings are a bit more mixed.

For the code, I'm using the project from "Test-Driven Development in the Real World". The scenario is getting data from a service, parsing the JSON result, and then converting things to date/time objects that we can use. We'll look at tests written in FsUnit and tests written in NUnit. Granted, I might be "doing it wrong" with FsUnit (since F# has a different coding style), so if you have any suggestions to improve my "problems", I'd like to hear them.

The Good
We'll start by looking at some scenarios where I like the FsUnit tests a bit better. These are similar to the tests we saw in the previous article.

First up, parsing a particular value from a JSON string:

FsUnit testing JSON parsing

NUnit testing JSON parsing

These tests are pretty similar. For the FsUnit test, not having the intermediate variable helps with readability (and we could do something similar with the NUnit test). But here, the "fluent" syntax works pretty well.

The same is true when we're checking to see if our class implements a particular interface:

FsUnit checking for an interface

NUnit checking for an interface

Here again, the code is very similar, but I like the readability of the FsUnit. In fact, I think this could be improved a bit by removing the line break between "calculator" and the pipe-forward operator. This would give us a very sentence-like structure: "calculator should be an instance of ISunsetCalculator".

The Indifferent
There are a couple instances where I don't have a clear preference between the styles of the tests.

Here are two ways that we can check to see if a method threw a specific exception:

FsUnit checking for an exception

NUnit checking for an exception

Again, these are very similar. Part of that is because I'm using the NUnit-specific "Assert.Throws" method that takes a delegate as a parameter.

The FsUnit code also needs to have a function call passed in to the "should throw" function. The piece that I don't like about this code is the "|> ignore". I really wish this was not necessary. But without this bit of code, the test does not compile.

So I do like the more sentence-like code of FsUnit (where we have the function call first). But the "|> ignore" seems to clutter things a bit. That's why I'm putting this test in the middle. I'm not sure if I have a preference either way.

The Ugly
Once I got to the point where I had a bit more setup code and more complex test values, things got a bit uglier for FsUnit. Again, I'm open to suggestions on improving the code here, so feel free to send them in.

This test checks to see that a time string is turned into an appropriate DateTime value. For this, we need a couple of DateTime objects to use for the tests:

FsUnit with DateTime values

NUnit with DateTime values

This is where I prefer the NUnit test a bit better. One reason is that I can have a clear Arrange / Act / Assert layout to the code. The first 3 lines set up our variables that we use for the tests. The next line performs the action of calling the method. And the last line checks that the actual output matches the expected one.

The FsUnit test is a bit less clear. I ended up with a bunch of "let"s to set up the different variables. So, I added an "expectedTimeString" to the top of the file that had the "4:44:12 PM" value that we've seen in other tests. I don't know if this made things better or worse.

I don't like having a lot of setup code because that moves information out of the tests, and now I need to go look somewhere else to find it. But I do like factory methods for setting up things like mocks. This will take a bit more work to try to figure out the most readable way of putting this together.

Here's another example which checks the end-to-end functionality contained in the "GetSunset" method:

FsUnit checking the full process

NUnit checking the full process

I have the same readability concerns as the prior example. The NUnit test has a clear Arrange / Act / Assert organization. The FsUnit has some setup and then the calls. Because of this, the "fluent" syntax of the action and verification gets a bit lost. Again, there may be some ways to make this a bit better.

The fun part about this particular test is that it is incomplete. The code is not yet accessing the service that generates the JSON result. Once this is in place, we need to mock out the service call so that we can test independently. Here's what that NUnit test looks like:

NUnit with a mock service

I haven't written a similar test in FsUnit because I want to see what I can do about some of the concerns that I found with this set of experiments. I've got some more to look into.

[Update: I explored simple mocking with F#, and I *really* like what I found: Simple Mocking in FsUnit.]

Wrap Up
When dealing with more real-world OO code, the readability advantage of FsUnit seems to fade a bit. If you have suggestions on improving readability, please send them along.

For me, readability of unit tests is extremely important. There should be no trickery in tests (even if we need some trickery or optimization in the code we're testing). They should be approachable and easy to maintain.

Readability is a concern that I have with functional programming in general. And this may be something that I need to "get over". But I'm all about making code as approachable as possible. I don't want to exclude people who want to learn functional programming because the learning curve is too steep. I know that there is a happy medium there somewhere.

Happy Coding!