Unit Test Coverage: What Parts of Your Application Do Your Users *Not* Care About?

Developer events are a great place to talk to tons of folks and get different viewpoints. This past weekend at the South Florida Code Camp was no different. I had a chance to spend some time talking with my friend Barry Stahl (blog & Twitter) about unit testing, and he's got some great perspective.

My Unit Testing History
I'm a big proponent of unit testing. I've seen the advantages in my own work. With good tests in place, I code faster, I code more confidently, and I'm more productive. But I've also had an interesting path to get where I am today.

2-1/2 years ago, I first wrote about my journey into unit testing. I can't say that I'm proud of the path that I took, but eventually I ended up where I needed to be. Since then, I've been exploring different techniques to find things that work for me. This included looking at TDD and going through Kent Beck's book Test-Driven Development by Example.

I'm still not completely in the TDD world. I do use it from time to time. I like to experiment with it (like with TDD & Conway's Game of Life). But I still have a bit of disconnect of how to apply TDD in certain situations, for example when I need to get data out of an unfamiliar service.

So, right now I'm more of a "test along side" kind of person. I interleave writing code and unit tests, but usually the code gets written first.

Code Coverage
So back to Barry. Barry and I ended up talking about TDD and unit testing in general. So I brought up how there are certain things that I'm not sure how to test first and also how I don't necessarily agree that code needs 100% test coverage. (This is probably a knee-jerk reaction to some folks I worked with who *only* cared about having 100% coverage regardless of whether the tests were any good or not.)

And that's when Barry said,

What parts of your application do your users *not* care about?

That's pretty hard to argue with. If something is important to our users, then we should have a test that exercises that code. The tests are the proof that the code actually does what we think it does.

(And if there are parts of the application the users don't care about, then those parts probably should be removed altogether.)

Our discussion went a bit further on the parts that we unit test vs. manual test vs. something in between: UI elements/layout, business logic, presentation code, etc. We need to make sure we're using the best tool for the job.

But this isn't the only thing that Barry gave me a different perspective on.

Do I Need a Repository?
I use data repositories in a lot of my demo code -- primarily because it's an easy concept to grasp. This makes it an approachable way to show different abstraction techniques (whether interfaces, dependency injection, or something else). And I've already looked at the question of Do I Really Need a Repository?

One of the primary reasons I have a tendency to say "probably not" is because I don't like to add an abstraction that is not being used. This adds complexity to our code, and we really shouldn't do that unless we're getting some real benefit back.

The YAGNI principle guides us to not code based on speculation. So if someone asks if they should add a repository interface because they *might* want to switch from Microsoft SQL Server to Oracle (or some other data store) at some point in the future, I would tell them *not* to add the abstraction yet.

The truth is that we rarely need a 2nd data store in our application. At least that's what I thought. Barry's viewpoint was that we almost always need a 2nd data store:

The 2nd data store is the fake or mock object that we use in our unit tests.

Again, this is pretty hard to argue with. In order to adequately test our business logic and presentation code, we need to de-couple it from the concrete data store. We don't want a dependency in SQL Server to cause problems in a test of our business logic. (And in my presentations on interfaces and dependency injection, I specifically show how they can make unit testing much, much easier.)

So this has made me rethink things a little bit. I'm still reluctant to add complexity to my code for the sole purpose of unit testing. But we also need to strike a balance at some point. If my tests become overly complex (because I have to use reflection to break dependencies), then it's likely that I won't maintain tests or have adequate coverage. There is a balance point where we have code that is easy to maintain and also easy to test.

Wrap Up
I love talking to other developers. Barry is one of my friends who I met in Phoenix (where he lives). Since then we've talked at lots of other events including in Southern California (where I live). What's interesting about our last meeting is that we were both in South Florida -- several thousand miles from home for both of us.

As we interact with other developers, we can learn from their experiences. Barry is big on TDD, so I'll have to tap into him for a pair programming session to go through some of the areas that I struggle with.

I met a lot of great developers at the South Florida Code Camp -- mostly new people since I was far from home. But I saw a few familiar faces as well, like Jim Wooley. I really wanted to go to both of Jim's sessions -- one on Roslyn and one on Reactive Extensions -- but I was scheduled to present during the same time slots. (That happens a lot as a speaker.) But we had a chance to talk again toward the end of the day, and I got the short version of his Roslyn talk.

As we get different perspectives, we expand on the way we think about things. I find myself getting nudged in new directions, and I look forward to using them to become a more effective developer.

Happy Coding!

Applying the Interface Segregation Principle to a Repository

Repositories appear in many of my example applications. This is because it's a fairly easy problem to describe and understand. Last month, we took a look at whether we really need this in our applications: Do I Really Need a Repository?

My friend, Jim, replied with the following:

@jeremybytes Scenario 4: Apply the interface segregation principle to repositories and get ILoader and ISaver https://t.co/MLVqeKrQJT
— James R Counts (@jamesrcounts) August 19, 2013

This is a great point, so let's take a closer look at this.

A Generic Repository
Before looking at the Interface Segregation Principle, let's update the existing interface to use generic parameters.  Here's our original interface from the last article:

This is an example of a CRUD Repository -- where CRUD stands for Create, Read, Update, and Delete. (Another common repository type is CQRS -- Command Query Responsibility Separation -- we'll talk about this in a bit.)

If we add generic parameters to this, we can get this interface to work with more than just Person objects. Everywhere we see "Person", we'll replace with "T", and everywhere we see "string" (which is the type for the primary key), we'll replace with "TKey". Then we just update the method names so that they make sense for any types.

Here's our updated generic repository interface:

This interface will work with any types we have (Customers, Orders, Products, etc.) as well as whatever key types we have (int, string, GUID, etc.).

Now let's look at how we can apply the Interface Segregation Principle to this.

The Interface Segregation Principle
The Interface Segregation Principle is one of the S.O.L.I.D. principles of Object-Oriented Design (specifically, the "I"). Here's what the principle states:

Clients should not be forced to depend upon methods that they do not use. Interfaces belong to clients, not to hierarchies.

What this means is that we should make our interfaces granular so that we only get the methods that we actually need.

When this repository interface is used in the sample applications, we generally only use the "Read" portions. Because of this, we really should not saddle our client object with the update methods (Create, Update, Delete) since it does not use them.

To apply the Interface Segregation Principle, we just need to segregate out the Read operations into a different interface.

Here's what that could look like:

This way, our application that only does Read operations uses this more granular interface. And it only needs to depend on the methods it actually uses.

If we do need a full CRUD repository, then we can simply inherit from this interface and add the other methods:

Because IRepository inherits IReadOnlyRepository, it includes the GetItems() and GetItem() methods, and then adds the other operations. This lets us be selective about the specific interface, and we only need to depend on the methods we actually use.

A CQRS Repository
But what about Jim's suggestion of having an ILoader and ISaver? This has a simplified set of methods. Here's what those interfaces look like:

We can easily imagine that the Loader would work very similar to our read-only repository. This would handle our "Read" (or "Query") operation. The Saver would handle the operations that update the data (the "Command" operations).

This actually follows the pattern of a CQRS repository. As mentioned above, CQRS stands for Command Query Responsibility Separation. This means that we keep the Command operations (the Saver) separate from the Query operations (the Loader).

What About Implementation?
One thing to notice is that the Saver has a much simpler interface. When we have the CRUD repository, we had separate methods for Create, Update, and Delete operations. With the ISaver interface, we have a single Save() method.

So, how do we know what operations we need to perform on the actual data store: Insert, Update, or Delete? That all depends on how we're storing our data.

One possibility is to only have Insert operations on our data store. We always insert the most recent record into the data store (with a current timestamp). If we have a deleted record, we insert the most recent record with a delete flag set to "true". When pulling data out, we make sure that we're fetching the most current record, but we have the full history in the database. This is fairly easy to implement in a document database. It can also be used with a relational database, however, the pattern isn't quite as common with RDBMS.

If we are using a typical RDBMS data store technique, we actually want to make Insert, Update, or Delete calls as appropriate (along with potentially logging the changes in separate change-tracking tables). But how do we know which operation we need to call on a particular record? That's where things get a bit more complex.

One solution that I've used is to use a business object framework that includes change tracking on the application objects. I've used CSLA.NET very successfully (http://www.cslanet.com/). This is a business object framework that works very well for the types of apps that we often build for companies (get data out of a database, let the user change it, put it back into the database). CSLA includes tons of useful features including change tracking, validation, authorization, and it works on a variety of platforms (web, desktop, mobile).

Using a framework such as this, we simply call "Save" on the business object, and the framework figures out which of the actual update operations to call depending on whether the record is new, changed, or deleted. (As a side note, this is a good example of the Facade design pattern where a complex API is hidden behind a simpler interface.)

Implementing this type of functionality yourself is not trivial, which is why we should look to see if there's an existing implementation that we can take advantage of.

Wrap Up
So, if we determine that we do need a repository (Do I Really Need a Repository?), our job is not necessarily done. We need to consider what type of repository will work best in our situation (CRUD vs. CQRS), and we also need to keep in mind whether we're including more methods in our interface than we actually need. If we find that we are not using all of the methods (or we find that we're using some of the methods in one class and other methods in others), we should think about splitting up our interface so that we have the right level of granularity. This is what the Interface Segregation Principle is all about.

This gives us quite a bit to think about. As always, we need to weigh the options and come up with the best solution for our particular environment.

Happy Coding!

Do I Really Need a Repository?

I use the Repository pattern in a number of my presentations, whether talking about Interfaces, Generics, or Dependency Injection. The primary reason I use this is because it's a very easy problem to describe: we need to access data from different data sources (Web Service, SQL, Oracle, NoSQL, XML, CSV), and we want to keep the complexity out of our core application code.

But the question often comes up: Do I really need a Repository in my application?

And I'll give my standard answer: It depends.

Let's take a closer look at the Repository pattern and how it fits (or doesn't fit) with some scenarios that I've dealt with.

What is a Repository?
We'll start with a definition of the Repository pattern:

Mediates between the domain and data mapping layers using a collection-like interface for accessing domain objects. 

[Fowler, et al. Patterns of Enterprise Application Architecture. Addison-Wesley, 2003.]

The idea behind having a repository is to insulate the application from the data access code. Here's a fairly typical interface that shows a CRUD (Create, Read, Update, Delete) repository:

This means that from our application code, we simply call "Repository.GetPeople()" to get a collection of Person objects from the data store. Our application code does not need to know what's on the other side of that "GetPeople" call. It could be a SQL query; it could be a service call; it could be reading from a local file. The application is insulated from that knowledge.

[Editor's Note: For more exploration of Repositories, see "Applying the Interface Segregation Principle to a Repository".]

But do we really need this in our application?

Add Abstraction as You Need It
Developers generally fall into two groups: over-abstractors and under-abstractors. Neither group is better than the other; they are just natural tendencies. But the challenge for both groups is to find just the right level of abstraction which is generally somewhere in-between.  (For a closer look, see Abstraction: The Goldilocks Principle.)

By nature, I am an under-abstractor. This is because early in my career, I supported an application that was severely over-abstracted. The application was extremely difficult to maintain, and even the senior devs had trouble getting expected behavior out of the code base. So, my first inclination is to limit abstraction as much as possible.

But I've learned some good lessons over the years, which included creating an under-abstracted application that was difficult to maintain. So now I'm always looking to find the balance. Here is the best advice that I've come across so far:

Add abstraction as you need it.

And this literally means "as you need it", not "because you think you might need it at some point in the future." This has helped keep me out of a lot of trouble, and it's also helped me when working with teammates who are over-abstractors by nature.

How does this apply to repositories?

Add a Repository When You Need It
A Repository is a layer of abstraction. So, let's look at some times when we might or might not need an explicit repository.

Scenario 1: No Repository
I worked for many years in corporate development in a fixed environment. All of our custom-built applications used Microsoft SQL Server for data. That was it. And having that consistency made it very easy to support our application portfolio.

But what this also meant is that we did not usually need an explicit Repository. It was very unlikely that we would swap out the data storage layer; it was always SQL Server. So, we did not have a repository.

Now, even though we didn't have a physical repository layer, we still kept all of our data access code nicely corralled. We did not have SQL queries scattered across the code base; we kept them in well-defined parts of our business layer.

Scenario 2: Repository
But there was a time when I needed to add a Repository. One of my applications imported data from a third-party application. That application was going through an upgrade and some changes. One of those changes was switching from Microsoft SQL Server to an Oracle Database.

So, for this application, I added a Repository layer. This allowed me to prepare the application for the switch-over once that third-party application went into production. Based on a configuration change, I could switch the application from using a SQL Server repository to an Oracle repository. This meant that I could develop, test, and deploy my updated code before the switch-over happened. And the upgrade process went very smoothly.

Scenario 3: Maybe Repository
Another reason we may want to add a repository is to facilitate automated testing. With a repository layer in place, it's easy to swap out a fake or mock repository that we can use for testing. But whether we need this will be based on how our objects are laid out.

For example, we may be working with smart-entities; a smart-entity is basically a collection of properties with retrieval and update capabilities. In this case, there really isn't anything to test. The data access methods populate the properties and update the data store. Since we don't really have any logic to test in these entities, we would not get much advantage by adding a repository layer.

On the other hand, we may be working with robust business objects; a robust business object is one which has business logic (business rules and validation) built in. In this case, we do have logic that we want to check in automated tests. A repository layer would make those tests easier, so we would get an advantage from that.

Wrap Up
Do we really need to have a repository in our application? It depends. A repository is a layer of abstraction. If we add abstraction where we don't need it, then we add unnecessary complexity to our application. So, let's add abstraction as we need it.

We've seen two main reasons why we may want to add a repository layer: (1) to swap-out data access code for different data stores, and (2) to facilitate unit testing. If either of these apply, then we should consider adding a repository.

I use the Repository pattern in my presentations because it is an easily-understandable abstraction. But whether we actually need it in our own applications depends on our environment. In most of the applications I've written, I have not needed it. But in the few where I have needed it, it has been a great asset.

Happy Coding!