In the last article (Abstraction: The Goldilocks Principle), we took a look at some steps for determining the right level of abstraction for an application. We skipped a big reason why we might want to add abstractions to our code: testability. Some developers say that unit testing is optional (I used to be in this camp). But most developers agree that in order to be a true professional, we must unit test our code (and this is where I am now).
But this leads us to a question: Should we modify our perfectly good code in order to make it easily testable? I'll approach this from another direction: one of the qualities of "good code" is code that is easily testable.
We'll loop back to why I take this approach in just a moment. First, I want to describe my personal journey with unit testing.
A Rocky Start
Several years back, I didn't think that I needed unit testing. I was working on a very agile team of developers (not formally "big A" Agile, but we self-organized and followed most of the Agile principles). We had a very good relationship with our end users, we had a tight development cycle (which would naturally vary depending on the size of the application), we had a good deployment system, and we had a quick turnaround on fixes after an application was deployed. With a small team, we supported a large number of applications (12 developers with 100 applications) that covered every line of business in the company. In short, we were [expletive deleted] good.
We weren't using unit testing. And quite honestly, we didn't really feel like we were missing anything by not having it. But companies reorganize (like they often do), and our team became part of a larger organization of five combined teams. Some of these other teams did not have the same level of productivity that we had. Someone had the idea that if we all did unit testing, then our code would get instantly better.
So, I had a knee-jerk reaction (I had a lot of those in the past; I've mellowed a lot since then). The problem is that unit testing does not make bad code into good code. Unit testing was being proposed as a "silver bullet". So, I fought against it -- not because unit testing is a bad idea, but because it was presented as a cure-all.
Warming Up
After I have a knee-jerk reaction, my next step is to research the topic to see if I can support my position. And my position was not that unit testing was bad (I knew that unit testing was a good idea); my position was that unit testing does not instantly make bad code into good code. So, I started reading. One of the books I read was The Art of Unit Testing by Roy Osherove. This is an interesting read, but the book is not designed to sell you on the topic of unit testing; it is designed to show you different techniques and assumes that you have already bought into the benefits of unit testing.
Not a Silver Bullet
I pulled passages out of the book that confirmed my position: that if you have developers who are writing bad code, then unit testing will not fix that. Even worse, it is very easy to write bad unit tests (that pass) that give you the false impression that the code is working as intended.
But Good Practice
On a personal basis, I was looking into unit testing to see if it was a tool that I wanted to add to my tool box. I had heard quite a few really smart developers talk about unit testing and Test Driven Development (TDD), so I knew that it was something that I wanted to explore further.
TDD is a topic that really intrigued me. It sounded really good, but I was skeptical about how practical it was to follow TDD in a complex application -- especially one that is using a business object framework or application framework that supplies a lot of the plumbing code.
Actual Unit Testing
I've since moved on and have used unit testing on several projects. Now that I am out of the environment that caused the knee-jerk reaction, I fully embrace unit testing as part of my development activities. And I have to say that I see the benefits frequently.
Unit Tests have helped me to manage a code base with complex business rules. I have several hundred tests that I run frequently. As I refactor code (break up methods and classes; rearrange abstractions; inject new dependencies), I can make sure that I didn't alter the core functionality of my code. A quick "Run Tests" shows that my refactoring did not break anything. When I add new functionality, I can quickly make sure that I don't break any existing functionality.
And there are times when I do break existing functionality. But when I do, I get immediate feedback; I'm not waiting for a tester to find a bug in a feature that I've already checked off. Sometimes, the tests themselves are broken and need to be fixed (for example, a new dependency was introduced in the production code that the test code is not aware of).
I'm not quite at TDD yet, but I can feel myself getting very close. There are a few times that I've been writing unit tests for a function that I just added. As I'm writing out the tests, I go through several test cases and sometimes come across a scenario that I did not code for. In that case, I'll go ahead and write the test (knowing that it will fail), and then go fix the code. So, I've got little bits of TDD poking through my unit testing process. But since I'm writing the code first, I do end up with some areas that are difficult to unit test. I usually end up refactoring these areas into something more testable. With a test-first approach, I might have gotten it "right" the first time.
Over the next couple weeks, I'll be diving into a personal project where I'm going to use TDD to see how it will work in the types of apps that I normally build.
One thing to note: my original position has not changed. Unit Testing is a wonderful tool, and I believe that all developers should be unit testing at some level (hopefully, a level that covers the majority of the code). But unit testing will not suddenly fix broken code nor will it fix a broken development process.
TDD
Now that I've seen the benefits of unit testing firsthand, I've become a big proponent of it. And even though I didn't see the need for it several years back, I look back now and see how it could have been useful in those projects. This would include everything from verifying that automated messages are sent out at the right times to ensuring that business rule parameters are applied properly to verifying that a "cleaning" procedure on user data was running properly. With these types of tests in place, I could have made feature changes with greater confidence.
Advantages
When a developer is using TDD, he/she takes the Red-Green-Refactor approach. There are plenty of books and articles that describe TDD, so I won't go into the details. Here's the short version: The first step (Red) is to write a failing test based on a function that is required in the application. The next step (Green) is to write just enough code for the test to pass. The last step (Refactor) comes once you're a bit further into the process, but it is the time to re-arrange and/or abstract the code so that the pieces fit together.
The idea is that we only write enough code to meet the functionality of our application (and no more). This gets us used to writing small units of code. And when we start composing these small units of code into larger components, we need to maintain the testability (to make sure our tests keep working). This encourages us to stay away from tight coupling and putting in abstraction layers as we need them (and not before). In addition, we'll be more likely to write small classes that work in isolation. And when we do need to compose the classes, we can more easily adhere to the S.O.L.I.D. principles.
Back to the Question
So this brings us back to the question we asked at the beginning: Should we modify our perfectly good code in order to make it easily testable? And the answer is that if we write our tests first, then well-abstracted testable code will be the natural result.
I've done quite a bit of reading and other study of Unit Testing. One of the things that I didn't like about some of the approaches is when someone "cracked open" a class in order to test it. Because the unit they wanted to test was inaccessible (perhaps due to a "protected" access modifier), they wrote a test class that wrapped the code in order to make those members visible. I never liked the idea of writing test classes that wrapped the production classes -- this means that you aren't really testing your production code.
However, if we take the approach of test-first development (with TDD or another methodology), then the classes we build will be inherently testable. In addition, if a member needs to be injected or exposed for testing, it grows more organically with the corresponding tests.
Abstractions
And back to the last article about abstractions: if we approach our code from the test side, we will add abstractions naturally as we need them. If we need to compose two objects, we can use interfaces to ensure loose-coupling and make it easier to mock dependencies in our tests. If we find that we need to inject a method from another class, we can use delegates (and maybe the Strategy pattern) to keep the separation that we need to maintain our tests.
Wrap Up
Whole books have been written on unit testing (by folks with much more experience than I have). The goal here was to describe my personal journey toward making unit tests a part of my regular coding activities. I fought hard against it in one environment; this made me study hard so that I understood the domain; this led to a better understanding of the techniques and pitfalls involved. There are three or four key areas that I have changed my thinking on over the years (one of which is the necessity of unit testing). When I change my mind on a topic, it's usually due to a deep-dive investigation and much thought. Unit testing was no different.
You may be working in a group that is not currently unit testing. You may also think that you are doing okay without it. You may even think that it will take more time to implement. I've been there. Start looking at the cost/benefit studies that have been done on unit testing. The reality is that overall development time is reduced -- remember, development is not simply slapping together new code; it is also maintenance and bug-hunting. I wish that I had done this much sooner; I know that my past code could have benefited from it.
My journey is not yet complete; it will continue. And I expect that I
will be making the transition to TDD for my own projects very soon.
Maybe the other developers on your team don't see the benefits of unit testing, and maybe your manager sees it as an extension to the timeline. It's easy to just go along with the group on these things. But at some point, if we truly want to excel as developers, we need to stop being followers and start being leaders.
Happy Coding!
Sunday, October 21, 2012
Saturday, October 20, 2012
Abstraction: The Goldilocks Principle
Abstractions are an important part of object-oriented programming. One of the primary principles is to program to an abstraction rather than a concrete type. But that leads to a question: What is the right level of abstraction for our applications? If we have too much abstraction, then our applications can become difficult to understand and maintain. If we have too little abstraction, then our applications can become difficult to understand and maintain. This means that there is some "sweet spot" right in the middle. We'll call this the Goldilocks level of abstraction: "Just Right."
So, what is the right level? Unfortunately, there are no hard-and-fast rules that are appropriate for all development scenarios. Like so many other decisions that we have to make in software development, the correct answer is "it depends." No one really likes this answer (especially when you're just getting started and are looking for guidance), but it's the reality of our field.
Here are a few steps that we can use to get started.
Step 1: Know Your Tools
The first step to figuring out what abstractions to use is to understand what types of abstractions are available. This is what I focus on in my technical presentations. I speak about delegates, design patterns, interfaces, generics, and dependency injection (among other things). You can get information on these from my website: http://www.jeremybytes.com/Demos.aspx. The goal of these presentations is to provide an overview of the technologies and how they are used. This includes examples such as how to use delegates to implement the Strategy pattern or how to use interfaces with the Repository and Decorator patterns.
We need to understand our tools before we can use them. I try to show these technologies in such a way that if we run into them in someone else's code, they don't just look like magic incantations. If we can start looking at other people's code, we can get a better feel for how these tools are used in the real world.
A while back, I wrote an article about this: Design Patterns: Understand Your Tools. Although that pertains specifically to design patterns, the principle extends to all of the tools we use. We need to know the benefits and limitations to everything in our toolbox. Only then can we make an informed choice.
I remember seeing a woodworking show on television where the host used only a router (a carpentry router, not a network router). He showed interesting and unintended uses for the router as he used it as a cutting tool and a shaping tool and a sanding tool. He pushed the tool to its limits. But at the same time, he was limiting himself. He could use it as a cutting tool, but not as effectively as a band saw or a jigsaw or a table saw. I understand why this may be appealing: power tools are expensive. But in the software world, many of the "tools" that we use are simply ideas (such as design patterns or delegates or interfaces). There isn't a monetary investment required, but we do need to make a time investment to learn them.
Step 2: Know Your Environment
The next step is to understand your environment. Whether we are working for a company writing business applications, doing custom builds for customers, or writing shrink-wrap software, this means that we need to understand our users and the system requirements. We need to know what things are likely to change and which are not.
As an example, I worked for many years building line-of-business applications for a company that used Microsoft SQL Server. At that company, we always used SQL Server. Out of the 20 or so applications that I worked on, the data store was SQL Server. Because of this, we did not spend time abstracting the data access code so that it could easily use a different data store. Note: we did have proper layering and isolation of the data access code (meaning, all of our database calls were isolated to specific data access methods in a specific layer of the application).
On the other hand, I worked on several applications that used business rules for processing data. These rules were volatile and would change frequently. Because of this, we created rule interfaces that made it very easy to plug-in new rule types.
I should mention that these applications could have benefited from abstraction of the database layer to facilitate unit testing. We were not doing unit testing. In my next article, I will talk more about unit testing (and my particular history with it), and why it really is something we should all be doing.
Step 3: Learn the Smells
A common term that we hear in the developer community is "code smells". This basically comes with experience. As a developer, you look at a bit of code and something doesn't "smell" right -- it just feels off. Sometimes you can't put your finger on something specific; there's just something that makes you uncomfortable.
There are a couple of ways to learn code smells. The preferred way is through mentorship. Find a developer with more experience than you and learn from him/her. As a young developer, I had access to some really smart people on my development team. And by listening to them, I saved myself a lot of pain over the years. If you can learn from someone else, then be sure to take advantage of it.
The less preferred (but very effective) way of learning code smells is through trial and error. I had plenty of this as a young developer as well. I took approaches to applications that I later regretted. And in that environment, I got to live with that regret -- whenever we released a piece of software, we also became primary support for that software. This is a great way to encourage developers to produce highly stable code that is really "done" before release. While these applications were fully functional from a user standpoint, they were more difficult to maintain and add new features than I would have liked. But that's another reality of software development: constant learning. If we don't look at code that we wrote six months ago and say "What was I thinking?", then we probably haven't learned anything in the meantime.
Step 4: Abstract as You Need It
I've been burned by poorly designed applications in the past -- abstractions that added complexity to the application without very much (if any) benefit. As a result, my initial reaction is to lean toward low-abstraction as an initial state. I was happy to come across the advice to "add abstraction as you need it". This is an extremely good approach if you don't (yet) know the environment or what things are likely to change.
As an example, let's go back to database abstractions. It turns out that while working at the company that used SQL Server, I had an application that needed to convert from SQL Server to Oracle. The Oracle database was part of a third-party vendor product. For this application, I added a fully-abstracted repository that was able to talk to either SQL Server or the Oracle database. But I just did this for one application when I needed it.
Too Much Abstraction
As mentioned above, too much abstraction can make an application difficult to understand and maintain. I encountered an application that had a very high level of abstraction. There were new objects at each layer (even if the object had exactly the same properties as one in another layer). The result of this abstraction meant that if someone wanted to add a new field to the UI (and have it stored in the database), the developer would need to modify 17 different code files. In addition, much of the application was wired-up at runtime (rather than compile time), meaning that if you missed a change to a file, you didn't find out about it until you got a runtime error. And since the files were extremely decoupled, it was very difficult to hook up the debugger to the appropriate assemblies.
Too Little Abstraction
At the other end of the spectrum, too little abstraction can make an application difficult to understand and maintain. Another application that I encountered had 2600 lines of code in a single method. It was almost impossible to follow the logic (lots of nested if/else conditions in big blocks of code). And figuring out the proper place to make a change was nearly impossible.
"Just Right" Abstraction
My biggest concern as a developer is finding the right balance -- the Goldilocks Principle: not too much, not too little, but "just right". I've been programming professionally for 12 years now, so I've had the benefit of seeing some really good code and some really bad code (as well as writing some really good code and some really bad code).
Depending on what kind of development work we do, we can end up spending a lot of time supporting and maintaining someone else's code. When I'm writing code, I try to think of the person who will be coming after me. And I ask myself a few key questions. Will this abstraction make sense to someone else? Will this abstraction make the code easier or harder to maintain? How does this fit in with the approach used in the rest of this application? If you are working in a team environment, don't be afraid to grab another developer and talk through a couple of different options. Having another perspective can make the decision a lot easier.
The best piece of advice I've heard that helps me write maintainable code: Always assume that the person who has to maintain your code is a homicidal maniac who knows where you live.
One other area that will impact how much abstraction you add to your code is unit testing. Abstraction often helps us isolate code so that we can make more practical tests. I'll be putting down my experiences and thoughts regarding unit testing in the next article. Until then...
Happy Coding!
So, what is the right level? Unfortunately, there are no hard-and-fast rules that are appropriate for all development scenarios. Like so many other decisions that we have to make in software development, the correct answer is "it depends." No one really likes this answer (especially when you're just getting started and are looking for guidance), but it's the reality of our field.
Here are a few steps that we can use to get started.
Step 1: Know Your Tools
The first step to figuring out what abstractions to use is to understand what types of abstractions are available. This is what I focus on in my technical presentations. I speak about delegates, design patterns, interfaces, generics, and dependency injection (among other things). You can get information on these from my website: http://www.jeremybytes.com/Demos.aspx. The goal of these presentations is to provide an overview of the technologies and how they are used. This includes examples such as how to use delegates to implement the Strategy pattern or how to use interfaces with the Repository and Decorator patterns.
We need to understand our tools before we can use them. I try to show these technologies in such a way that if we run into them in someone else's code, they don't just look like magic incantations. If we can start looking at other people's code, we can get a better feel for how these tools are used in the real world.
A while back, I wrote an article about this: Design Patterns: Understand Your Tools. Although that pertains specifically to design patterns, the principle extends to all of the tools we use. We need to know the benefits and limitations to everything in our toolbox. Only then can we make an informed choice.
I remember seeing a woodworking show on television where the host used only a router (a carpentry router, not a network router). He showed interesting and unintended uses for the router as he used it as a cutting tool and a shaping tool and a sanding tool. He pushed the tool to its limits. But at the same time, he was limiting himself. He could use it as a cutting tool, but not as effectively as a band saw or a jigsaw or a table saw. I understand why this may be appealing: power tools are expensive. But in the software world, many of the "tools" that we use are simply ideas (such as design patterns or delegates or interfaces). There isn't a monetary investment required, but we do need to make a time investment to learn them.
Step 2: Know Your Environment
The next step is to understand your environment. Whether we are working for a company writing business applications, doing custom builds for customers, or writing shrink-wrap software, this means that we need to understand our users and the system requirements. We need to know what things are likely to change and which are not.
As an example, I worked for many years building line-of-business applications for a company that used Microsoft SQL Server. At that company, we always used SQL Server. Out of the 20 or so applications that I worked on, the data store was SQL Server. Because of this, we did not spend time abstracting the data access code so that it could easily use a different data store. Note: we did have proper layering and isolation of the data access code (meaning, all of our database calls were isolated to specific data access methods in a specific layer of the application).
On the other hand, I worked on several applications that used business rules for processing data. These rules were volatile and would change frequently. Because of this, we created rule interfaces that made it very easy to plug-in new rule types.
I should mention that these applications could have benefited from abstraction of the database layer to facilitate unit testing. We were not doing unit testing. In my next article, I will talk more about unit testing (and my particular history with it), and why it really is something we should all be doing.
Step 3: Learn the Smells
A common term that we hear in the developer community is "code smells". This basically comes with experience. As a developer, you look at a bit of code and something doesn't "smell" right -- it just feels off. Sometimes you can't put your finger on something specific; there's just something that makes you uncomfortable.
There are a couple of ways to learn code smells. The preferred way is through mentorship. Find a developer with more experience than you and learn from him/her. As a young developer, I had access to some really smart people on my development team. And by listening to them, I saved myself a lot of pain over the years. If you can learn from someone else, then be sure to take advantage of it.
The less preferred (but very effective) way of learning code smells is through trial and error. I had plenty of this as a young developer as well. I took approaches to applications that I later regretted. And in that environment, I got to live with that regret -- whenever we released a piece of software, we also became primary support for that software. This is a great way to encourage developers to produce highly stable code that is really "done" before release. While these applications were fully functional from a user standpoint, they were more difficult to maintain and add new features than I would have liked. But that's another reality of software development: constant learning. If we don't look at code that we wrote six months ago and say "What was I thinking?", then we probably haven't learned anything in the meantime.
Step 4: Abstract as You Need It
I've been burned by poorly designed applications in the past -- abstractions that added complexity to the application without very much (if any) benefit. As a result, my initial reaction is to lean toward low-abstraction as an initial state. I was happy to come across the advice to "add abstraction as you need it". This is an extremely good approach if you don't (yet) know the environment or what things are likely to change.
As an example, let's go back to database abstractions. It turns out that while working at the company that used SQL Server, I had an application that needed to convert from SQL Server to Oracle. The Oracle database was part of a third-party vendor product. For this application, I added a fully-abstracted repository that was able to talk to either SQL Server or the Oracle database. But I just did this for one application when I needed it.
Too Much Abstraction
As mentioned above, too much abstraction can make an application difficult to understand and maintain. I encountered an application that had a very high level of abstraction. There were new objects at each layer (even if the object had exactly the same properties as one in another layer). The result of this abstraction meant that if someone wanted to add a new field to the UI (and have it stored in the database), the developer would need to modify 17 different code files. In addition, much of the application was wired-up at runtime (rather than compile time), meaning that if you missed a change to a file, you didn't find out about it until you got a runtime error. And since the files were extremely decoupled, it was very difficult to hook up the debugger to the appropriate assemblies.
Too Little Abstraction
At the other end of the spectrum, too little abstraction can make an application difficult to understand and maintain. Another application that I encountered had 2600 lines of code in a single method. It was almost impossible to follow the logic (lots of nested if/else conditions in big blocks of code). And figuring out the proper place to make a change was nearly impossible.
"Just Right" Abstraction
My biggest concern as a developer is finding the right balance -- the Goldilocks Principle: not too much, not too little, but "just right". I've been programming professionally for 12 years now, so I've had the benefit of seeing some really good code and some really bad code (as well as writing some really good code and some really bad code).
Depending on what kind of development work we do, we can end up spending a lot of time supporting and maintaining someone else's code. When I'm writing code, I try to think of the person who will be coming after me. And I ask myself a few key questions. Will this abstraction make sense to someone else? Will this abstraction make the code easier or harder to maintain? How does this fit in with the approach used in the rest of this application? If you are working in a team environment, don't be afraid to grab another developer and talk through a couple of different options. Having another perspective can make the decision a lot easier.
The best piece of advice I've heard that helps me write maintainable code: Always assume that the person who has to maintain your code is a homicidal maniac who knows where you live.
One other area that will impact how much abstraction you add to your code is unit testing. Abstraction often helps us isolate code so that we can make more practical tests. I'll be putting down my experiences and thoughts regarding unit testing in the next article. Until then...
Happy Coding!
Thursday, October 11, 2012
Session Spotlight - Dependency Injection: A Practical Introduction
I'll be speaking at SoCal Code Camp (Los Angeles) on October 13 & 14. If you're not signed up yet, head over to the website and let them know that you're coming: http://www.socalcodecamp.com/.
Also, I've got a brand new session: Dependency Injection: A Practical Introduction
How Do We Get Started With Dependency Injection?
One of the big problems with getting started with Dependency Injection (DI) is that there are a lot of different opinions on exactly what DI is and the best ways to use it. At its core, DI is just a set of patterns for adding good abstraction to our code. The objects we create should be focused on doing one thing and doing it well. If there is functionality that is not core to the the object, then we "outsource" it to another class. This external class is a dependency (our primary object depends on this other class to provide a functional whole in our application).
Dependency Injection gives us a way to keep these external dependencies separate from our objects. Rather than the object being responsible for creating/managing the dependency, we "inject" it from the outside. This makes sure that our classes are loosely coupled which gives us good maintainability, extensibility, and testability.
In this session, I combine my personal experience using DI with the excellent information provided by Mark Seemann in Dependency Injection in .NET (I posted a review on this book a few weeks back).
DI is an enormous topic. I have picked out a few key areas (like examining tight-coupling and loose-coupling) and design patterns that will give us a good starting point for the world of DI. (As a side note, the sample code also gives a quick example of using the Model-View-ViewModel (MVVM) design pattern.)
If you can't make it out to the SoCal Code Camp, you have another chance to see this session at the Desert Code Camp (in the Phoenix, AZ area) in November. And as always, the code samples and walkthrough are posted on my website.
Hope to see you at an upcoming event.
Happy Coding!
Also, I've got a brand new session: Dependency Injection: A Practical Introduction
How Do We Get Started With Dependency Injection?
One of the big problems with getting started with Dependency Injection (DI) is that there are a lot of different opinions on exactly what DI is and the best ways to use it. At its core, DI is just a set of patterns for adding good abstraction to our code. The objects we create should be focused on doing one thing and doing it well. If there is functionality that is not core to the the object, then we "outsource" it to another class. This external class is a dependency (our primary object depends on this other class to provide a functional whole in our application).
Dependency Injection gives us a way to keep these external dependencies separate from our objects. Rather than the object being responsible for creating/managing the dependency, we "inject" it from the outside. This makes sure that our classes are loosely coupled which gives us good maintainability, extensibility, and testability.
In this session, I combine my personal experience using DI with the excellent information provided by Mark Seemann in Dependency Injection in .NET (I posted a review on this book a few weeks back).
DI is an enormous topic. I have picked out a few key areas (like examining tight-coupling and loose-coupling) and design patterns that will give us a good starting point for the world of DI. (As a side note, the sample code also gives a quick example of using the Model-View-ViewModel (MVVM) design pattern.)
If you can't make it out to the SoCal Code Camp, you have another chance to see this session at the Desert Code Camp (in the Phoenix, AZ area) in November. And as always, the code samples and walkthrough are posted on my website.
Hope to see you at an upcoming event.
Happy Coding!
Subscribe to:
Posts (Atom)