Throughout this series we've talked about the importance of testability and have looked at techniques to make it easier to test our system. It goes without saying that a major benefit of writing tests for our system is the ability to deliver a better product to our client. Although this is true for unit tests as well, the main reason I write unit tests is that nothing comes close to improving the maintainability of a system as much as a properly developed suite of unit tests. You'll often hear unit test advocates speak of how much confidence unit tests give them – and that's what it's really all about. On a project I'm currently working on, we're continuously making changes and tweaks to improve the system (functional improvements, performance, refactorings, you name it). Being that it's a fairly large system, we're sometimes asked to make a change that flat out scares us. Is it doable? Will it have some weird side effect? What bugs will be introduced? Without unit tests, we'd likely refuse to make the higher risk changes. But we know, and our client knows, that it's the high risk changes that have the most potential for success. It turns out that having 700+ unit tests which run within a couple minutes lets us rip components apart, reorganize code, and build features we never thought about a year ago, without worrying too much about it. Because we are confident in the completeness of the unit tests, we know that we aren't likely to introduce bugs into our production environment – our changes might still cause bugs but we'll know about them right away.
Unit tests aren't only about mitigating high-risk changes. In my programming life, I've been responsible for major bugs caused from seemingly low-risk changes as well. The point is that I can make a fundamental or minor change to our system, right click the solution, select "Run Tests" and within 2 minutes know where we stand.
Why wasn't I unit testing 3 years ago?
For those of us who've discovered the joy of unit testing, it's hard to understand why everyone isn't doing it. For those who haven't adopted it, you probably wish we'd shut up about it already. For many years I'd read blogs and speak to colleagues who were really into unit testing, but didn't practice myself. Looking back, here's why it took me a while to get on the bandwagon:
- I had misconception about the goals of unit testing. As I've already said, unit testing does improve the quality of a system, but it's really all about making it easier to change / maintain the system later on. Furthermore, if you go to the next logical step and adopt Test Driven Development, unit testing really becomes about design. To paraphrase Scott Bellware, TDD isn't about testing because you're not thinking as a tester when doing TDD – you're thinking as a designer.
- Like many, I used to think developers shouldn't write tests! I don't know the history behind this belief, but I now think this is just an excuse for lazy programmers. Testing is the process of both finding bugs in a system as well as validating that it works as expected. Maybe developers aren't good at finding bugs in their own code, but they are the best suited to make sure it works the way they intended it to (and clients are best suited to test that it works like it should (if you're interested to find out more about that, I suggest you research Acceptance Testing and FitNess (http://fitnesse.org/)). Even though unit testing isn't all that much about testing, developers who don't believe they should test their own code simply aren't accountable.
- Testing isn't fun. Sitting in front of a monitor, inputting data and making sure everything's ok sucks. But unit testing is coding, which means there are a lot of challenges and metrics to gauge your success. Sometimes, like coding, it's a little mundane, but all in all it's no different than the other programming you do every day.
- It takes time. Advocates will tell you that unit testing doesn't take time, it SAVES time. This is true in that the time you spend writing unit tests is likely to be small compared to the time you save on change requests and bug fixes. That's a little too Ivory Tower for me. In all honesty, unit testing DOES take a lot of time (especially when you just start out). You may very well not have enough time to unit test or you client might not feel the upfront cost is justified. In these situations I suggest you identifier the most critical code and test it as thoroughly as possible – even a couple hours spent writing unit tests can have a big impact.
Ultimately, unit testing seemed like a complicated and mysterious thing that was only used in edge cases. The benefits seemed unattainable and timelines didn't seem to allow for it anyways. It turns out it took a lot of practice (I had a hard time learning what to unit test and how to go about it), but the benefits were almost immediately noticeable.
The Tools
With StructureMap already in place from the last part, we need only add 2 frameworks and 1 tool to our unit testing framework: nUnit, RhinoMocks and TestDriven.NET.
TestDriven.NET is an addon for Visual Studio that basically adds a "Run Test" option to our context (right-click) menu. We won't spend any time talking about it. The personal license of TestDriven.NET is only valid for open source and trial users. Don't worry too much if the licensing doesn't suite you, nUnit has its own test runner tool, just not integrated in VS.NET. (http://www.testdriven.net/) (Resharper users can also use its built-in functionality).
nUnit is the testing framework we'll actually use. There are alternatives, such as mbUnit, but I don't know nearly as much about them as I ought to. (http://www.nunit.org/)
RhinoMocks is the mocking framework we'll use. In the last part we manually created our mock – which was both rather limited and time consuming. RhinoMocks will automatically generate a mock class from an interface and allow us to verify and control the interaction with it. (http://www.ayende.com/projects/rhino-mocks.aspx)
nUnit
The first thing to do is to add a reference to the nunit.framework.dll and the Rhino.Mocks.dll. My own preference is to put my unit tests into their own assembly. For example, if my domain layer was located in CodeBetter.Foundations, I'd likely create a new assembly called CodeBetter.Foundations.Tests. This does mean that we won't be able to test private methods (more on this shortly). In .NET 2.0+ we can use the InternalsVisibleToAttribute to allow the Test assembly access to our internal method (we'd open Properties/AssemblyInfo.cs and add [assembly: InternalsVisibleTo("CodeBetter.Foundations.Tests")] – which is something I typically do.
There are two things you need to know about nUnit. First, you configure your tests via the use of attributes. The TestFixtureAttribute is applied to the class that contains your tests, setup and teardown methods. The SetupAttribute is applied to the method you want to have executed before each test – you won't always need this. Similarly, the TearDownAttribute is applied to the method you want executed after each test. Finally, the TestAttribute is applied to your actual unit tests. (There are other attributes, but these 4 are the most important). This is what it might look like:
using NUnit.Framework;
[TestFixture]
public class CarTests
{
[SetUp]
public void SetUp()
{
//todo
}
[TearDown]
public void TearDown()
{
//todo
}
[Test]
public void SaveThrowsExceptionWhenInvalid()
{
//todo
}
[Test]
public void SaveCallsDataAccessAndSetsId()
{
//todo
}
//more tests
}
Notice that each unit test has a very explicit name – it's important to state exactly what the test is going to do, and since your test should never do too much, you'll rarely have obscenely long names.
The second thing to know about nUnit is that you confirm that your test executed as expected via the use of the Assert class and its many methods. I know this is lame, but if we had a method that took a param int[] numbers and returned the sum, our unit test would look like:
[TestFixture]
public class MathUtilityTester
{
[Test]
public void MathUtilityReturnsZeroWhenNoParameters()
{
Assert.AreEqual(0, MathUtility.Add());
}
[Test]
public void MathUtilityReturnsValueWhenPassedOneValue()
{
Assert.AreEqual(10, MathUtility.Add(10));
}
[Test]
public void MathUtilityReturnsValueWhenPassedMultipleValues()
{
Assert.AreEqual(29, MathUtility.Add(10,2,17));
}
[Test]
public void MathUtilityWrapsOnOverflow()
{
Assert.AreEqual(-2, MathUtility.Add(int.MaxValue, int.MaxValue));
}
}
You wouldn't know it from the above example, but the Assert class has more than one function, such as Assert.IsFalse, Assert.IsTrue, Assert.IsNull, Assert.IsNotNull, Assert.AreSame, Assert.AreNotEqual, Assert.Greater, Assert.IsInstanceOfType and so on.
What is a Unit Test
Unit tests are methods that test behavior at a very granular level. Developers new to unit testing often let the scope of their tests grow. Most unit tests follow the same pattern: execute some code from your system and assert that it worked as expected. The goal of a unit test is to validate a specific behavior. You might have noticed that the above two examples use multiple unit tests on the same function. We don't want to write an all encompassing test for Save, but rather want to write a test for each of the behavior it contains – failing when the object is in an invalid state, calling our data access's Save method and setting the id, and calling the data access's Update method. It's important that our unit test pinpoint a failure as best possible.
I'm sure some of you will find the 4 tests used to cover the MathUtility.Add method a little excessive. You may think that all 4 tests could be grouped into the same one – and in this minor case I'd say whatever you prefer. However, when I started unit testing, I fell into the bad habit of letting the scope of my unit tests grow. I'd have my test which created an object, executed some of its members and asserted the functionality. But I'd always end up saying, well as long as I'm here, I might as well throw in a couple extra asserts to make sure these fields are set the way they ought to be. This is very dangerous because a change in your code could break numerous unrelated tests – definitely a sign that you've given your tests too little focus.
This brings us back to the topic about testing private methods. If you google you'll find a number of discussions on the topic, but the general consensus seems to be that you shouldn't test private methods. I think the most compelling reason not to test private methods is that our goal is not to test methods or lines of code, but rather to test behavior. This is something you must always remember. If you thoroughly test your code's public interface, then private methods should automatically get tested. Another argument against testing private methods is that it breaks encapsulation. We talked about the importance of information hiding already. Private methods contain implementation detail that we want to be able to change without breaking calling code. If we test private methods directly, implementation changes will likely break our tests, which doesn't bode well for higher maintainability. Here's a question for you: should we care that a change to a private method broke a test?
...
...
Hopefully you're starting to get my drift, the answer is NO. What we care about is whether the behavior/functionality is broken, which our tests against the public API will do.
Mocking
To get started, it's a good idea to test simple pieces of functionality. Before long though, you'll want to test a method that has a dependency on an outside component – such as the database. For example, you might want to complete your test coverage of the Car class by testing the Save method. Since we want to keep our tests as granular as possible (and as light as possible – tests should be quick to run so we can execute them often and get instant feedback) we really don't want to figure out how we'll set up a test database with fake data and make sure it's kept in a predictable state from test to test. In keeping with this spirit, all we want to do is make sure that Save interacts property with the DAL. Later on we can unit test the DAL on its own. If Save works as expected and the DAL works as expected and they interact properly with each other, we have a good base to move to more traditional testing.
In the previous part we saw the beginnings of testing with mocks. We were using a manually created mock class which had some pretty major limitations. The most significant of which was our inability to confirm that calls to our mock objects were occurring as expected. That, along with ease of use, is exactly the problem RhinoMock is meant to solve. Using RhinoMock couldn't be simpler, tell it what you want to mock (an interface or a class – preferably an interface), tell it what method(s) you expect to be called, along with the parameters, execute the call, and have it verify that your expectations were met.
Before we can get started, we need to give RhinoMock access to our internal types. This is quickly achieved by adding [assembly: InternalsVisibleTo("DynamicProxyGenAssembly2")] to our Properties/AssemblyInfo.cs file.
Now we can start coding by writing a test to cover the update path of our Save method:
using NUnit.Framework;
[TestFixture]
public class CarTest
{
[Test]
public void SaveCarCallsUpdateWhenAlreadyExistingCar()
{
MockRepository mocks = new MockRepository();
IDataAccess dataAccess = mocks.CreateMock<IDataAccess>();
ObjectFactory.InjectStub(typeof(IDataAccess), dataAccess);
Car car = new Car();
dataAccess.Update(car);
mocks.ReplayAll();
car.Id = 32;
car.Save();
mocks.VerifyAll();
ObjectFactory.ResetDefaults();
}
}
Once a mock object is created, which took 1 line of code to do, we inject it into our dependency injection framework (StructureMap in this case). When a mock is created, it enters record-mode, which means any subsequent operations against it, such as the call to dataAccess.UpdateCar(car), is recorded by RhinoMock. We exit record-mode by calling ReplayAll, which means we are now ready to execute our real code and have it verified against the recorded sequence. When we then call VerifyAll after having called Save on our Car object, RhinoMock will make sure that our actual call behaved the same as what we expected. In other words, you can think of everything before ReplayAll as stating our expectations, everything after it as our actual test code with VerifyAll doing the final check.
If we were to change the test to something along the lines of (notice the extra dataAccess.Update call):
using NUnit.Framework;
[TestFixture]
public class CarTest
{
[Test]
public void SaveCarCallsUpdateWhenAlreadyExistingCar()
{
MockRepository mocks = new MockRepository();
IDataAccess dataAccess = mocks.CreateMock<IDataAccess>();
ObjectFactory.InjectStub(typeof(IDataAccess), dataAccess);
Car car = new Car();
dataAccess.Update(car);
dataAccess.Update(car);
mocks.ReplayAll();
car.Id = 32;
car.Save();
mocks.VerifyAll();
ObjectFactory.ResetDefaults();
}
}
RhinoMock would cause our test to fail and tell us that it didn't expect two calls to update.
For the save behavior, the interaction is slightly more complex – we have to make sure the return value is properly handled by the Save method. Here's the test:
using NUnit.Framework;
[TestFixture]
public class CarTest
{
[Test]
public void SaveCarCallsSaveWhenNew()
{
MockRepository mocks = new MockRepository();
IDataAccess dataAccess = mocks.CreateMock<IDataAccess>();
ObjectFactory.InjectStub(typeof(IDataAccess), dataAccess);
Car car = new Car();
Expect.Call(dataAccess.Save(car)).Return(389);
mocks.ReplayAll();
car.Save();
mocks.VerifyAll();
Assert.AreEqual(389, car.Id);
ObjectFactory.ResetDefaults();
}
}
Using the Expect.Call method allows us to specify the return value we want. Also notice the Assert.Equals we've added – which is the last step in validating the interaction. Hopefully the possibilities of having control over return values (as well as output/ref values) lets you see how easy it is to test for edge cases. Imagine that we changed our Save function to throw an exception if the returned id was invalid, our test would look like:
using NUnit.Framework;
[TestFixture]
public class CarTest
{
private MockRepository _mocks;
private IDataAccess _dataAccess;
[SetUp]
public void SetUp()
{
_mocks = new MockRepository();
_dataAccess = _mocks.CreateMock<IDataAccess>();
ObjectFactory.InjectStub(typeof(IDataAccess), _dataAccess);
}
[TearDown]
public void TearDown()
{
_mocks.VerifyAll();
}
[Test, ExpectedException("CodeBetter.Foundations.PersistenceException")]
public void SaveCarCallsSaveWhenNew()
{
Car car = new Car();
using (_mocks.Record())
{
Expect.Call(_dataAccess.Save(car)).Return(0);
}
using (_mocks.Playback())
{
car.Save();
}
}
}
We've actually changed a lot. First, we've shown how nUnit ExpectedException attribute can be used to test for an exception. Secondly, we've extracted the repetitive code that creates, sets up and verifies the mock object into the SetUp and TearDown methods. Finally, we used a different, more explicit, RhinoMocks syntax for setting up our record and playback states - I generally prefer this syntax.
More on nUnit and RhinoMocks
So far we've only looked at the basic features offered by nUnit and RhinoMocks, but there's a lot more than can actually be done with them. For example, RhinoMocks can be setup to ignore the order of method calls, instantiate multiple mocks but only replay/verify specific ones, or mock some but not other methods of a class (a partial mock), or simply create a stub.
Combined with a utility like NCover (http://www.ncover.com/), you can also get reports on your tests coverage. Coverage basically tells you what percentage of an assembly/namespace/class/method was executed by your tests. NCover has a visual code browser that'll highlight any un-executed lines of code in red. Generally speaking, I dislike coverage as a means of measuring the completeness of unit tests. After all, just because you've executed a line of code does not mean you've actually tested it. What I do like NCover for is to highlight any code that has no coverage. In other words, just because of line of code or method has been executed by a test, doesn't mean you're test is good. But if a line of code or method hasn't been executed, then you need to look at adding some tests.
We've mentioned Test Driven Development briefly throughout this series. As has already been mentioned, Test Driven Development, or TDD, is about design, not testing. TDD means that you write your test first and then write corresponding code to make your test pass. In TDD we'd write our Save test before having any functionality in the Save method. Of course, our test would fail. We'd then write the specific behavior and test again. The general mantra for developers is red ? green ? refactor. Meaning the first step is to get a failing unit testing, then to make it pass, then to refactor the code as required.
In my experience, TDD goes very well with Domain Driven Design, because it really lets us focus on the business rules of the system. If our client says tracking dependencies between upgrades has been a major pain-point for them, then we set off right away with writing tests that'll define the behavior and API of that specific feature. I recommend that you familiarize yourself with unit testing in general before adopting TDD.
UI and Database Testing
Unit testing your ASP.NET pages probably isn't worth the effort. The ASP.NET framework is complicated and suffers from very tight coupling. More often than not you'll require an actual HTTPContext, which requires quite a bit of work to setup. If you're making heavy use of custom HttpHandlers, you should be able to test those with quite a bit of ease.
On the other hand, testing your Data Access Layer is possible and I would recommend it. There may be better methods, but my approach has been to maintain all my CREATE Tables / CREATE Sprocs in text files along with my project, create a test database on the fly, and to use the Setup and Teardown methods to keep the database in a known state. The topic might be worth of a future blog post, but for now, I'll leave it up to your creativity.
Conclusion
Unit testing wasn't nearly as difficult as I first thought it was going to be. Sure my initial tests weren't the best – sometimes I would write near-meaningless test (like testing that a plain-old property was working as it should) and sometimes they were far too complex and well outside of a well-defined scope. But after my first project, I learnt a lot about what did and didn't work. One thing that immediately became clear was how much cleaner my code became. I quickly came to realize that if something was hard to test and I rewrote it to make it more testable, the entire code became more readable, better decoupled and overall easier to work with. The best advice I can give is to start small, experiment with a variety of techniques, don't be afraid to fail and learn from your mistakes. And of course, don't wait until your project is complete to unit test – write them as you go!