Basic Components of a Unit Test
About 7 years ago (okay okay, it was ooooooooooooonly 3 months ago) I wrote part 1 of a hopeful multi-part Introduction to Unit Testing. In that inaugural post I gave a high level over view of what Unit Testing is, the benefits and costs of using Unit Tests as part of your development process, and that you should just believe me and start writing them because they are good and I am smart and you are also smart and great minds think alike. In Part 2 I will define the different components of a Unit Test and how they work together. The technology I will use to illustrate my examples is NUnit – a free, open source unit test framework available for .NET.
The 50,000 Foot View
A unit test can be broken down into 5 parts:
- The System Under Test (SUT) – the particular chunk of code we wish to validate.
- The Testcase Class – The class that contains all the tests we will execute against the SUT
- The Test Method – The actual test
- The Test Fixture – The class responsible for preparing the System Under Test for testing
- The Test Runner – The utility responsible for actually running the test
Each component listed above has specific responsibilities and is relatively self contained. Together, however, they’re like the Voltron of Coding. They come together to create something beautiful and extremely powerful; something that destroys bugs and keeps your universe (domain) a safe and happy place for all travellers (programmers, project managers, marketting peeps, customers, and pointy haired bosses alike).
The System Under Test
The SUT is the class, feature, or some small snippet of code that you want to test. It is the thing that the Test Methods will validate to ensure all is working (or not working) as expected. This is the Universe that Voltron is protecting.
When I first started practicing Test-Driven Development I used to have a pretty strict mindset that the SUT was always the entire class that I was developing. I have found that, over time, this can create Really Really Big Testcase Classes and, just like “normal” classes, Really Really Big Testcase Classes are difficult to maintain and create a lot of headaches and tears of joy pain.
What I do now is I start going down this road, and then I refactor my tests as needed. For example, if I want my class to have the ability to save data to file, then I’ll develop a Testcase Class that tests only the saving features of my class. If that class also has a Load or Parse feature, than I’ll create a Testcase Class that tests only those features. In both these cases it is very likely that I can use the same Test Fixture, so my code reuse goes up, along with my productivity.
Test Fixtures
The Test Fixture is a class responsibile for setting up the SUT and initializing its state in preperation for testing. Sounds pretty vague, doesn’t it? I’ll help illustrate this with an example, because examples solve everything.
Over the past 5 months I’ve been working on a system that’s heavily file based. As such, the unit testing has been a real pain because when it comes to file based systems, it’s hard to keep the results of the previous test from impacting the results of the next outcome without doing a lot of file copy/move/etc. work. In TDD lingo, file-based systems are hard to test because they are Persistent Fixtures by nature (more on that in a future article). To help make it easier, I created a Test Fixture that several of my Test Classes use. This Test Fixture creates, on the fly, the directory and file structure that’s required in order to interact with the SUT, as well as the data-containing files. At the end of my test (during the Teardown phase), this directory and file structure is removed. The Test Fixture creates the directory and file structure from a templated directory and file structure at the start of each test, and destroys that structure at the end of each test. If the Test Fixture didn’t do this, then my tests would manipulate the original files, and subsequent tests would become unpredictable (and you haven’t experienced pain until you’ve experienced the pain of randomly failing tests).
Sidenote: The downfall to the above strategy is that, depending on the size of the files/directory structure that needs to be created, the time it takes each test ot complete is blown up big time, resulting in Slow Tests. Slow Tests are bad because they discourage you from running them. Try to keep the total time to run all your tests to under 30 seconds, if possible, and your life will be full of happiness and beautiful people.
But not only did my Test Fixture create the physical environment required for me to successfully interact with the SUT, it also initialized the various objects that interact with the physical environment. By putting all this (tedious) logic in the Test Fixture, my Testcase Class and Test Methods can concentrate on doing what they’re supposed to do: ensuring the SUT does what it’s (not) supposed to do. Also, it allows each Test Method to have its own Test Fixture, thus isolating the that Test Method from any other Test Method. Hot.
Testcase Classes
The Testcase Class is just a normal everyday class. There’s nothing special about it – it doesn’t hold magical properties or have any powers that are beyond the powers of any other class. The only special thing about a Testcase Class is that it is denoted as a Testcase Class. Some testing frameworks use a strict naming convention to denote a class as a Testcase Class. NUnit denotes a class as a Testcase Class by decorating the class with an unfortunately named Custom Attribute called “TestFixture” (located in the NUnit.Framework namespace).
Wait – A Test Fixture Is A Test Class?!
The above confused me for a great deal of time. The developers of NUnit assumed that each Testcase Class would be written for a Test Fixture. That is, a 1-to-1 relationship of Testcase Classes to Test Fixtures. This assumption starts to break down if you decide to create a Testcase Class per Class, or Testcase Class per Feature (such as my example above). In the real world, I’ve found that “A Testcase Class is a Test Fixture” can create difficult to maintain test classes, and fixture code responsible for preparing the System Under Test tends to be duplicated across multiple Testcase Classes and test suites. For the rest of this article and related articles – in fact, for the rest of your career – define the Testcase Class and Test Fixture as two completely different, unrelated entities and I promise you that your job, your family, your loved ones and all those around you will be better for it.
Test Methods
The Test Method is a good ol’ standard method that validates (or invalidates) something in the SUT. Writing a Test Method is almost more an art than a science, but typically the Test Method has 4 phases:
- Setup the System Under Test (using your Test Fixture, of course)
- Interact with the SUT
- Determine whether the expected outcome has been obtained
- “Teardown” the test fixture and put the universe back into it’s original state (kinda like a soft reset)
Test Methods should be as concise and precise as possible. Writing a Test Method called “ItWorks()” that checks a dozen different things is not a good approach, because when “ItWorks()” fails, you’ll have no idea what is failing, or why it’s failing. A Test Method should be clear and concise. Ideally you should be able to look at a failed test and determine what failed (and why) by just looking at its name alone.
The above implies that you have really good names for Test Methods. Naming is hard. Really hard. I find naming to be one of the hardest things to do in programming. When writing a Test Method I always think “Fact: [State the Fact about the SUT]“. The Fact (minus the word Fact) becomes the name for my Test Method.
For example, lets say I’m writing a Stack class and I’m currently writing the Pop method. I could either write a Test Method called “TestPop()”, or I could write a Test Method called “Pop_Removes_Item_From_Stack()”. When the “Pop_Removes_Item_From_Stack()” test fails (and it will fail), you’ll know exactly what is wrong, where as “TestPop()” will force you to read through n-lines of code, and fire up a debugger, and step through line by line, and cry.
But it doesn’t stop there – because we’ve written test explicitly called “Pop_Removes_Item_From_Stack()”, we must write other explict tests, such as “Pop_Throws_Invalid_Operation_Exception_When_Stack_Is_Empty()”, and other such explicit tests. Sure, the method names are not what we’re used to, but they serve a different purpose. In a sense, the test methods are documented examples of what the System Under Test does, not clear, concise, clever chunks of API that you’re exposing to other developers. Fear not the long method name!
Test Runners
The Test Runner is the thing that executes all the tests. Typically this is some piece of software that reflects over your Test Classes, finding Test Methods, and executing them one by one. Out of the box, NUnit comes with a really basic Test Runner (called NUnit-Gui) which gets the job done, but isn’t the greatest experience. Alternatives are the free Test Driven.NET, which supports a vast array of xUnit frameworks and supporting frameworks (such as mock frameworks, etc.) and plugs right into Visual Studio. The best solution that I’ve used, however, is the pay-for Unit Test Runner that’s part of JetBrains’ ReSharper which, if you are programming in Visual Studio, you absolutely postively have to have. It ain’t free, but it’s worth every single penny, and it’ll pay for itself in productivity boosts within a matter of days.
A Short and Sweet Summary
So, in essence, we have the following. A Testcase Class has lots of Test Methods. Each Test Method is an actual test performed against the System Under Test. In an ideal world, each Test Method will create the Test Fixture, interact with the SUT, validate the SUT does what it’s supposed to do (or not supposed to do, depending on your test), and then restore the world back to it’s original form. The Test Runner ignites the whole thing. Pretty simple, really.
But like most simple things, the idea is simple – actually putting it into practice in a maintainable, sustainable, and useful manner is hard. Hopefully Part 3 – which at this rate will get written some time after the 2010 Vancouver Olympics – can shed some light on how we can approach this.
