Basic Components of a Unit Test
About 7 years ago (okay okay, it was ooooooooooooonly 3 months ago) I wrote part 1 of a hopeful multi-part Introduction to Unit Testing. In that inaugural post I gave a high level over view of what Unit Testing is, the benefits and costs of using Unit Tests as part of your development process, and that you should just believe me and start writing them because they are good and I am smart and you are also smart and great minds think alike. In Part 2 I will define the different components of a Unit Test and how they work together. The technology I will use to illustrate my examples is NUnit – a free, open source unit test framework available for .NET.
The 50,000 Foot View
A unit test can be broken down into 5 parts:
- The System Under Test (SUT) – the particular chunk of code we wish to validate.
- The Testcase Class – The class that contains all the tests we will execute against the SUT
- The Test Method – The actual test
- The Test Fixture – The class responsible for preparing the System Under Test for testing
- The Test Runner – The utility responsible for actually running the test
Each component listed above has specific responsibilities and is relatively self contained. Together, however, they’re like the Voltron of Coding. They come together to create something beautiful and extremely powerful; something that destroys bugs and keeps your universe (domain) a safe and happy place for all travellers (programmers, project managers, marketting peeps, customers, and pointy haired bosses alike).
The System Under Test
The SUT is the class, feature, or some small snippet of code that you want to test. It is the thing that the Test Methods will validate to ensure all is working (or not working) as expected. This is the Universe that Voltron is protecting.
When I first started practicing Test-Driven Development I used to have a pretty strict mindset that the SUT was always the entire class that I was developing. I have found that, over time, this can create Really Really Big Testcase Classes and, just like “normal” classes, Really Really Big Testcase Classes are difficult to maintain and create a lot of headaches and tears of joy pain.
What I do now is I start going down this road, and then I refactor my tests as needed. For example, if I want my class to have the ability to save data to file, then I’ll develop a Testcase Class that tests only the saving features of my class. If that class also has a Load or Parse feature, than I’ll create a Testcase Class that tests only those features. In both these cases it is very likely that I can use the same Test Fixture, so my code reuse goes up, along with my productivity.
Test Fixtures
The Test Fixture is a class responsibile for setting up the SUT and initializing its state in preperation for testing. Sounds pretty vague, doesn’t it? I’ll help illustrate this with an example, because examples solve everything.
Over the past 5 months I’ve been working on a system that’s heavily file based. As such, the unit testing has been a real pain because when it comes to file based systems, it’s hard to keep the results of the previous test from impacting the results of the next outcome without doing a lot of file copy/move/etc. work. In TDD lingo, file-based systems are hard to test because they are Persistent Fixtures by nature (more on that in a future article). To help make it easier, I created a Test Fixture that several of my Test Classes use. This Test Fixture creates, on the fly, the directory and file structure that’s required in order to interact with the SUT, as well as the data-containing files. At the end of my test (during the Teardown phase), this directory and file structure is removed. The Test Fixture creates the directory and file structure from a templated directory and file structure at the start of each test, and destroys that structure at the end of each test. If the Test Fixture didn’t do this, then my tests would manipulate the original files, and subsequent tests would become unpredictable (and you haven’t experienced pain until you’ve experienced the pain of randomly failing tests).
Sidenote: The downfall to the above strategy is that, depending on the size of the files/directory structure that needs to be created, the time it takes each test ot complete is blown up big time, resulting in Slow Tests. Slow Tests are bad because they discourage you from running them. Try to keep the total time to run all your tests to under 30 seconds, if possible, and your life will be full of happiness and beautiful people.
But not only did my Test Fixture create the physical environment required for me to successfully interact with the SUT, it also initialized the various objects that interact with the physical environment. By putting all this (tedious) logic in the Test Fixture, my Testcase Class and Test Methods can concentrate on doing what they’re supposed to do: ensuring the SUT does what it’s (not) supposed to do. Also, it allows each Test Method to have its own Test Fixture, thus isolating the that Test Method from any other Test Method. Hot.
Testcase Classes
The Testcase Class is just a normal everyday class. There’s nothing special about it – it doesn’t hold magical properties or have any powers that are beyond the powers of any other class. The only special thing about a Testcase Class is that it is denoted as a Testcase Class. Some testing frameworks use a strict naming convention to denote a class as a Testcase Class. NUnit denotes a class as a Testcase Class by decorating the class with an unfortunately named Custom Attribute called “TestFixture” (located in the NUnit.Framework namespace).
Wait – A Test Fixture Is A Test Class?!
The above confused me for a great deal of time. The developers of NUnit assumed that each Testcase Class would be written for a Test Fixture. That is, a 1-to-1 relationship of Testcase Classes to Test Fixtures. This assumption starts to break down if you decide to create a Testcase Class per Class, or Testcase Class per Feature (such as my example above). In the real world, I’ve found that “A Testcase Class is a Test Fixture” can create difficult to maintain test classes, and fixture code responsible for preparing the System Under Test tends to be duplicated across multiple Testcase Classes and test suites. For the rest of this article and related articles – in fact, for the rest of your career – define the Testcase Class and Test Fixture as two completely different, unrelated entities and I promise you that your job, your family, your loved ones and all those around you will be better for it.
Test Methods
The Test Method is a good ol’ standard method that validates (or invalidates) something in the SUT. Writing a Test Method is almost more an art than a science, but typically the Test Method has 4 phases:
- Setup the System Under Test (using your Test Fixture, of course)
- Interact with the SUT
- Determine whether the expected outcome has been obtained
- “Teardown” the test fixture and put the universe back into it’s original state (kinda like a soft reset)
Test Methods should be as concise and precise as possible. Writing a Test Method called “ItWorks()” that checks a dozen different things is not a good approach, because when “ItWorks()” fails, you’ll have no idea what is failing, or why it’s failing. A Test Method should be clear and concise. Ideally you should be able to look at a failed test and determine what failed (and why) by just looking at its name alone.
The above implies that you have really good names for Test Methods. Naming is hard. Really hard. I find naming to be one of the hardest things to do in programming. When writing a Test Method I always think “Fact: [State the Fact about the SUT]“. The Fact (minus the word Fact) becomes the name for my Test Method.
For example, lets say I’m writing a Stack class and I’m currently writing the Pop method. I could either write a Test Method called “TestPop()”, or I could write a Test Method called “Pop_Removes_Item_From_Stack()”. When the “Pop_Removes_Item_From_Stack()” test fails (and it will fail), you’ll know exactly what is wrong, where as “TestPop()” will force you to read through n-lines of code, and fire up a debugger, and step through line by line, and cry.
But it doesn’t stop there – because we’ve written test explicitly called “Pop_Removes_Item_From_Stack()”, we must write other explict tests, such as “Pop_Throws_Invalid_Operation_Exception_When_Stack_Is_Empty()”, and other such explicit tests. Sure, the method names are not what we’re used to, but they serve a different purpose. In a sense, the test methods are documented examples of what the System Under Test does, not clear, concise, clever chunks of API that you’re exposing to other developers. Fear not the long method name!
Test Runners
The Test Runner is the thing that executes all the tests. Typically this is some piece of software that reflects over your Test Classes, finding Test Methods, and executing them one by one. Out of the box, NUnit comes with a really basic Test Runner (called NUnit-Gui) which gets the job done, but isn’t the greatest experience. Alternatives are the free Test Driven.NET, which supports a vast array of xUnit frameworks and supporting frameworks (such as mock frameworks, etc.) and plugs right into Visual Studio. The best solution that I’ve used, however, is the pay-for Unit Test Runner that’s part of JetBrains’ ReSharper which, if you are programming in Visual Studio, you absolutely postively have to have. It ain’t free, but it’s worth every single penny, and it’ll pay for itself in productivity boosts within a matter of days.
A Short and Sweet Summary
So, in essence, we have the following. A Testcase Class has lots of Test Methods. Each Test Method is an actual test performed against the System Under Test. In an ideal world, each Test Method will create the Test Fixture, interact with the SUT, validate the SUT does what it’s supposed to do (or not supposed to do, depending on your test), and then restore the world back to it’s original form. The Test Runner ignites the whole thing. Pretty simple, really.
But like most simple things, the idea is simple – actually putting it into practice in a maintainable, sustainable, and useful manner is hard. Hopefully Part 3 – which at this rate will get written some time after the 2010 Vancouver Olympics – can shed some light on how we can approach this.
Introduction to Unit Testing
The last couple months I’ve been doing a lot of unit testing at work. A lot. And I’ve been doing it more consistently than I have in the past. The result has been dramatic, not only in the quality of the code that I’m producing, but also in the confidence that I have when making changes to existing code. Even more importantly, however, is the impact that this dedication has had on my knowledge base, as I’ve had to do a lot of reading out of the necessity of creating maintainable, sustainable test suites. This article (and likely a few articles to come) is a breakdown of what I’ve learned so far. If you’ve learned something (or if you know something and you see me learning the wrong things) then I request that you, by all means, please share your knowledge with me and the rest of the world!
What Is a Unit Test?
In a nutshell, a unit test is a chunk of code that tests a small, specific chunk of production code (the System Under Test, or SUT). The Unit Test can assert that the SUT does something, does not do something, returns data (or no data), or exhibits some specific behaviour. Essentially, the Unit Test asserts that the SUT does what it’s supposed to do consistently and predictably.
There’s a subtle but important point in the above that I should emphasize: the SUT is a small, specific chunk of production code. People sometimes want to test bigger chunks of code, or multiple pieces of code (such as testing how a piece of data progresses from Point A through Point B and finally out of Point C). There’s nothing wrong with these types of tests, and indeed these tests (called Integration Tests) should, by all means, be written. But do not confuse them with unit tests, as unit tests are much, much smaller in scope.
Why is Unit Testing Beneficial?
There’s a number of benefits to unit testing, but the majority of them are not immediately noticeable. First and foremost, unit testing can help you detect and avoid defects in your code not only before you ship your code, but before you commit your code to whatever code repository you’re using (you are using source control, right?). On the outset this might not seem like a big deal if you’re writing a small, simple class; however if/when that class gets grows and grows and then gets refactored, those unit tests will be a God send by telling you what you just accidentally broke in the system. From a regression testing and quality standpoint Unit Tests are enormously beneficial.
Well written unit tests (and unit test suites) also act as documentation which illustrates what the SUT is and is not supposed to do, and also how the SUT is used. In essence, the tests are the specifications of the SUT. They act as examples, and these examples are significantly important to the people that will inherit your code after you’ve left your current employer (which, be honest, is inevitable). In fact, they’ll even help you when you return to that class 8 months after you wrote it because somebody in marketing wants your Duck class to be able to play the banjo. Having the ability to review your code, how it works, how a programmer should interact with it etc. all from a single location, without having to do a Find All across tens, hundreds or thousands of code files is pretty convenient. Enjoy The Convenience Of Knowing Where To Look! Yay Lazyness!
There are about a zillion other benefits, but I’ll only mention one more because it ties into a bigger concept of unit testing; under certain conditions unit tests can help you design better, leaner code; the practice of what’s called Test Driven Development. The idea behind TDD is that you define, via tests, what the SUT needs to do, and then you write the least amount of code possible in the SUT to pass the tests.
What Are Some Unit Testing Cons
Not all that glitters is gold, and while the benefits and advantages of unit testing are enormous, they do come at a price.
For starters, unit tests aren’t free. At the end of the day, a Unit Test is software, and that software has to get written by a developer, and developers cost money. If you’re going to use TDD or implement unit tests at a later date you will have to take that into account when making estimates. If your biggest goal is to hammer out as much code as possible in the absolute shortest amount of time possible, quality and scalability be damned!, than unit testing might not be for you. However, if you’re working on a long term project (either for yourself, your company, or somebody else’s company), the cost of adding on some time for a higher quality product is a no brainer (hint: do it).
Another con is one you’ll notice pretty quick: unit test organization (and unit tests themselves) can get complex. Just like production code is difficult to write in a sustainable and maintainable way, test code can be even more difficult. The principles and strategies that exist in writing “real world” code are still applicable to test code. You will experience the same heart-ache, frustration, and demoralization of poorly structured tests as you do with poorly structured production code.
Many people don’t take this into account when writing unit tests, and when things start to go awry and the various smells of unit tests start to rear their ugly head, it’s easy to just say “forget it” and drop unit testing because “it isn’t worth it.” But these test smells, much like code smells, can be avoided (or at least minimized) by learning from the mistakes of others. The book xUnit Design Patterns – Refactoring Test Code is the holy bible of unit testing. If you’re a beginner or experienced TDDer, this book should be on your shelf. However, if you’re like me, you’ll find the book is right next to your keyboard – there’s no sense putting it back on the shelf! I consult it daily!
Don’t treat unit tests as a second class citizen; just because they aren’t getting shipped doesn’t mean they take a back seat to basic OOPL principles. Be kind and thoughtful to your unit tests, and they will repay you like, ten fold, dude.
Just do Them!
Just like everyday production code, there can be many gotchas to unit testing; but they’re nothing a bit of reading, research, and careful design can’t take care of. The long term benefits of unit testing are enormous. The project that I’m currently working on has only been under development for 2 months, and we’re still learning the discipline of unit testing, but I already have 300 tests (77 of them are on a small, single component of the code). That’s 300 tests that will not only let me know when something no longer works, but if my tests are structured well, will tell me exactly where the something went wrong.
This article was a (very) brief introduction to Unit Testing. In my next article I’ll outline for you the different components of a unit testing framework, and hopefully from there we can start building some examples.
Stay tuned!
Do you have experience with TDD (or Unit Tests in general)? Share your knowledge and experience in the comments!
