Microsoft Yet Again Confounds Me In Ways I Never Would Have Predicted – A Trajedy in One Part
I know this will come as a surprise to some of you, but Microsoft Windows really is a terrible operating system. A joke. A bad car accident. The underage kid that’s getting drunk for the first time at a yuck-a-flucks party and is vomiting uncontrollably in the back corner of the hosts basement where the mess won’t be found for a few days. I discovered the most obscure, nonsensical bug in Windows XP today (for the record I’m running SP3). I’d be very curious to know if this bug exists in Vista – please chime in if you know.
What I Tried to Do
I’m in the middle of writing an application that’s heavily file based. As part of this application I have an archive of definition files that can be updated by the user whenever they want. When the application starts up it checks for a new and improved version of the archive and decompresses it. We’re not talking rocket science here… pretty straight forward and simple, hey? I thought so too… until my Unit Test was inexplicably failing… repeatedly… for hours.
I debugged and debugged… I read and I read… what the heck is wrong? The error made zero sense:
System.ArgumentException: FileStream will not open Win32 devices such as disk partitions and tape drives. Avoid use of “\\.\” in the path.
Sure, the message makes sense, but the context doesn’t, because I was passing an absolute path to an area deep inside the bowels of my local file system… not on a fileshare or UNC’d uri. No sir, mine was starting at the good ol’ c:\ and all was fine.
More debugging. More reading. More frustration. Less hair. Lots of tears.
A Revelation!
And then my good friend Janak showed me… THIS! Well well well, it just so happens that the file I was attempting to decompress was named “con.xsd”. Apparently that violates Windows’… err… guidelines/internal-workings/stupid-programmer-hack-fix.
Now, I can understand that perhaps “con” is a reserved filename for keyboard input… i can understand that Windows needs to reserve it… but does it need to reserve that name across an entire file system that spans hundreds of gigs? Am I the only one that thinks that it’s absolutely ridiculous to have blanket protection like that? It’s crazy! Compare it to, oh, I don’t know… POSIX’s guidelines for device files.
Microsoft – get your crap together and stop puking all over Peter’s new bath towels. Yeesh….
The Fall of Rome: The Microsoft Way
Recently Guy Kelsey of VendAsta fame and fortune posted a brief, but thought provoking article about how our company is going through an iPhone Epidemic. At least I found it to be fairly thought provoking considering I’m one of the people that shelled out the cashola to buy one. I’ve always been a fan of smart phones, and for the most part I’ve been a Microsoft faithful (there was that brief moment back around 2000 though when I bought a HandSpring). I was loyal to Microsoft’s form factor because those devices supported Microsoft’s Compact Framework (a mobile version of their full-on .NET Framework) which allowed me to have programmatic fun-time if I ever wanted, in an environment that I was already familiar with.
However, there are 2 things that have changed for me over the past couple years.
- As I grow as a software developer I am becoming more and more interested in non-MS technologies, especially open source technologies. .NET isn’t satisfying me anymore and I want to grow, and as I grow I want to be challenged to apply the software engineering techniques that I’ve learned.
- It has become unbearably obvious that Microsoft couldn’t innovate itself out of a wet paper bag.
At the risk of sounding cliche, it was Apple that woke me up and gave me the motivation to try out some new things.
In 2003 I purchased my first Apple product – an iPod. It was the 3rd generation, 30GB, and I was smitten. It was beautiful. It felt like I had picked it off a tree. It reaked of techno-hipster, and it glowed when I touched it. The iPod quickly became an integral part of my life. I was a walking iPod commercial, never leaving the house without the 2 long, white plugs in my ears denoting that I was part of the exclusive club (with about a billion other people), and I frequently played air drums while iPodding.
About a year later my computer was hit 3 times with some pretty severe worms (the first time because I was unprotected, the subsequent two times occured immediately after the first post-install boot of my computer, while Windows was downloading the latest service packs). In a string for 4 letter words I vowed I would never boot XP again, and within 24 hours I had dropped 1,600$ on an Apple iBook G4. Again, I was smitten. The sleak design, the ease of use, the physical sense of touch. Again, the whole thing felt organic, it felt natural, it felt like an extension of my life. And it integrated seamlessly with my first love, the iPod. After that I went on a multi-year binge that consisted of me selling off old iPods, buying the new ones, purchasing the smaller iPods so I could exercise with something light weight, etc.
I upgraded my iBook G4′s Panther OS to Tiger, and was again falling in love with what Apple was doing. Spotlight, Dashboard, Automator. Awesome tools that improved how users interact with the file system on their computer. Tools which extended the computer’s processing from localized files and information to a fully internet-integrated device on which you could quickly and easily look up words in a dictionary, check the weather in any city you wanted, check movie times, etc. – all without opening a browser.
During this time Microsoft was busy busy busy, designing and developing their new operating system Windows Vista. Vista promised to have a lot of interesting features, including a new 3D rendered user interface (which ended up being horrible to use), something called “WinFS” (essentially a database layered overtop of NTFS technology that’s going on 1,000 years old and was eventually stripped), PC-to-PC Synchronization (also stripped before release) and more security (which was implemented by constantly nagging you to deny or confirm any action that happens on your computer, ever – unfortunately not stripped). 6 years later, when it was finally released, so many of Vista’s “killer” features had been stripped that Vista was essentially just a new user interface that took serious hardware to run… and it cost 500$ for the full version. 500$. For an operating system. Seriously. I’m not even making this up.
In 2007, when Microsoft finally crapped out Vista upon the masses (with a feature set still years behind what Tiger included in 2005), Apple rolled out another OS called Leopard. Leopard sported a lot of cool innovative (and usable) new features, including easy data backup with Time Machine, and the ability to easily install Windows on your Mac (along side Leopard), and 300 other new features. And it cost 129$. A quarter of what Microsoft wanted for its new flagship. Let me repeat that – better features, built on a better foundation, for a quarter of the price.
Now I don’t mean to ridicule Microsoft here, and dishing on them is not the intention of this article. What I wanted to outline, however, was that while other companies were actually innovating, Microsoft just put a bunch of new lipstick on an old pig and called it Vista. There was no innovation, and what little innovation that did exist within Vista was innovation that your average user could care less about.
Now this is nothing new. Micorosoft hasn’t been known for top notch innovation – for the most part they’ve bought out some other software company, re-packaged their products and sold them as a new MS product. This is fine from a business perspective, and it’s okay when you don’t have a whole lot of competition, but that strategy will no longer work for a number of reasons.
- Love’em or hate’em, Apple is pumping out innovative products that are sleak, sexy, easy to use, and that people want. The iPhone, Apple’s first shot at the “smartphone” market, is a shining example of something that absolutely destroys anything Microsoft has tried to do even with their most recent line of Pocket PC form factor and Windows Mobile 6.
- A huge portion of the most innovative software technologies are coming out of the open source movement. This is bad news for a company that traditinoally buys out competitors, because when it comes to open source, there’s nobody to buy out.
- Other big players have emerged to dominate the areas that Microsoft ignored for too long. Google has successfully organized the Internet and is now raking in billions as a result. But Google is much more than just search, they’re huge in the web based email market with their GMail platform (and much like the iPhone destroying the Pocket PC, GMail destroyed Hotmail), their increasing acceptance with their Google Docs (a web-based competitor to Microsoft Office which is free and, dare I say, better for the majority of what most people do with a word processor)… then there’s Google Calendar, Google Reader, Google Earth, Google Photos, Google Video… essentially 99% of what you used to do on your desktop can now be done online, through Google, using nothing but a Web Browser – which is OS agnostic.
- The Open Source Software movement is being used, and uplifted, by the world’s biggest companies including IBM, Apple, and Google – where as Microsoft is still wanting to go in alone. The name of the game in today’s technology world is co-operation, but Microsoft is stubbonrly refusing to shift its gears.
- With the emergence of Google, Apple, and some seriously killer open source software frameworks (such as Django and Rails, to name only a couple), Microsoft is no longer alone in providing great tools to build great software, and as a result, developers are losing interest and leaving.
It’s point 5 that describes where I and several of my colleagues sit. Microsoft is slow to adopt emerging technologies, and as such a large portion of their developers are slow to adopt those emerging technologies. If you want to stay current with emerging technology chances are you’re going to have to walk away from Microsoft’s core suite and move towards Open Source and those companies that support Open Source. And this spells big trouble for Microsoft. One of the reasons why Windows remains the major operating system in the world is because there’s approximately 17 Billion programs that have been written for Windows. There’s a lot of software, and software developers, under the Windows umbrella. But when those developers start walking away to join Microsoft’s competition, it’s going to be bad news for the Redmond Giant.
In fact, we’ve already seen Microsoft trying to combat this by “opening up” the .NET Framework libraries (in the sense you get to see and debug their code, but you can’t make changes and re-distribute it). Microsoft also had a huge presence at this year’s OSCON. Microsoft is starting to feel the heat and they know they need to win over the “hearts and minds” of developers. But giving away free shirts and keynote speeches isn’t going to cut it; Microsoft has to start revolutionizing the computing experience for the consumer market, and they’re going to have to do it themselves – something they’ve never done before.
VendAsta Gets 3,000,000 Smackers in Funding!
The company I work for (and dare I say, am helping startup?) has successfully secured 3 million buckaroonies in funding for our new product, MyFrontSteps. That’s a whole lotta moola, and it’s gonna do a whole lotta good-a!
So, you might be asking, what are we going to use the money for? Hiring, Hiring, and more Hiring! We have a metric tonne of positions to fill, and it sure would be dandy like candy if you applied for one of them. So if you hate your job and you want to start at something new, that’s located in the heart of Saskatoon’s technology district, and you want to work with some incredicly smart, incredibly cool, incredibly fun people, then you should most definitely apply. Trust me, it’s a lot of fun and it’s only gonna get better!
Basic Components of a Unit Test
About 7 years ago (okay okay, it was ooooooooooooonly 3 months ago) I wrote part 1 of a hopeful multi-part Introduction to Unit Testing. In that inaugural post I gave a high level over view of what Unit Testing is, the benefits and costs of using Unit Tests as part of your development process, and that you should just believe me and start writing them because they are good and I am smart and you are also smart and great minds think alike. In Part 2 I will define the different components of a Unit Test and how they work together. The technology I will use to illustrate my examples is NUnit – a free, open source unit test framework available for .NET.
The 50,000 Foot View
A unit test can be broken down into 5 parts:
- The System Under Test (SUT) – the particular chunk of code we wish to validate.
- The Testcase Class – The class that contains all the tests we will execute against the SUT
- The Test Method – The actual test
- The Test Fixture – The class responsible for preparing the System Under Test for testing
- The Test Runner – The utility responsible for actually running the test
Each component listed above has specific responsibilities and is relatively self contained. Together, however, they’re like the Voltron of Coding. They come together to create something beautiful and extremely powerful; something that destroys bugs and keeps your universe (domain) a safe and happy place for all travellers (programmers, project managers, marketting peeps, customers, and pointy haired bosses alike).
The System Under Test
The SUT is the class, feature, or some small snippet of code that you want to test. It is the thing that the Test Methods will validate to ensure all is working (or not working) as expected. This is the Universe that Voltron is protecting.
When I first started practicing Test-Driven Development I used to have a pretty strict mindset that the SUT was always the entire class that I was developing. I have found that, over time, this can create Really Really Big Testcase Classes and, just like “normal” classes, Really Really Big Testcase Classes are difficult to maintain and create a lot of headaches and tears of joy pain.
What I do now is I start going down this road, and then I refactor my tests as needed. For example, if I want my class to have the ability to save data to file, then I’ll develop a Testcase Class that tests only the saving features of my class. If that class also has a Load or Parse feature, than I’ll create a Testcase Class that tests only those features. In both these cases it is very likely that I can use the same Test Fixture, so my code reuse goes up, along with my productivity.
Test Fixtures
The Test Fixture is a class responsibile for setting up the SUT and initializing its state in preperation for testing. Sounds pretty vague, doesn’t it? I’ll help illustrate this with an example, because examples solve everything.
Over the past 5 months I’ve been working on a system that’s heavily file based. As such, the unit testing has been a real pain because when it comes to file based systems, it’s hard to keep the results of the previous test from impacting the results of the next outcome without doing a lot of file copy/move/etc. work. In TDD lingo, file-based systems are hard to test because they are Persistent Fixtures by nature (more on that in a future article). To help make it easier, I created a Test Fixture that several of my Test Classes use. This Test Fixture creates, on the fly, the directory and file structure that’s required in order to interact with the SUT, as well as the data-containing files. At the end of my test (during the Teardown phase), this directory and file structure is removed. The Test Fixture creates the directory and file structure from a templated directory and file structure at the start of each test, and destroys that structure at the end of each test. If the Test Fixture didn’t do this, then my tests would manipulate the original files, and subsequent tests would become unpredictable (and you haven’t experienced pain until you’ve experienced the pain of randomly failing tests).
Sidenote: The downfall to the above strategy is that, depending on the size of the files/directory structure that needs to be created, the time it takes each test ot complete is blown up big time, resulting in Slow Tests. Slow Tests are bad because they discourage you from running them. Try to keep the total time to run all your tests to under 30 seconds, if possible, and your life will be full of happiness and beautiful people.
But not only did my Test Fixture create the physical environment required for me to successfully interact with the SUT, it also initialized the various objects that interact with the physical environment. By putting all this (tedious) logic in the Test Fixture, my Testcase Class and Test Methods can concentrate on doing what they’re supposed to do: ensuring the SUT does what it’s (not) supposed to do. Also, it allows each Test Method to have its own Test Fixture, thus isolating the that Test Method from any other Test Method. Hot.
Testcase Classes
The Testcase Class is just a normal everyday class. There’s nothing special about it – it doesn’t hold magical properties or have any powers that are beyond the powers of any other class. The only special thing about a Testcase Class is that it is denoted as a Testcase Class. Some testing frameworks use a strict naming convention to denote a class as a Testcase Class. NUnit denotes a class as a Testcase Class by decorating the class with an unfortunately named Custom Attribute called “TestFixture” (located in the NUnit.Framework namespace).
Wait – A Test Fixture Is A Test Class?!
The above confused me for a great deal of time. The developers of NUnit assumed that each Testcase Class would be written for a Test Fixture. That is, a 1-to-1 relationship of Testcase Classes to Test Fixtures. This assumption starts to break down if you decide to create a Testcase Class per Class, or Testcase Class per Feature (such as my example above). In the real world, I’ve found that “A Testcase Class is a Test Fixture” can create difficult to maintain test classes, and fixture code responsible for preparing the System Under Test tends to be duplicated across multiple Testcase Classes and test suites. For the rest of this article and related articles – in fact, for the rest of your career – define the Testcase Class and Test Fixture as two completely different, unrelated entities and I promise you that your job, your family, your loved ones and all those around you will be better for it.
Test Methods
The Test Method is a good ol’ standard method that validates (or invalidates) something in the SUT. Writing a Test Method is almost more an art than a science, but typically the Test Method has 4 phases:
- Setup the System Under Test (using your Test Fixture, of course)
- Interact with the SUT
- Determine whether the expected outcome has been obtained
- “Teardown” the test fixture and put the universe back into it’s original state (kinda like a soft reset)
Test Methods should be as concise and precise as possible. Writing a Test Method called “ItWorks()” that checks a dozen different things is not a good approach, because when “ItWorks()” fails, you’ll have no idea what is failing, or why it’s failing. A Test Method should be clear and concise. Ideally you should be able to look at a failed test and determine what failed (and why) by just looking at its name alone.
The above implies that you have really good names for Test Methods. Naming is hard. Really hard. I find naming to be one of the hardest things to do in programming. When writing a Test Method I always think “Fact: [State the Fact about the SUT]“. The Fact (minus the word Fact) becomes the name for my Test Method.
For example, lets say I’m writing a Stack class and I’m currently writing the Pop method. I could either write a Test Method called “TestPop()”, or I could write a Test Method called “Pop_Removes_Item_From_Stack()”. When the “Pop_Removes_Item_From_Stack()” test fails (and it will fail), you’ll know exactly what is wrong, where as “TestPop()” will force you to read through n-lines of code, and fire up a debugger, and step through line by line, and cry.
But it doesn’t stop there – because we’ve written test explicitly called “Pop_Removes_Item_From_Stack()”, we must write other explict tests, such as “Pop_Throws_Invalid_Operation_Exception_When_Stack_Is_Empty()”, and other such explicit tests. Sure, the method names are not what we’re used to, but they serve a different purpose. In a sense, the test methods are documented examples of what the System Under Test does, not clear, concise, clever chunks of API that you’re exposing to other developers. Fear not the long method name!
Test Runners
The Test Runner is the thing that executes all the tests. Typically this is some piece of software that reflects over your Test Classes, finding Test Methods, and executing them one by one. Out of the box, NUnit comes with a really basic Test Runner (called NUnit-Gui) which gets the job done, but isn’t the greatest experience. Alternatives are the free Test Driven.NET, which supports a vast array of xUnit frameworks and supporting frameworks (such as mock frameworks, etc.) and plugs right into Visual Studio. The best solution that I’ve used, however, is the pay-for Unit Test Runner that’s part of JetBrains’ ReSharper which, if you are programming in Visual Studio, you absolutely postively have to have. It ain’t free, but it’s worth every single penny, and it’ll pay for itself in productivity boosts within a matter of days.
A Short and Sweet Summary
So, in essence, we have the following. A Testcase Class has lots of Test Methods. Each Test Method is an actual test performed against the System Under Test. In an ideal world, each Test Method will create the Test Fixture, interact with the SUT, validate the SUT does what it’s supposed to do (or not supposed to do, depending on your test), and then restore the world back to it’s original form. The Test Runner ignites the whole thing. Pretty simple, really.
But like most simple things, the idea is simple – actually putting it into practice in a maintainable, sustainable, and useful manner is hard. Hopefully Part 3 – which at this rate will get written some time after the 2010 Vancouver Olympics – can shed some light on how we can approach this.
