That’s NOT Unit Testing

Jul 28, 2009

I’ve been rather quiet in the last couple of weeks.  Mostly I’ve been busy trying to learn WPF so you can expect to see a couple of blogs on that once I get the hang of it.  First thoughts on WPF:  it’s tricky.

This post was triggered by me attending yet another code evaluation session where some very corporate coders show off their horrible code.  The lead developer proudly revealed that all the code was covered by Unit Tests.  After having a look at the code I quickly realized that there were no Unit Tests to be found.  The tests were actually Integration Tests implemented with NUnit.

Signs that you’re not doing Unit Testing

Before I get into the underlying problem, here are a couple of indications that you’re not really doing Unit Testing (even though you may be using NUnit or another Unit Testing framework).

  1. You have database scripts that need to be run before you can run your Unit Tests.
  2. Your testing project has a config file.
  3. Unit Tests can only be run in a special ‘test environment’ – developers can’t run it on their machines.
  4. Unit Tests need to be run a specified order.

There are probably a few others – these are the most obvious ones that I could think of.  Here’s the most important point – just because you’re using NUnit doesn’t mean you’re doing Unit Testing.

There’s no clear rule on what the ‘Unit’ in Unit Testing is, but in general a Unit Test would test a single method on a single class.  If you’re accessing the database in your tests, you’re doing Integration Tests.  If you’re accessing any external component in your tests, you’re doing Integration Tests.

What’s the big deal?

So what does it matter if we’re doing Integration Tests or Unit Tests – it’s just a name, isn’t it?  Actually, it does matter – for 2 reasons.

  1. If you’re really doing Integration Tests it means that you don’t have any Unit Tests!  This probably indicates that your code is tightly coupled, difficult to test and breaking changes will not be detected.  This effectively means that we are back at square one – we can’t refactor or re-use code without the fear of introducing bugs. Your Unit Tests don’t work as a safety net for making code changes.
  2. If your Unit Tests depend heavily on external components you’re probably not running your Unit Tests as often as you should be.  In the best scenario you will only detect breaking changes before you commit changes or during your Continuous Integration (CI) build.  In the worst-case scenario (which is what I encountered in this scenario) you will only detect breaking changes once you deploy to a production/testing environment.

So what should we do?

This is not the first time that I have encountered this problem.  I think the most common cause here is inexperience with Unit Testing – the code is hacked together by a senior developer and junior developers are instructed to write tests to test the functionality.  This is often caused by senior developers not having experience with Unit Tests and considering it a general waste of time.

Don’t get me wrong – using Unit Tests and Integration Tests together is a great solution.  The only problem is when no Unit Tests are done at all.  Keep in mind that there are specific frameworks for implementing Integration Tests – Fitnesse, Selenium, WatiN, etc.  If you are going to use a Unit Testing framework for doing Integration Tests I would make a clear separation between the two – using separate projects or folders is a good idea.