So you want to do unit testing

What is a unit test? Wikipedia describes unit testing as testing individual units of code in isolation. If the code has external dependencies, you simulate the dependencies using mock objects.

For example, if I am testing code which gets data from a database, hopefully access to the database is via something like ODBC or JDBC. In which case, it is possible to use a fake database (file system or memory based) rather than say an Oracle or SQL Server driver.

If my database connection is hard coded to a particular machine or assumes the machine is localhost then my first step is to refactor the code to remove this dependency.

Part of the purpose of having unit test cases is so that we can safely change the code and know we didn't break any existing functionality. So if we need to modify the code to be able to add unit tests we have a bit of a Catch-22 situation. The truth of the matter is, if we have been changing the code without unit tests, changing it one more time in order to add unit tests is actually a step in the right direction and no worse than previous development.

Another important feature of unit tests are speed. If I am adding a new feature and I want to be sure it hasn't broken anything, I want to know as soon as possible. I don't want to write the feature, run the tests and check the results tomorrow. Ideally, I want to know in seconds. Realistically, I might have to live with minutes at first.

Test runs should be automated. If I have to make a change and figure out what tests to run, run them and check the results there is a strong chance I will stop running them. Especially if I'm on a tight timeline.

Ideally, I would check in my code. This will fire a trigger which builds my code (not the entire product, just my code) and run the unit tests against it. Putting such a build system in place is a great deal of work but worth the effort. Every minute it takes to create this build system should be weighed against how much time developers spend testing their code before they check in, how many minutes testers spend finding bugs, how much time developers take understanding the bug and fixing it. Numerous studies have shown fixing bugs is much more expensive than never introducing them in the first place.

So what do we need so far?

First, we need a unit test framework. You wouldn't create your own replacement for JDBC/ODBC. So why create your own unit test framework. There are plenty of them out there.

Second, we need mocking frameworks for the technologies we are utilizing. Which mock object frameworks you require depends on what you are using in your application. If it is a web application, you might need to mock out the web server. If it accesses a database, you will need to mock out the database.

Third, we need a build system to automate the running and reporting of the unit tests. Reporting the results is important too. Most systems will either report back to the source control client or send you an email. If the tests run in, literally, seconds, you can afford to reject the checkin if a unit test fails. If it takes more than say 5 seconds, you might want to send an email when a checkin fails.

Fourth, we need commitment from management and the team. If you don't believe there is benefit to unit testing there will be no benefit to unit testing. Training people on how to create good unit tests and maintain them is critical. If I'm starting a new project and writing tests from the beginning it is easy but the majority of you will be adding unit tests to existing code.

The first three things are relatively easy to obtain. There are plenty of technologies and examples of people using them. The fourth requirement is the biggest reason adopting unit testing fails. If you don’t get buy in from everyone involved it just won’t work. The developers need to understand this will benefit them in the long run. The testers need to understand that less testing will be required and they need to focus on things unit testing will not catch. There will always be plenty of things to test. So there should be no fear unit testing will replace  integration or system testing. Management has to understand if they cut timelines for a project, they will not give developers time to write the unit tests. If you reward the Project Manager for getting the project out on time, he will get the project out on time even if it means giving developers no time for unit test creation. As a Project Manager, if reducing the number of issues AFTER the project has shipped is not a metric I’m evaluated on, I’m happy to ship a product which will make the next project difficult to get out on time.

So, you have the tools and you have buy in from everyone. Now what? If you have 100,000+ lines of code, where do you start writing unit tests? The answer is actually really simple. For example piece of code a developer touches, they should add unit tests. Bug fixing is the best place to start. I would FIRST write a unit test which would have caught the bug. Then I’d fix the bug and see the unit test pass.

By focusing on unit tests for bug fixes it reduces the need for regression testing, it focuses on the features customers are using and the developers are in that code anyways. If we need to refactor the code to support unit testing, might as well happen as we are changing the code. The code was broken when  we started the bug fix. So we’ll have to manually test the fix without unit tests. Hopefully, with a unit test in place, it will be the last time we manually test changes to this code.

If we are modifying the code for feature creation, not bug fixing, we want to write unit tests to confirm the current behaviour. Once we have a test which passes with the current code, we can add the feature and the tests should continue to pass.

At this point we know what we need and where to start. So let’s cover some of the how to write a unit test.

First, a unit test is going to be a function/method which calls our code. We want the name of the unit test to reflect what it is testing. When results are published they will go out to the developer but they will also be seen by the backup developer, project management and various other people as well. If I got an email telling me test17() failed I’m going to have to open the code and read what test17() is testing. You added comments and kept them up to date, right? Or course you didn’t. The comments shouldn’t be necessary. The test name should tell me what it is doing. If the test method was called, callingForgotPasswordWhenNoEmailInUserPreferences() then we all know what is being tested.

Second, what failed? Most unit test frameworks has assert statements. There is the basic fail() call but there are also things like AssertTrue, AssertEquals, AssertNotNull, etc. They can be called with just what you are checking or with a message and what you are checking. You don’t want to code any more than you have to but enough that someone receiving the results will know what failed. If the requirement for my software is “When a user clicks the Forgot Password button but they have not set an email address in their preferences, they should be presented with a message telling them to contact the system administrator.” Then the result message from my example here might be something like, “callingForgotPasswordWhenNoEmailInUserPreferences() failed. Was expecting: ‘No email address was set for your account. Please contact the System Administrator.’ but received: ‘No email address.’”. From this is it pretty clear what was expected and what we received instead. Failing to tell the user how to proceed should be considered a show stopper for the customer. On the other hand, if the result was: “callingForgotPasswordWhenNoEmailInUserPreferences() failed. Was expecting: ‘No email address was set for your account. Please contact the System Administrator.’ but received: ‘No email address was set for your account. Please contact the system administrator.’” the customer might consider this acceptable. We might even update the unit test case to ignore case so the test becomes a pass.

Unit test frameworks are pretty well established now. The general structure of a unit test is:

• set up for the test
• run the test
• assert the test passed
• clean up so the next test starts at the same point

The set up would be things like creating mock objects, initializing the inputs for the test, etc. The running of the test would be a call to the method being testing. Next would be an assert statement confirming that we received the expected results or side effect. Finally, clean up (often called tear down) the environment so it is at the exact same condition it was before the set up occurred.

Often you will group similar tests in one test suite. If I have 12 tests and they all require the same set up I will put them all in one suite. The code will then have one setUp() method that creates the environment for each test, one method for each test (12 methods in total for this example) and one tearDown(). The setUp() method will create any mock objects, initial global variables, etc. The test method will create anything particular to that test, call the method being tested then make an assert call. The tearDown() method will then clean up the environment so it is just like it was before the setUp() method was called. This is important because most unit test frameworks do no guarantee the order the tests will be run. Assuming one test starts where a previous test left off is just bad practice. I have worked on a project with 45,000 unit tests. All test are run as part of the nightly build. Rather than running all the tests on one machine, they are distributed to 238 different machines. If they all ran on one machine they would take 378 hours (over 2 weeks) to run. By distributing them over 238 computers they run in approximately 3 hours. However, if test1932 depends on test1931 and the two tests get sent to different machines, test1932 will not run correctly. Each test must be independent of all other tests. This will not seem important at first but 1 year later you might find yourself needing weeks (possibly months) to refactor all your unit tests. Moments like these often cause management to abandon unit testing.

This is unit testing is a nutshell. I will warn you, ‘the devil is in the details.’ Hiring someone who has gone through the pains of setting up a unit test framework is always a good idea. Either find a good consultant or hire someone full time to work on the framework for you. Some unit test frameworks are jUnit for Java, cppUnit for C++, nUnit for .NET, etc. Gerard Meszaros has written an excellent book called “xUnit Test Patterns: Refactoring Test Code”. In it he talks about “Test Smells”. Essentially, you can sometimes look at a piece of code and say, “This code stinks.” A code or test ‘smell’ is an indicating that the code has problems, i.e. it stinks. I have found reading Gerard Meszaros book I know what to look for before I do it. Originally the book was designed for people who created unit tests, found the tests have issues, i.e. they ‘smell’ and are looking to fix them, i.e. refactor. By reading the book, I avoid creating the bad unit tests in the first place.

Good luck and have fun!